Skip to content

Memory leak in RSAKEY.createPrivateKey method #417

@jasonkatonica

Description

@jasonkatonica

By placing the test testRSAPlainKeySignatureWithPSS into a tight loop with many iterations it can be observed that the following data is being leaked as per a leak report:

STACK OF 89950 INSTANCES OF 'ROOT LEAK: <malloc in CRYPTO_zalloc>':
10  ???                                   0x15278736c 0x7fffffffffffffff + 9223372042533368685
9   libjgskit.dylib                       0x109517fd0 Java_com_ibm_crypto_plus_provider_ock_NativeInterface_RSAKEY_1createPrivateKey + 460
8   libicclib085.dylib                    0x113d72698 d2i_PrivateKey + 116
7   libicclib085.dylib                    0x113dfd230 old_rsa_priv_decode + 28
6   libicclib085.dylib                    0x113e19c34 ASN1_item_d2i + 68
5   libicclib085.dylib                    0x113e1a06c asn1_item_embed_d2i + 912
4   libicclib085.dylib                    0x113e1cb88 asn1_item_embed_new + 124
3   libicclib085.dylib                    0x113dfe0e8 rsa_cb + 40
2   libicclib085.dylib                    0x113e00000 RSA_new_method + 32
1   libicclib085.dylib                    0x113ddd558 CRYPTO_zalloc + 76
0   libsystem_malloc.dylib                0x193133820 _malloc_zone_malloc_instrumented_or_legacy + 264 
====
    1438869 (140M) << TOTAL >>
      ----
      16 (1.59K) ROOT LEAK: <malloc in CRYPTO_zalloc 0x1090b5710> [176]
         10 (928 bytes) <malloc in CRYPTO_zalloc 0x1090a5c30> [80]
            1 (208 bytes) <pthread_rwlock_t 0x1090b7940> [208]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090b78a0> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090b6fb0> [128]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090b78c0> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090c00f0> [128]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090b78e0> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090b7cd0> [128]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090b8550> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090adb70> [128]
         1 (208 bytes) <pthread_rwlock_t 0x1090b3730> [208]
         2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090b2ec0> [32]
            1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090b6440> [128]
         2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090b57c0> [32]
            1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090b8800> [128]
      ----
      16 (1.59K) ROOT LEAK: <malloc in CRYPTO_zalloc 0x1090ba390> [176]
         10 (928 bytes) <malloc in CRYPTO_zalloc 0x1090ba680> [80]
            1 (208 bytes) <pthread_rwlock_t 0x1090b73c0> [208]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090b7490> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090b74b0> [128]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090b7530> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090b7690> [128]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090b7550> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090b7710> [128]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090b7570> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090b7590> [128]
         1 (208 bytes) <pthread_rwlock_t 0x1090ba440> [208]
         2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090ba590> [32]
            1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090ba5b0> [128]
         2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090bd6d0> [32]
            1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090ba510> [128]
      ----
      16 (1.59K) ROOT LEAK: <malloc in CRYPTO_zalloc 0x1090bc580> [176]
         10 (928 bytes) <malloc in CRYPTO_zalloc 0x1090bffc0> [80]
            1 (208 bytes) <pthread_rwlock_t 0x1090bc900> [208]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090b9540> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090bcb90> [128]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090bc870> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090bc9d0> [128]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090bca50> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090bce30> [128]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090bca70> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090bca90> [128]
         1 (208 bytes) <pthread_rwlock_t 0x1090bc630> [208]
         2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090bc780> [32]
            1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090bc7a0> [128]
         2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090be900> [32]
            1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090bc700> [128]
      ----
      16 (1.59K) ROOT LEAK: <malloc in CRYPTO_zalloc 0x1090bd2e0> [176]
         10 (928 bytes) <malloc in CRYPTO_zalloc 0x1090baee0> [80]
            1 (208 bytes) <pthread_rwlock_t 0x1090baf30> [208]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090bb080> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090bb1e0> [128]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090bb0a0> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090bb160> [128]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090bb0c0> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090bb0e0> [128]
            2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090bcd40> [32]
               1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090bb000> [128]
         1 (208 bytes) <pthread_rwlock_t 0x1090bd390> [208]
         2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090ac480> [32]
            1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090bd460> [128]
         2 (160 bytes) <malloc in CRYPTO_zalloc 0x1090b8d00> [32]
            1 (128 bytes) <malloc in CRYPTO_zalloc 0x1090bd4e0> [128]
      ----
...
...
ADDITONAL ENTRIES SIMILAR TO ABOVE REDACTED
...

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions