BUG: RTLD_GLOBAL dlopen flag causes segfaults w/ pynacl & libsodium

[dwoz]

What pyzmq version?

25.1.0

What libzmq version?

4.3.4

Python version (and how it was installed)

Python 3.10.9

OS

Linux

What happened?

When pyzmq is built without bundling libsodium, tweetnacl is bundled. Pyzmq performs a dlopen of libzmq with the RTLD_GLOBAL which causes tweetnacl's functions to resolved rather than libsodium's functions. This will cause segfaults if using PyNaCl with pyzmq.

Code to reproduce bug

virtualenv venv
source venv
# When no libsodium is present, libzmq bundles tweetnacl
pip install --no-binary=':all:' pyzmq

nacl_test.py

import zmq
from nacl.public import PublicKey, PrivateKey, SealedBox
# Generate Bob's private key, as we've done in the Box example
skbob = PrivateKey.generate()
pkbob = skbob.public_key
# Alice wishes to send a encrypted message to Bob,
# but prefers the message to be untraceable
sealed_box = SealedBox(pkbob)
# This is Alice's message
message =  b"Kill all kittens"
# Encrypt the message, it will carry the ephemeral key public part
# to let Bob decrypt it
encrypted = sealed_box.encrypt(message)
unseal_box = SealedBox(skbob)
# decrypt the received message
plaintext = unseal_box.decrypt(encrypted)
print('done')

Running nacl_test.py will result in a segfault:

$ python nacl_test.py 
Segmentation fault

The segfault is called because libsodium's crypto_secretbox_detached calls crypto_core_hsalsa20 bit it's tweetnacl's version of crypto_core_hsalsa20 instead of libsodium's version.

### Traceback, if applicable

```shell
38 bt
 39 #0  0x00007f568e191b8f in raise () from /lib64/libpthread.so.0
 40 #1  <signal handler called>
 41 #2  0x00007f5685953ab5 in core (out=out@entry=0x7ffc12448270 "\204\202\324ф\326\a\225\247\362\235\203\331\060\251V \204D\022\374\177", in=in@entry=0x7ffc124484c0 "\273\003\241    \302ֽ\025\312NK\265\274\030\020\371\231Ou\313]:\330\"\250GvO",
 42     k=k@entry=0x7ffc12448420 "\227\247V\363\246\355\364I2\277\346!\030ב\213\n9\306w\232\002\301\366 t\314u/C\351s)\267", c=c@entry=0x0, h=h@entry=1) at bundled/zeromq/src/tweetnacl.c:116
 43 #3  0x00007f568595638f in crypto_core_hsalsa20 (out=out@entry=0x7ffc12448270 "\204\202\324ф\326\a\225\247\362\235\203\331\060\251V \204D\022\374\177",
 44     in=in@entry=0x7ffc124484c0 "\273\003\241\302ֽ\025\312NK\265\274\030\020\371\231Ou\313]:\330\"\250GvO", k=k@entry=0x7ffc12448420 "\227\247V\363\246\355\364I2\277\346!\030ב2    13\n9\306w\232\002\301\366 t\314u/C\351s)\267", c=c@entry=0x0)
 45     at bundled/zeromq/src/tweetnacl.c:163
 46 #4  0x00007f56823a445c in crypto_secretbox_detached (c=c@entry=0x32c2240 "", mac=mac@entry=0x32c2230 "", m=m@entry=0x7f56878eadd0 "Kill all kittens", mlen=mlen@entry=16,
 47     n=n@entry=0x7ffc124484c0 "\273\003\241\302ֽ\025\312NK\265\274\030\020\371\231Ou\313]:\330\"\250GvO", k=k@entry=0x7ffc12448420 "\227\247V\363\246\355\364I2\277\346!\030ב\213    \n9\306w\232\002\301\366 t\314u/C\351s)\267")

More info

To work around the issue you can make sure to import nacl before pyzmq.. That'll allow nacl to work properly, however I'm un-suer if it would have any adverse effects on pyzmq with curve support. I suspect it may.
Another work around is to make sure libsodium's dlopen call uses the RTLD_DEEPBIND flag. This could also have adverse effects.

[minrk]

Thanks for the report! It's unclear what, if anything, pyzmq should do to address this. Seems like some kind of name mangling or something, but I wouldn't know how to go about it.

Another work around is to make sure libsodium's dlopen call uses the RTLD_DEEPBIND flag

Do you mean libzmq? I don't think pyzmq can do anything about how libsodium is loaded when it's not bundled.

[dwoz]

@minrk Sorry for the confusion. The issue we ran into is using PyNaCl with pyzmq when pyzmq does not have libsodium bundled. In this scenario importing pyzmq makes tweetnacl's functions global. Then when using PyNaCl, calls to libsodium result in functions from tweetnacl get called which causes segfaults.

The proposed workaround was setting RTLD_DEEPBIND when importing the libsodium provided by PyNaCl.

Does that make sense?

[minrk]

I think it does, but I don't quite understand what I can do about it in pyzmq (or those building pyzmq for pynacl), since it's when pynacl loads the symbol. I wonder if some amount of symbol name mangling in pyzmq's bundled libzmq would help avoid the collision.

Would it perhaps be better to build pyzmq linked against libzmq and libsodium, and then bundle libzmq but not libsodium, so the shared libsodium ought to be used?

This is a level of linking and loading I'm not experienced with, so I'm not sure what I can do about these things.

[minrk]

tweetnacl is now gone, and all pyzmq builds actually use libsodium (test with pyzmq v26 prereleases). Does that help? Sorry, I still don't understand the details of this issue, but it if tweetnacl was the problem, it may not come up anymore.