fix shuffle manager doc on ucx library path

[rongou]

Signed-off-by: Rong Ou rong.ou@gmail.com

The UCX 1.9.0 debian package puts libjucx.so in /usr/lib, which conflicts with the one in our jar file.

[abellina]

I can't reproduce this myself, with a system that has libjucx.so in /usr/lib with LD_LIBRARY_PATH set to /usr/lib:/usr/lib/ucx (which the part you removed should be there by default). I see us opening the libjucx.so that is bundled via the jucx jar. I am not sure why it is picking up the jucx lib from /usr/lib in your case, instead of the one we bundle in our dist jar.

If I force it to load, I do get a segfault (using LD_PRELOAD=/usr/lib/libjucx.so). It seems the libjucx.so for UCX 1.9 in the packaging is different than the one we bundle in our jar (seeing some symbols in the read-only data section that are missing from the /usr/lib one, but there may be other differences). Can you verify using LD_DEBUG=libs that the library in /usr/lib is the one that is getting loaded for you?

libjucx.so shouldn't be included in UCX packaging. I chatted with @petro-rudenko, and he opened a PR to remove libjucx.so from the packaging (openucx/ucx#6440). UCX 1.10 PR is pending. So starting ucx 1.10, libjucx.so is not going to be in /usr/lib.

In the doc we do say: set LD_LIBRARY_PATH if the UCX library is in a non-default location. In terms of this PR I think we should remove the LD_LIBRARY_PATH config provided, as that is just confusing. We could leave the bit after it where we make the user aware this needs native libraries and LD_LIBRARY_PATH may need to be set in some circumstances.

[rongou]

Removed the line.

[abellina]

build

[abellina]

Note that @petro-rudenko's PR went into 1.10: openucx/ucx#6441.