[Question] Issues building cuquantum-python from source #95

Sinestro38 · 2023-10-18T18:48:49Z

Sinestro38
Oct 18, 2023

I have the cuquantum-23.06 appliance container mounted with a volume containing my project directories and I'm trying to build cuquantum-python from source (specifically using cutensornet) but I run into errors following the given instructions.

First I pull the container with my mount docker run --gpus all -it --rm -v /home/ubuntu:/home/ubuntu -e HOME=/home/ubuntu nvcr.io/nvidia/cuquantum-appliance:23.06
Clone the repo: git clone https://github.com/NVIDIA/cuQuantum
Set CUDA_PATH: export CUDA_PATH=/usr/local/cuda
CD into the pkg lvl dir cd cuQuantum/python
Run pip install: pip install -e .

But returns this error saying it couldn't find cutensornet or custatevec:

    g++ -pthread -B /home/cuquantum/conda/envs/cuquantum-23.06/compiler_compat -shared -Wl,--allow-shlib-undefined -Wl,-rpath,/home/cuquantum/conda/envs/cuquantum-23.06/lib -Wl,-rpath-link,/home/cuquantum/conda/envs/cuquantum-23.06/lib -L/home/cuquantum/conda/envs/cuquantum-23.06/lib -Wl,--allow-shlib-undefined -Wl,-rpath,/home/cuquantum/conda/envs/cuquantum-23.06/lib -Wl,-rpath-link,/home/cuquantum/conda/envs/cuquantum-23.06/lib -L/home/cuquantum/conda/envs/cuquantum-23.06/lib build/temp.linux-x86_64-cpython-39/cuquantum/cutensornet/cutensornet.o -L/tmp/pip-build-env-h7eauy9b/normal/lib/python3.9/site-packages/cuquantum/lib -L/tmp/pip-build-env-h7eauy9b/normal/lib/python3.9/site-packages/cuquantum/lib64 -lcutensornet -o build/lib.linux-x86_64-cpython-39/cuquantum/cutensornet/cutensornet.cpython-39-x86_64-linux-gnu.so
    /home/cuquantum/conda/envs/cuquantum-23.06/compiler_compat/ld: cannot find -lcutensornet: No such file or directory
    collect2: error: ld returned 1 exit status
    g++ -pthread -B /home/cuquantum/conda/envs/cuquantum-23.06/compiler_compat -shared -Wl,--allow-shlib-undefined -Wl,-rpath,/home/cuquantum/conda/envs/cuquantum-23.06/lib -Wl,-rpath-link,/home/cuquantum/conda/envs/cuquantum-23.06/lib -L/home/cuquantum/conda/envs/cuquantum-23.06/lib -Wl,--allow-shlib-undefined -Wl,-rpath,/home/cuquantum/conda/envs/cuquantum-23.06/lib -Wl,-rpath-link,/home/cuquantum/conda/envs/cuquantum-23.06/lib -L/home/cuquantum/conda/envs/cuquantum-23.06/lib build/temp.linux-x86_64-cpython-39/cuquantum/custatevec/custatevec.o -L/tmp/pip-build-env-h7eauy9b/normal/lib/python3.9/site-packages/cuquantum/lib -L/tmp/pip-build-env-h7eauy9b/normal/lib/python3.9/site-packages/cuquantum/lib64 -lcustatevec -o build/lib.linux-x86_64-cpython-39/cuquantum/custatevec/custatevec.cpython-39-x86_64-linux-gnu.so
    /home/cuquantum/conda/envs/cuquantum-23.06/compiler_compat/ld: cannot find -lcustatevec: No such file or directory
    collect2: error: ld returned 1 exit status
    error: command '/usr/bin/g++' failed with exit code 1
    [end of output]

Shouldn't they be built and installed in the process of running this command?

Answered by leofang

Oct 18, 2023

Yes, if you go all manually, you'll need to follow the instruction to do so, e.g.

But if you do pip install ., all these would be taken care for you (and most likely pip would reuse the already existing wheels cached somewhere in the container) 🙂

View full answer

leofang · 2023-10-18T18:54:27Z

leofang
Oct 18, 2023
Maintainer

Hi @Sinestro38 May I first ask what's the purpose of using our container and building cuquantum-python inside? Note that it's already installed in the container (so that python -c "import cuquantum" just works).

8 replies

Sinestro38 Oct 18, 2023
Author

Here is the full output:

Using pip 23.2.1 from /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/pip (python 3.9)
Obtaining file:///home/ubuntu/cuQuantum/python
  Running command pip subprocess to install build dependencies
  Collecting Cython<3,>=0.29.22
    Obtaining dependency information for Cython<3,>=0.29.22 from https://files.pythonhosted.org/packages/35/cb/24999b0f2774ddc498d8e885f0b5f0441e008e1c100b7810498913b5bf55/Cython-0.29.36-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata
    Using cached Cython-0.29.36-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (3.1 kB)
  Collecting packaging
    Obtaining dependency information for packaging from https://files.pythonhosted.org/packages/ec/1a/610693ac4ee14fcdf2d9bf3c493370e4f2ef7ae2e19217d7a237ff42367d/packaging-23.2-py3-none-any.whl.metadata
    Using cached packaging-23.2-py3-none-any.whl.metadata (3.2 kB)
  Collecting setuptools>=61.0.0
    Obtaining dependency information for setuptools>=61.0.0 from https://files.pythonhosted.org/packages/bb/26/7945080113158354380a12ce26873dd6c1ebd88d47f5bc24e2c5bb38c16a/setuptools-68.2.2-py3-none-any.whl.metadata
    Using cached setuptools-68.2.2-py3-none-any.whl.metadata (6.3 kB)
  Collecting wheel
    Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/b8/8b/31273bf66016be6ad22bb7345c37ff350276cfd46e389a0c2ac5da9d9073/wheel-0.41.2-py3-none-any.whl.metadata
    Using cached wheel-0.41.2-py3-none-any.whl.metadata (2.2 kB)
  Using cached Cython-0.29.36-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (1.9 MB)
  Using cached packaging-23.2-py3-none-any.whl (53 kB)
  Using cached setuptools-68.2.2-py3-none-any.whl (807 kB)
  Using cached wheel-0.41.2-py3-none-any.whl (64 kB)
  Installing collected packages: wheel, setuptools, packaging, Cython
  Successfully installed Cython-0.29.36 packaging-23.2 setuptools-68.2.2 wheel-0.41.2
  Installing build dependencies ... done
  Running command Checking if build backend supports build_editable
  Checking if build backend supports build_editable ... done
  Running command Getting requirements to build wheel
  warning: cuquantum/custatevec/custatevec.pxd:111:8: 'CUSTATEVEC_VER_MAJOR' redeclared
  warning: cuquantum/custatevec/custatevec.pxd:112:8: 'CUSTATEVEC_VER_MINOR' redeclared
  warning: cuquantum/custatevec/custatevec.pxd:113:8: 'CUSTATEVEC_VER_PATCH' redeclared
  warning: cuquantum/custatevec/custatevec.pxd:114:8: 'CUSTATEVEC_VERSION' redeclared
  warning: cuquantum/custatevec/custatevec.pxd:115:8: 'CUSTATEVEC_ALLOCATOR_NAME_LEN' redeclared
  warning: cuquantum/custatevec/custatevec.pxd:116:8: 'CUSTATEVEC_MAX_SEGMENT_MASK_SIZE' redeclared
  Compiling cuquantum/custatevec/custatevec.pyx because it depends on /tmp/pip-build-env-5hkeejk4/overlay/lib/python3.9/site-packages/Cython/Includes/libcpp/vector.pxd.
  Compiling cuquantum/cutensornet/cutensornet.pyx because it depends on /tmp/pip-build-env-5hkeejk4/overlay/lib/python3.9/site-packages/Cython/Includes/libcpp/vector.pxd.
  Compiling cuquantum/utils.pyx because it depends on /tmp/pip-build-env-5hkeejk4/overlay/lib/python3.9/site-packages/Cython/Includes/libc/string.pxd.
  [1/3] Cythonizing cuquantum/custatevec/custatevec.pyx
  [2/3] Cythonizing cuquantum/cutensornet/cutensornet.pyx
  [3/3] Cythonizing cuquantum/utils.pyx
  /tmp/pip-build-env-5hkeejk4/overlay/lib/python3.9/site-packages/setuptools/dist.py:314: InformationOnly: Normalizing '23.06.0' to '23.6.0'
    self.metadata.version = self._normalize_version(
  running egg_info
  writing cuquantum_python_cu11.egg-info/PKG-INFO
  writing dependency_links to cuquantum_python_cu11.egg-info/dependency_links.txt
  writing requirements to cuquantum_python_cu11.egg-info/requires.txt
  writing top-level names to cuquantum_python_cu11.egg-info/top_level.txt
  dependency /usr/local/cuda/include/driver_types.h won't be automatically included in the manifest: the path must be relative
  dependency /usr/local/cuda/include/library_types.h won't be automatically included in the manifest: the path must be relative
  dependency /usr/local/cuda/include/vector_types.h won't be automatically included in the manifest: the path must be relative
  reading manifest file 'cuquantum_python_cu11.egg-info/SOURCES.txt'
  adding license file 'LICENSE'
  writing manifest file 'cuquantum_python_cu11.egg-info/SOURCES.txt'
  Getting requirements to build wheel ... done
  Running command pip subprocess to install backend dependencies
  Collecting custatevec-cu11~=1.4
    Obtaining dependency information for custatevec-cu11~=1.4 from https://files.pythonhosted.org/packages/c8/e6/0693fd010797710e6f046d301d3abfe6bd6d97ba1fda35f960fccd7e08ba/custatevec_cu11-1.4.1-py3-none-manylinux2014_x86_64.whl.metadata
    Using cached custatevec_cu11-1.4.1-py3-none-manylinux2014_x86_64.whl.metadata (2.1 kB)
  Collecting cutensornet-cu11~=2.2
    Obtaining dependency information for cutensornet-cu11~=2.2 from https://files.pythonhosted.org/packages/28/f8/2509d4ec49e0771a2feb1e0964499576fe9408392032ec5ba1f510d48c28/cutensornet_cu11-2.2.1-py3-none-manylinux2014_x86_64.whl.metadata
    Using cached cutensornet_cu11-2.2.1-py3-none-manylinux2014_x86_64.whl.metadata (2.1 kB)
  Collecting cutensor-cu11<2,>=1.6.1 (from cutensornet-cu11~=2.2)
    Using cached cutensor_cu11-1.7.0-py3-none-manylinux2014_x86_64.whl (142.1 MB)
  Using cached custatevec_cu11-1.4.1-py3-none-manylinux2014_x86_64.whl (38.9 MB)
  Using cached cutensornet_cu11-2.2.1-py3-none-manylinux2014_x86_64.whl (2.2 MB)
  Installing collected packages: cutensor-cu11, custatevec-cu11, cutensornet-cu11
  Successfully installed custatevec-cu11-1.4.1 cutensor-cu11-1.7.0 cutensornet-cu11-2.2.1
  Installing backend dependencies ... done
  Running command Preparing metadata (pyproject.toml)
  /tmp/pip-build-env-5hkeejk4/overlay/lib/python3.9/site-packages/setuptools/dist.py:314: InformationOnly: Normalizing '23.06.0' to '23.6.0'
    self.metadata.version = self._normalize_version(
  running dist_info
  creating /tmp/pip-modern-metadata-xpu1muab/cuquantum_python_cu11.egg-info
  writing /tmp/pip-modern-metadata-xpu1muab/cuquantum_python_cu11.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-modern-metadata-xpu1muab/cuquantum_python_cu11.egg-info/dependency_links.txt
  writing requirements to /tmp/pip-modern-metadata-xpu1muab/cuquantum_python_cu11.egg-info/requires.txt
  writing top-level names to /tmp/pip-modern-metadata-xpu1muab/cuquantum_python_cu11.egg-info/top_level.txt
  writing manifest file '/tmp/pip-modern-metadata-xpu1muab/cuquantum_python_cu11.egg-info/SOURCES.txt'
  dependency /usr/local/cuda/include/driver_types.h won't be automatically included in the manifest: the path must be relative
  dependency /usr/local/cuda/include/library_types.h won't be automatically included in the manifest: the path must be relative
  dependency /usr/local/cuda/include/vector_types.h won't be automatically included in the manifest: the path must be relative
  reading manifest file '/tmp/pip-modern-metadata-xpu1muab/cuquantum_python_cu11.egg-info/SOURCES.txt'
  adding license file 'LICENSE'
  writing manifest file '/tmp/pip-modern-metadata-xpu1muab/cuquantum_python_cu11.egg-info/SOURCES.txt'
  creating '/tmp/pip-modern-metadata-xpu1muab/cuquantum_python_cu11-23.6.0.dist-info'
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy>=1.21 in /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages (from cuquantum-python-cu11==23.6.0) (1.23.5)
Requirement already satisfied: custatevec-cu11~=1.4 in /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages (from cuquantum-python-cu11==23.6.0) (1.4.1)
Requirement already satisfied: cutensornet-cu11~=2.2 in /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages (from cuquantum-python-cu11==23.6.0) (2.2.1)
Requirement already satisfied: cutensor-cu11<2,>=1.6.1 in /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages (from cutensornet-cu11~=2.2->cuquantum-python-cu11==23.6.0) (1.7.0)
Installing collected packages: cuquantum-python-cu11
  Attempting uninstall: cuquantum-python-cu11
    Found existing installation: cuquantum-python-cu11 23.6.0
    Uninstalling cuquantum-python-cu11-23.6.0:
      Removing file or directory /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/__init__.pxd
      Removing file or directory /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/__init__.py
      Removing file or directory /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/__main__.py
      Removing file or directory /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/__pycache__/
      Removing file or directory /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/_version.py
      Removing file or directory /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/custatevec/
      Removing file or directory /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/cutensornet/
      Removing file or directory /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/utils.cpython-39-x86_64-linux-gnu.so
      Removing file or directory /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/utils.pxd
      Removing file or directory /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/utils.pyx
      Removing file or directory /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum_python_cu11-23.6.0.dist-info/
      Successfully uninstalled cuquantum-python-cu11-23.6.0
  Running setup.py develop for cuquantum-python-cu11
    Running command python setup.py develop
    /tmp/pip-build-env-5hkeejk4/overlay/lib/python3.9/site-packages/setuptools/dist.py:314: InformationOnly: Normalizing '23.06.0' to '23.6.0'
      self.metadata.version = self._normalize_version(
    running develop
    /tmp/pip-build-env-5hkeejk4/overlay/lib/python3.9/site-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://github.com/pypa/setuptools/issues/917 for details.
            ********************************************************************************

    !!
      easy_install.initialize_options(self)
    /tmp/pip-build-env-5hkeejk4/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************

    !!
      self.initialize_options()
    running egg_info
    writing cuquantum_python_cu11.egg-info/PKG-INFO
    writing dependency_links to cuquantum_python_cu11.egg-info/dependency_links.txt
    writing requirements to cuquantum_python_cu11.egg-info/requires.txt
    writing top-level names to cuquantum_python_cu11.egg-info/top_level.txt
    dependency /usr/local/cuda/include/driver_types.h won't be automatically included in the manifest: the path must be relative
    dependency /usr/local/cuda/include/library_types.h won't be automatically included in the manifest: the path must be relative
    dependency /usr/local/cuda/include/vector_types.h won't be automatically included in the manifest: the path must be relative
    reading manifest file 'cuquantum_python_cu11.egg-info/SOURCES.txt'
    adding license file 'LICENSE'
    writing manifest file 'cuquantum_python_cu11.egg-info/SOURCES.txt'
    running build_ext

    ********************************************************************************
    CUDA version: 11.8
    CUDA path: /usr/local/cuda
    cuStateVec path: /tmp/pip-build-env-5hkeejk4/normal/lib/python3.9/site-packages/cuquantum
    cuTensorNet path: /tmp/pip-build-env-5hkeejk4/normal/lib/python3.9/site-packages/cuquantum
    ********************************************************************************

    building 'cuquantum.custatevec.custatevec' extension
    gcc -pthread -B /home/cuquantum/conda/envs/cuquantum-23.06/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/cuquantum/conda/envs/cuquantum-23.06/include -fPIC -O2 -isystem /home/cuquantum/conda/envs/cuquantum-23.06/include -fPIC -I/usr/local/cuda/include -I/tmp/pip-build-env-5hkeejk4/normal/lib/python3.9/site-packages/cuquantum/include -I/home/cuquantum/conda/envs/cuquantum-23.06/include/python3.9 -c cuquantum/custatevec/custatevec.cpp -o build/temp.linux-x86_64-cpython-39/cuquantum/custatevec/custatevec.o
    building 'cuquantum.cutensornet.cutensornet' extension
    building 'cuquantum.utils' extension
    gcc -pthread -B /home/cuquantum/conda/envs/cuquantum-23.06/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/cuquantum/conda/envs/cuquantum-23.06/include -fPIC -O2 -isystem /home/cuquantum/conda/envs/cuquantum-23.06/include -fPIC -I/usr/local/cuda/include -I/tmp/pip-build-env-5hkeejk4/normal/lib/python3.9/site-packages/cuquantum/include -I/home/cuquantum/conda/envs/cuquantum-23.06/include/python3.9 -c cuquantum/cutensornet/cutensornet.cpp -o build/temp.linux-x86_64-cpython-39/cuquantum/cutensornet/cutensornet.o
    gcc -pthread -B /home/cuquantum/conda/envs/cuquantum-23.06/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/cuquantum/conda/envs/cuquantum-23.06/include -fPIC -O2 -isystem /home/cuquantum/conda/envs/cuquantum-23.06/include -fPIC -I/usr/local/cuda/include -I/home/cuquantum/conda/envs/cuquantum-23.06/include/python3.9 -c cuquantum/utils.c -o build/temp.linux-x86_64-cpython-39/cuquantum/utils.o
    cuquantum/cutensornet/cutensornet.cpp: In function ‘PyObject* __pyx_f_9cuquantum_11cutensornet_11cutensornet_get_output_tensor_details(intptr_t, intptr_t, int)’:
    cuquantum/cutensornet/cutensornet.cpp:5731:196: warning: ‘cutensornetStatus_t cutensornetGetOutputTensorDetails(cutensornetHandle_t, cutensornetNetworkDescriptor_t, int32_t*, size_t*, int32_t*, int64_t*, int64_t*)’ is deprecated: please use cutensornetGetOutputTensorDescriptor instead [-Wdeprecated-declarations]
     5731 |         __pyx_v_status = cutensornetGetOutputTensorDetails(((cutensornetHandle_t)__pyx_v_handle), ((cutensornetNetworkDescriptor_t)__pyx_v_tn_desc), (&__pyx_v_numModesOut), NULL, NULL, NULL, NULL);
          |                                                                                                                                                                                                    ^
    In file included from cuquantum/cutensornet/cutensornet.cpp:785:
    /tmp/pip-build-env-5hkeejk4/normal/lib/python3.9/site-packages/cuquantum/include/cutensornet.h:245:21: note: declared here
      245 | cutensornetStatus_t cutensornetGetOutputTensorDetails(const cutensornetHandle_t handle,
          |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    cuquantum/cutensornet/cutensornet.cpp:5942:220: warning: ‘cutensornetStatus_t cutensornetGetOutputTensorDetails(cutensornetHandle_t, cutensornetNetworkDescriptor_t, int32_t*, size_t*, int32_t*, int64_t*, int64_t*)’ is deprecated: please use cutensornetGetOutputTensorDescriptor instead [-Wdeprecated-declarations]
     5942 |         __pyx_v_status = cutensornetGetOutputTensorDetails(((cutensornetHandle_t)__pyx_v_handle), ((cutensornetNetworkDescriptor_t)__pyx_v_tn_desc), (&__pyx_v_numModesOut), NULL, __pyx_v_mPtr, __pyx_v_ePtr, __pyx_v_sPtr);
          |                                                                                                                                                                                                                            ^
    In file included from cuquantum/cutensornet/cutensornet.cpp:785:
    /tmp/pip-build-env-5hkeejk4/normal/lib/python3.9/site-packages/cuquantum/include/cutensornet.h:245:21: note: declared here
      245 | cutensornetStatus_t cutensornetGetOutputTensorDetails(const cutensornetHandle_t handle,
          |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    cuquantum/cutensornet/cutensornet.cpp: In function ‘PyObject* __pyx_f_9cuquantum_11cutensornet_11cutensornet_workspace_compute_sizes(intptr_t, intptr_t, intptr_t, intptr_t, int)’:
    cuquantum/cutensornet/cutensornet.cpp:7140:257: warning: ‘cutensornetStatus_t cutensornetWorkspaceComputeSizes(cutensornetHandle_t, cutensornetNetworkDescriptor_t, cutensornetContractionOptimizerInfo_t, cutensornetWorkspaceDescriptor_t)’ is deprecated: please use cutensornetWorkspaceComputeContractionSizes instead [-Wdeprecated-declarations]
     7140 |         __pyx_v_status = cutensornetWorkspaceComputeSizes(((cutensornetHandle_t)__pyx_v_handle), ((cutensornetNetworkDescriptor_t)__pyx_v_tn_desc), ((cutensornetContractionOptimizerInfo_t)__pyx_v_info), ((cutensornetWorkspaceDescriptor_t)__pyx_v_workspace));
          |                                                                                                                                                                                                                                                                 ^
    In file included from cuquantum/cutensornet/cutensornet.cpp:785:
    /tmp/pip-build-env-5hkeejk4/normal/lib/python3.9/site-packages/cuquantum/include/cutensornet.h:307:21: note: declared here
      307 | cutensornetStatus_t cutensornetWorkspaceComputeSizes(const cutensornetHandle_t handle,
          |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    cuquantum/cutensornet/cutensornet.cpp: In function ‘uint64_t __pyx_f_9cuquantum_11cutensornet_11cutensornet_workspace_get_size(intptr_t, intptr_t, int, int, int)’:
    cuquantum/cutensornet/cutensornet.cpp:7584:259: warning: ‘cutensornetStatus_t cutensornetWorkspaceGetSize(cutensornetHandle_t, cutensornetWorkspaceDescriptor_t, cutensornetWorksizePref_t, cutensornetMemspace_t, uint64_t*)’ is deprecated: please use cutensornetWorkspaceGetMemorySize instead [-Wdeprecated-declarations]
     7584 |         __pyx_v_status = cutensornetWorkspaceGetSize(((cutensornetHandle_t)__pyx_v_handle), ((cutensornetWorkspaceDescriptor_t)__pyx_v_workspace), ((cutensornetWorksizePref_t)__pyx_v_pref), ((cutensornetMemspace_t)__pyx_v_mem_space), (&__pyx_v_workspaceSize));
          |                                                                                                                                                                                                                                                                   ^
    In file included from cuquantum/cutensornet/cutensornet.cpp:785:
    /tmp/pip-build-env-5hkeejk4/normal/lib/python3.9/site-packages/cuquantum/include/cutensornet.h:339:21: note: declared here
      339 | cutensornetStatus_t cutensornetWorkspaceGetSize(const cutensornetHandle_t handle,
          |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
    cuquantum/cutensornet/cutensornet.cpp: In function ‘PyObject* __pyx_f_9cuquantum_11cutensornet_11cutensornet_workspace_set(intptr_t, intptr_t, int, intptr_t, uint64_t, int)’:
    cuquantum/cutensornet/cutensornet.cpp:8057:255: warning: ‘cutensornetStatus_t cutensornetWorkspaceSet(cutensornetHandle_t, cutensornetWorkspaceDescriptor_t, cutensornetMemspace_t, void*, uint64_t)’ is deprecated: please use cutensornetWorkspaceSetMemory instead [-Wdeprecated-declarations]
     8057 |         __pyx_v_status = cutensornetWorkspaceSet(((cutensornetHandle_t)__pyx_v_handle), ((cutensornetWorkspaceDescriptor_t)__pyx_v_workspace), ((cutensornetMemspace_t)__pyx_v_mem_space), ((void *)__pyx_v_workspace_ptr), ((uint64_t)__pyx_v_workspace_size));
          |                                                                                                                                                                                                                                                               ^
    In file included from cuquantum/cutensornet/cutensornet.cpp:785:
    /tmp/pip-build-env-5hkeejk4/normal/lib/python3.9/site-packages/cuquantum/include/cutensornet.h:384:21: note: declared here
      384 | cutensornetStatus_t cutensornetWorkspaceSet(const cutensornetHandle_t handle,
          |                     ^~~~~~~~~~~~~~~~~~~~~~~
    cuquantum/cutensornet/cutensornet.cpp: In function ‘PyObject* __pyx_f_9cuquantum_11cutensornet_11cutensornet_workspace_get(intptr_t, intptr_t, int, int)’:
    cuquantum/cutensornet/cutensornet.cpp:8536:239: warning: ‘cutensornetStatus_t cutensornetWorkspaceGet(cutensornetHandle_t, cutensornetWorkspaceDescriptor_t, cutensornetMemspace_t, void**, uint64_t*)’ is deprecated: please use cutensornetWorkspaceGetMemory instead [-Wdeprecated-declarations]
     8536 |         __pyx_v_status = cutensornetWorkspaceGet(((cutensornetHandle_t)__pyx_v_handle), ((cutensornetWorkspaceDescriptor_t)__pyx_v_workspace), ((cutensornetMemspace_t)__pyx_v_mem_space), (&__pyx_v_workspace_ptr), (&__pyx_v_workspace_size));
          |                                                                                                                                                                                                                                               ^
    In file included from cuquantum/cutensornet/cutensornet.cpp:785:
    /tmp/pip-build-env-5hkeejk4/normal/lib/python3.9/site-packages/cuquantum/include/cutensornet.h:426:21: note: declared here
      426 | cutensornetStatus_t cutensornetWorkspaceGet(const cutensornetHandle_t handle,
          |                     ^~~~~~~~~~~~~~~~~~~~~~~
    cuquantum/cutensornet/cutensornet.cpp: In function ‘PyObject* __pyx_f_9cuquantum_11cutensornet_11cutensornet_contraction(intptr_t, intptr_t, PyObject*, intptr_t, intptr_t, int64_t, intptr_t, int)’:
    cuquantum/cutensornet/cutensornet.cpp:13963:291: warning: ‘cutensornetStatus_t cutensornetContraction(cutensornetHandle_t, cutensornetContractionPlan_t, const void* const*, void*, cutensornetWorkspaceDescriptor_t, int64_t, cudaStream_t)’ is deprecated: please use cutensornetContractSlices instead [-Wdeprecated-declarations]
    13963 |         __pyx_v_status = cutensornetContraction(((cutensornetHandle_t)__pyx_v_handle), ((cutensornetContractionPlan_t)__pyx_v_plan), __pyx_v_rawDataInPtr, ((void *)__pyx_v_raw_data_out), ((cutensornetWorkspaceDescriptor_t)__pyx_v_workspace), __pyx_v_slice_id, ((cudaStream_t)__pyx_v_stream));
          |                                                                                                                                                                                                                                                                                                   ^
    In file included from cuquantum/cutensornet/cutensornet.cpp:785:
    /tmp/pip-build-env-5hkeejk4/normal/lib/python3.9/site-packages/cuquantum/include/cutensornet.h:767:21: note: declared here
      767 | cutensornetStatus_t cutensornetContraction(const cutensornetHandle_t handle,
          |                     ^~~~~~~~~~~~~~~~~~~~~~
    gcc -pthread -B /home/cuquantum/conda/envs/cuquantum-23.06/compiler_compat -shared -Wl,--allow-shlib-undefined -Wl,-rpath,/home/cuquantum/conda/envs/cuquantum-23.06/lib -Wl,-rpath-link,/home/cuquantum/conda/envs/cuquantum-23.06/lib -L/home/cuquantum/conda/envs/cuquantum-23.06/lib -Wl,--allow-shlib-undefined -Wl,-rpath,/home/cuquantum/conda/envs/cuquantum-23.06/lib -Wl,-rpath-link,/home/cuquantum/conda/envs/cuquantum-23.06/lib -L/home/cuquantum/conda/envs/cuquantum-23.06/lib build/temp.linux-x86_64-cpython-39/cuquantum/utils.o -o build/lib.linux-x86_64-cpython-39/cuquantum/utils.cpython-39-x86_64-linux-gnu.so
    In function ‘PyObject* __pyx_f_9cuquantum_11cutensornet_11cutensornet_compute_gradients_backward(intptr_t, intptr_t, PyObject*, intptr_t, PyObject*, int, intptr_t, intptr_t, int)’,
        inlined from ‘PyObject* __pyx_pf_9cuquantum_11cutensornet_11cutensornet_162compute_gradients_backward(PyObject*, intptr_t, intptr_t, PyObject*, intptr_t, PyObject*, int, intptr_t, intptr_t)’ at cuquantum/cutensornet/cutensornet.cpp:22347:88,
        inlined from ‘PyObject* __pyx_pw_9cuquantum_11cutensornet_11cutensornet_163compute_gradients_backward(PyObject*, PyObject*, PyObject*)’ at cuquantum/cutensornet/cutensornet.cpp:22331:90:
    cuquantum/cutensornet/cutensornet.cpp:22151:61: warning: call to ‘cutensornetComputeGradientsBackward’ declared with attribute warning: cutensornetComputeGradientsBackward is an experimental API and subject to future change [-Wattribute-warning]
    22151 |         __pyx_v_status = cutensornetComputeGradientsBackward(((cutensornetHandle_t)__pyx_v_handle), ((cutensornetContractionPlan_t)__pyx_v_plan), __pyx_v_rawDataInPtr, ((void *)__pyx_v_output_gradient), __pyx_v_gradientsPtr, ((int32_t)__pyx_v_accumulate_output), ((cutensornetWorkspaceDescriptor_t)__pyx_v_workspace), ((cudaStream_t)__pyx_v_stream));
          |                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    g++ -pthread -B /home/cuquantum/conda/envs/cuquantum-23.06/compiler_compat -shared -Wl,--allow-shlib-undefined -Wl,-rpath,/home/cuquantum/conda/envs/cuquantum-23.06/lib -Wl,-rpath-link,/home/cuquantum/conda/envs/cuquantum-23.06/lib -L/home/cuquantum/conda/envs/cuquantum-23.06/lib -Wl,--allow-shlib-undefined -Wl,-rpath,/home/cuquantum/conda/envs/cuquantum-23.06/lib -Wl,-rpath-link,/home/cuquantum/conda/envs/cuquantum-23.06/lib -L/home/cuquantum/conda/envs/cuquantum-23.06/lib build/temp.linux-x86_64-cpython-39/cuquantum/cutensornet/cutensornet.o -L/tmp/pip-build-env-5hkeejk4/normal/lib/python3.9/site-packages/cuquantum/lib -L/tmp/pip-build-env-5hkeejk4/normal/lib/python3.9/site-packages/cuquantum/lib64 -lcutensornet -o build/lib.linux-x86_64-cpython-39/cuquantum/cutensornet/cutensornet.cpython-39-x86_64-linux-gnu.so
    /home/cuquantum/conda/envs/cuquantum-23.06/compiler_compat/ld: cannot find -lcutensornet: No such file or directory
    collect2: error: ld returned 1 exit status
    g++ -pthread -B /home/cuquantum/conda/envs/cuquantum-23.06/compiler_compat -shared -Wl,--allow-shlib-undefined -Wl,-rpath,/home/cuquantum/conda/envs/cuquantum-23.06/lib -Wl,-rpath-link,/home/cuquantum/conda/envs/cuquantum-23.06/lib -L/home/cuquantum/conda/envs/cuquantum-23.06/lib -Wl,--allow-shlib-undefined -Wl,-rpath,/home/cuquantum/conda/envs/cuquantum-23.06/lib -Wl,-rpath-link,/home/cuquantum/conda/envs/cuquantum-23.06/lib -L/home/cuquantum/conda/envs/cuquantum-23.06/lib build/temp.linux-x86_64-cpython-39/cuquantum/custatevec/custatevec.o -L/tmp/pip-build-env-5hkeejk4/normal/lib/python3.9/site-packages/cuquantum/lib -L/tmp/pip-build-env-5hkeejk4/normal/lib/python3.9/site-packages/cuquantum/lib64 -lcustatevec -o build/lib.linux-x86_64-cpython-39/cuquantum/custatevec/custatevec.cpython-39-x86_64-linux-gnu.so
    /home/cuquantum/conda/envs/cuquantum-23.06/compiler_compat/ld: cannot find -lcustatevec: No such file or directory
    collect2: error: ld returned 1 exit status
    error: command '/usr/bin/g++' failed with exit code 1
    error: subprocess-exited-with-error
    
    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> See above for output.
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
    full command: /home/cuquantum/conda/envs/cuquantum-23.06/bin/python3.9 -c '
    exec(compile('"'"''"'"''"'"'
    # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
    #
    # - It imports setuptools before invoking setup.py, to enable projects that directly
    #   import from `distutils.core` to work with newer packaging standards.
    # - It provides a clear error message when setuptools is not installed.
    # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
    #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
    #     manifest_maker: standard file '"'"'-c'"'"' not found".
    # - It generates a shim setup.py, for handling setup.cfg-only projects.
    import os, sys, tokenize
    
    try:
        import setuptools
    except ImportError as error:
        print(
            "ERROR: Can not execute `setup.py` since setuptools is not available in "
            "the build environment.",
            file=sys.stderr,
        )
        sys.exit(1)
    
    __file__ = %r
    sys.argv[0] = __file__
    
    if os.path.exists(__file__):
        filename = __file__
        with tokenize.open(__file__) as f:
            setup_py_code = f.read()
    else:
        filename = "<auto-generated setuptools caller>"
        setup_py_code = "from setuptools import setup; setup()"
    
    exec(compile(setup_py_code, filename, "exec"))
    '"'"''"'"''"'"' % ('"'"'/home/ubuntu/cuQuantum/python/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' develop --no-deps
    cwd: /home/ubuntu/cuQuantum/python/
  Rolling back uninstall of cuquantum-python-cu11
  Moving to /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/__init__.pxd
   from /tmp/pip-uninstall-706e1hre/__init__.pxd
  Moving to /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/__init__.py
   from /tmp/pip-uninstall-706e1hre/__init__.py
  Moving to /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/__main__.py
   from /tmp/pip-uninstall-706e1hre/__main__.py
  Moving to /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/__pycache__/
   from /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/~_pycache__
  Moving to /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/_version.py
   from /tmp/pip-uninstall-706e1hre/_version.py
  Moving to /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/custatevec/
   from /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/~ustatevec
  Moving to /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/cutensornet/
   from /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/~utensornet
  Moving to /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/utils.cpython-39-x86_64-linux-gnu.so
   from /tmp/pip-uninstall-706e1hre/utils.cpython-39-x86_64-linux-gnu.so
  Moving to /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/utils.pxd
   from /tmp/pip-uninstall-706e1hre/utils.pxd
  Moving to /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum/utils.pyx
   from /tmp/pip-uninstall-706e1hre/utils.pyx
  Moving to /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/cuquantum_python_cu11-23.6.0.dist-info/
   from /home/cuquantum/conda/envs/cuquantum-23.06/lib/python3.9/site-packages/~uquantum_python_cu11-23.6.0.dist-info
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

leofang Oct 18, 2023
Maintainer

Thanks for reply. I see you wanted to do editable install (-e), which was what I missed in the beginning.

To do that, you'll need to do all 7 steps outlined in the docs (ex: https://github.com/NVIDIA/cuQuantum/tree/main/python#building-and-installing-cuquantum-python-from-source). But it means you'll need to set up CUQUANTUM_ROOT, CUTENSOR_ROOT, and LD_LIBRARY_PATH yourself. This happens because we designed the editable install for power developers (= our team) to use, and it's not what our container was designed for. (Things are there, but weren't designed for this workflow.)

I would suggest, for simplicity, that you get rid of the -e flag, and always do pip install . after you make local changes. This should work even inside the container. Let me know.

Sinestro38 Oct 18, 2023
Author

Okay, to set up CUQUANTUM_ROOT, CUTENSOR_ROOT, and LD_LIBRARY_PATH, do I need to download the so frameworks onto the container? @leofang

leofang Oct 18, 2023
Maintainer

Yes, if you go all manually, you'll need to follow the instruction to do so, e.g.

But if you do pip install ., all these would be taken care for you (and most likely pip would reuse the already existing wheels cached somewhere in the container) 🙂

Answer selected by Sinestro38

Sinestro38 Oct 18, 2023
Author

I see, removing the editable flag indeed worked. What about the -e editable flag messes it up?

leofang Oct 18, 2023
Maintainer

Nothing is messed up, as I said earlier it's by design 🙂

A large part of our Python package is co-evolving with our (close-sourced) C libraries, and this repo currently is not set up to accept external contributions, so for our internal convenience it makes sense we look up the locations of headers/DSOs via environment variables for local development. The assumption is we are the main users of the -e flag.

For external users trying to build from source, on the other hand, we still support them to fetch C libraries from PyPI.org when they do pip install, but because of Python wheels' constraints, as outlined in this post, the code path to support it has to diverge from that for our internal development workflow.

Sinestro38 Oct 18, 2023
Author

I see, sounds good!

Sinestro38 · 2023-10-18T18:57:20Z

Sinestro38
Oct 18, 2023
Author

I'm trying to extend cutensornet to accept circuits from a different library so I need an editable installation.

3 replies

leofang Oct 18, 2023
Maintainer

I'm trying to extend cutensornet to accept circuits from a different library

@Sinestro38 We would be interested in knowing more detail, so that we could try to offer you guidance/suggestion. Can you share a bit more what's planned?

Sinestro38 Oct 19, 2023
Author

I'm trying to extend the CircuitToEinsum class to support ingestion of a quantum circuit defined in a jax framework. Currently, it seems only Qiskit & cirq circuits are supported.

Sinestro38 Oct 19, 2023
Author

Do you know what the approximate performance differences are btw between the cython open-sourced version of custatevec vs the closed-source C distribution? @leofang

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Question] Issues building cuquantum-python from source #95

{{title}}

Replies: 2 comments 11 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

[Question] Issues building cuquantum-python from source #95

Sinestro38 Oct 18, 2023

Replies: 2 comments · 11 replies

leofang Oct 18, 2023 Maintainer

Sinestro38 Oct 18, 2023 Author

leofang Oct 18, 2023 Maintainer

Sinestro38 Oct 18, 2023 Author

leofang Oct 18, 2023 Maintainer

Sinestro38 Oct 18, 2023 Author

leofang Oct 18, 2023 Maintainer

Sinestro38 Oct 18, 2023 Author

Sinestro38 Oct 18, 2023 Author

leofang Oct 18, 2023 Maintainer

Sinestro38 Oct 19, 2023 Author

Sinestro38 Oct 19, 2023 Author

Sinestro38
Oct 18, 2023

Replies: 2 comments 11 replies

leofang
Oct 18, 2023
Maintainer

Sinestro38 Oct 18, 2023
Author

leofang Oct 18, 2023
Maintainer

Sinestro38 Oct 18, 2023
Author

leofang Oct 18, 2023
Maintainer

Sinestro38 Oct 18, 2023
Author

leofang Oct 18, 2023
Maintainer

Sinestro38 Oct 18, 2023
Author

Sinestro38
Oct 18, 2023
Author

leofang Oct 18, 2023
Maintainer

Sinestro38 Oct 19, 2023
Author

Sinestro38 Oct 19, 2023
Author