compile tch-rs against torch 2.1.0.dev

[3togo]

Whenever I call forward() repetitively to do inference, libtorch will crash within 10 cycles if libtorch version less than 2.0.1 or older is used. I accidentally find that this problem will not happen if I upgrade libtorch to 2.1.0.dev.

Because of this reason, I need to compile tch-rs with libtorch 2.1.0.dev. Before I do the compilation, I have done the followings:

1. modified build.rs to support c++17
2. export LIBTORCH_BYPASS_VERSION_CHECK=1
3. export LIBTORCH=/opt/libtorch-git
4. export LIBTORCH_USE_PYTORCH=1

I was then stuck at the following errors . Any pointers that help?

polin@MDL-001:~/work/git/tch-rs$ cargo build 2>&1|grep -i error
warning: libtch/torch_api_generated.cpp:451:19: error: ‘_chunk_grad_outputs_efficient_attention’ is not a member of ‘torch’
warning: libtch/torch_api_generated.cpp:823:111: error: invalid initialization of reference of type ‘const c10::optional<at::Tensor>&’ from expression of type ‘bool’
warning: libtch/torch_api_generated.cpp:1074:172: error: invalid initialization of reference of type ‘const at::Tensor&’ from expression of type ‘int64_t’ {aka ‘long int’}
warning: libtch/torch_api_generated.cpp:1696:29: error: ‘_native_decoder_only_multi_head_attention’ is not a member of ‘torch’
warning: libtch/torch_api_generated.cpp:1706:29: error: ‘_native_decoder_only_multi_head_attention_out’ is not a member of ‘torch’
warning: libtch/torch_api_generated.cpp:1795:99: error: invalid initialization of reference of type ‘const at::Tensor&’ from expression of type ‘at::IntArrayRef’ {aka ‘c10::ArrayRef<long int>’}
warning: libtch/torch_api_generated.cpp:1802:104: error: invalid initialization of reference of type ‘const at::Tensor&’ from expression of type ‘at::IntArrayRef’ {aka ‘c10::ArrayRef<long int>’}
warning: libtch/torch_api_generated.cpp:1809:114: error: invalid initialization of reference of type ‘const at::Tensor&’ from expression of type ‘at::IntArrayRef’ {aka ‘c10::ArrayRef<long int>’}
warning: libtch/torch_api_generated.cpp:2038:29: error: ‘_scaled_dot_product_attention’ is not a member of ‘torch’
warning: libtch/torch_api_generated.cpp:2054:91: error: invalid initialization of reference of type ‘const c10::optional<at::Tensor>&’ from expression of type ‘bool’
warning: libtch/torch_api_generated.cpp:2062:130: error: invalid initialization of reference of type ‘const at::Tensor&’ from expression of type ‘bool’
warning: libtch/torch_api_generated.cpp:2071:191: error: invalid initialization of reference of type ‘const at::Tensor&’ from expression of type ‘int64_t’ {aka ‘long int’}
warning: libtch/torch_api_generated.cpp:2724:29: error: ‘_transformer_decoder_only_layer_fwd’ is not a member of ‘torch’
warning: libtch/torch_api_generated.cpp:2733:29: error: ‘_transformer_decoder_only_layer_fwd_out’ is not a member of ‘torch’
warning: libtch/torch_api_generated.cpp:16460:32: error: no matching function for call to ‘std(at::Tensor&, c10::optional<c10::ArrayRef<long int> >, c10::optional<long int>, bool)’
warning: libtch/torch_api_generated.cpp:16467:36: error: no matching function for call to ‘std_out(at::Tensor&, at::Tensor&, c10::optional<c10::ArrayRef<long int> >, c10::optional<long int>, bool)’
warning: libtch/torch_api_generated.cpp:16489:37: error: no matching function for call to ‘std_mean(at::Tensor&, c10::optional<c10::ArrayRef<long int> >, c10::optional<long int>, bool)’
warning: libtch/torch_api_generated.cpp:16497:190: error: invalid initialization of reference of type ‘const c10::optional<c10::Scalar>&’ from expression of type ‘c10::optional<long int>’
warning: libtch/torch_api_generated.cpp:16997:29: error: ‘to_sparse_bsc_out’ is not a member of ‘torch’
warning: libtch/torch_api_generated.cpp:17011:29: error: ‘to_sparse_bsr_out’ is not a member of ‘torch’
warning: libtch/torch_api_generated.cpp:17025:29: error: ‘to_sparse_csc_out’ is not a member of ‘torch’
warning: libtch/torch_api_generated.cpp:17039:29: error: ‘to_sparse_csr_out’ is not a member of ‘torch’
warning: libtch/torch_api_generated.cpp:17046:29: error: ‘to_sparse_out’ is not a member of ‘torch’
warning: libtch/torch_api_generated.cpp:17060:29: error: ‘to_sparse_out’ is not a member of ‘torch’
warning: libtch/torch_api_generated.cpp:17851:32: error: no matching function for call to ‘var(at::Tensor&, c10::optional<c10::ArrayRef<long int> >, c10::optional<long int>, bool)’
warning: libtch/torch_api_generated.cpp:17858:36: error: no matching function for call to ‘var_out(at::Tensor&, at::Tensor&, c10::optional<c10::ArrayRef<long int> >, c10::optional<long int>, bool)’
warning: libtch/torch_api_generated.cpp:17880:37: error: no matching function for call to ‘var_mean(at::Tensor&, c10::optional<c10::ArrayRef<long int> >, c10::optional<long int>, bool)’
warning: libtch/torch_api_generated.cpp:17888:190: error: invalid initialization of reference of type ‘const c10::optional<c10::Scalar>&’ from expression of type ‘c10::optional<long int>’
warning: ToolExecError: Command "c++" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-4" "-fno-omit-frame-pointer" "-m64" "-I" "/home/polin/.local/lib/python3.11/site-packages/torch/include" "-I" "/home/polin/.local/lib/python3.11/site-packages/torch/include/torch/csrc/api/include" "-I" "/home/polin/.local/lib/python3.11/site-packages/torch/include/TH" "-I" "/home/polin/.local/lib/python3.11/site-packages/torch/include/THC" "-Wl,-rpath=/home/polin/.local/lib/python3.11/site-packages/torch/lib" "-std=c++17" "-D_GLIBCXX_USE_CXX11_ABI=0" "-o" "/mnt/data/home/polin/git/tch-rs/target/debug/build/torch-sys-e1c92ec6b2c1b4e6/out/libtch/torch_api_generated.o" "-c" "libtch/torch_api_generated.cpp" with args "c++" did not execute successfully (status code exit status: 1).
error: failed to run custom build command for `torch-sys v0.13.0 (/mnt/data/home/polin/git/tch-rs/torch-sys)`
  cargo:warning=libtch/torch_api_generated.cpp:451:19: error: ‘_chunk_grad_outputs_efficient_attention’ is not a member of ‘torch’
  cargo:warning=libtch/torch_api_generated.cpp:823:111: error: invalid initialization of reference of type ‘const c10::optional<at::Tensor>&’ from expression of type ‘bool’
  cargo:warning=libtch/torch_api_generated.cpp:1074:172: error: invalid initialization of reference of type ‘const at::Tensor&’ from expression of type ‘int64_t’ {aka ‘long int’}
  cargo:warning=libtch/torch_api_generated.cpp:1696:29: error: ‘_native_decoder_only_multi_head_attention’ is not a member of ‘torch’
  cargo:warning=libtch/torch_api_generated.cpp:1706:29: error: ‘_native_decoder_only_multi_head_attention_out’ is not a member of ‘torch’
  cargo:warning=libtch/torch_api_generated.cpp:1795:99: error: invalid initialization of reference of type ‘const at::Tensor&’ from expression of type ‘at::IntArrayRef’ {aka ‘c10::ArrayRef<long int>’}
  cargo:warning=libtch/torch_api_generated.cpp:1802:104: error: invalid initialization of reference of type ‘const at::Tensor&’ from expression of type ‘at::IntArrayRef’ {aka ‘c10::ArrayRef<long int>’}
  cargo:warning=libtch/torch_api_generated.cpp:1809:114: error: invalid initialization of reference of type ‘const at::Tensor&’ from expression of type ‘at::IntArrayRef’ {aka ‘c10::ArrayRef<long int>’}
  cargo:warning=libtch/torch_api_generated.cpp:2038:29: error: ‘_scaled_dot_product_attention’ is not a member of ‘torch’
  cargo:warning=libtch/torch_api_generated.cpp:2054:91: error: invalid initialization of reference of type ‘const c10::optional<at::Tensor>&’ from expression of type ‘bool’
  cargo:warning=libtch/torch_api_generated.cpp:2062:130: error: invalid initialization of reference of type ‘const at::Tensor&’ from expression of type ‘bool’
  cargo:warning=libtch/torch_api_generated.cpp:2071:191: error: invalid initialization of reference of type ‘const at::Tensor&’ from expression of type ‘int64_t’ {aka ‘long int’}
  cargo:warning=libtch/torch_api_generated.cpp:2724:29: error: ‘_transformer_decoder_only_layer_fwd’ is not a member of ‘torch’
  cargo:warning=libtch/torch_api_generated.cpp:2733:29: error: ‘_transformer_decoder_only_layer_fwd_out’ is not a member of ‘torch’
  cargo:warning=libtch/torch_api_generated.cpp:16460:32: error: no matching function for call to ‘std(at::Tensor&, c10::optional<c10::ArrayRef<long int> >, c10::optional<long int>, bool)’
  cargo:warning=libtch/torch_api_generated.cpp:16467:36: error: no matching function for call to ‘std_out(at::Tensor&, at::Tensor&, c10::optional<c10::ArrayRef<long int> >, c10::optional<long int>, bool)’
  cargo:warning=libtch/torch_api_generated.cpp:16489:37: error: no matching function for call to ‘std_mean(at::Tensor&, c10::optional<c10::ArrayRef<long int> >, c10::optional<long int>, bool)’
  cargo:warning=libtch/torch_api_generated.cpp:16497:190: error: invalid initialization of reference of type ‘const c10::optional<c10::Scalar>&’ from expression of type ‘c10::optional<long int>’
  cargo:warning=libtch/torch_api_generated.cpp:16997:29: error: ‘to_sparse_bsc_out’ is not a member of ‘torch’
  cargo:warning=libtch/torch_api_generated.cpp:17011:29: error: ‘to_sparse_bsr_out’ is not a member of ‘torch’
  cargo:warning=libtch/torch_api_generated.cpp:17025:29: error: ‘to_sparse_csc_out’ is not a member of ‘torch’
  cargo:warning=libtch/torch_api_generated.cpp:17039:29: error: ‘to_sparse_csr_out’ is not a member of ‘torch’
  cargo:warning=libtch/torch_api_generated.cpp:17046:29: error: ‘to_sparse_out’ is not a member of ‘torch’
  cargo:warning=libtch/torch_api_generated.cpp:17060:29: error: ‘to_sparse_out’ is not a member of ‘torch’
  cargo:warning=libtch/torch_api_generated.cpp:17851:32: error: no matching function for call to ‘var(at::Tensor&, c10::optional<c10::ArrayRef<long int> >, c10::optional<long int>, bool)’
  cargo:warning=libtch/torch_api_generated.cpp:17858:36: error: no matching function for call to ‘var_out(at::Tensor&, at::Tensor&, c10::optional<c10::ArrayRef<long int> >, c10::optional<long int>, bool)’
  cargo:warning=libtch/torch_api_generated.cpp:17880:37: error: no matching function for call to ‘var_mean(at::Tensor&, c10::optional<c10::ArrayRef<long int> >, c10::optional<long int>, bool)’
  cargo:warning=libtch/torch_api_generated.cpp:17888:190: error: invalid initialization of reference of type ‘const c10::optional<c10::Scalar>&’ from expression of type ‘c10::optional<long int>’
  cargo:warning=ToolExecError: Command "c++" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-4" "-fno-omit-frame-pointer" "-m64" "-I" "/home/polin/.local/lib/python3.11/site-packages/torch/include" "-I" "/home/polin/.local/lib/python3.11/site-packages/torch/include/torch/csrc/api/include" "-I" "/home/polin/.local/lib/python3.11/site-packages/torch/include/TH" "-I" "/home/polin/.local/lib/python3.11/site-packages/torch/include/THC" "-Wl,-rpath=/home/polin/.local/lib/python3.11/site-packages/torch/lib" "-std=c++17" "-D_GLIBCXX_USE_CXX11_ABI=0" "-o" "/mnt/data/home/polin/git/tch-rs/target/debug/build/torch-sys-e1c92ec6b2c1b4e6/out/libtch/torch_api_generated.o" "-c" "libtch/torch_api_generated.cpp" with args "c++" did not execute successfully (status code exit status: 1).
  error occurred: Command "c++" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-4" "-fno-omit-frame-pointer" "-m64" "-I" "/home/polin/.local/lib/python3.11/site-packages/torch/include" "-I" "/home/polin/.local/lib/python3.11/site-packages/torch/include/torch/csrc/api/include" "-I" "/home/polin/.local/lib/python3.11/site-packages/torch/include/TH" "-I" "/home/polin/.local/lib/python3.11/site-packages/torch/include/THC" "-Wl,-rpath=/home/polin/.local/lib/python3.11/site-packages/torch/lib" "-std=c++17" "-D_GLIBCXX_USE_CXX11_ABI=0" "-o" "/mnt/data/home/polin/git/tch-rs/target/debug/build/torch-sys-e1c92ec6b2c1b4e6/out/libtch/torch_api_generated.o" "-c" "libtch/torch_api_generated.cpp" with args "c++" did not execute successfully (status code exit status: 1).

[LaurentMazare]

We only support using 2.0.0 at this stage (though 2.0.1 is likely to work), I plan on getting support for 2.1.0 once it has been a bit more crafted out so in a week or two (we're at RC2 so far I think).

[3togo]

many thanks

[lmelinda]

We only support using 2.0.0 at this stage (though 2.0.1 is likely to work), I plan on getting support for 2.1.0 once it has been a bit more crafted out so in a week or two (we're at RC2 so far I think).

Hi, any updates on when we plan to support 2.1? (2.0.1 does work btw, I have tried)

[LaurentMazare]

Still waiting on the actual release, my understanding was that it's scheduled for the 2023/10/04 so pretty soon but I haven't checked lately, I might start with the latest RC before that.

[LaurentMazare]

I've started putting together #803 which should add support for v2.1.0 - the tests pass fine. I will wait for the official download links to be available before merging this version (this is scheduled for later this week afaik).

[LaurentMazare]

Just merged #803 so now tch-rs is based on the just released PyTorch v2.1.0!

[LaurentMazare]

Closing as hopefully all good now, the new crate is on crates.io.

[lmelinda]

btw just curious, would the updated tch-rs be compatible with system that run on pytorch2.0?

[LaurentMazare]

btw just curious, would the updated tch-rs be compatible with system that run on pytorch2.0?

Nope, versions of tch-rs match exactly one version of pytorch/libtorch so you would have to downgrade the crate to support pytorch2.0 instead of 2.1.