Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enables backend preprocessing to take place outside of the backend interface #51757

Closed
wants to merge 9 commits into from

Conversation

raziel
Copy link
Contributor

@raziel raziel commented Feb 5, 2021

Stack from ghstack:

Enables backend preprocessing to take place outside of the backend interface.

What's new:

  • A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction).
  • Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction.
  • A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key.
  • When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module.

Why?:
These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface.
This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary.

What didn't change:

  • Everything is backwards compatible:
    ** The existing "preprocess" method in PyTorchBackendInterface.
    ** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface.

Longer term, the plan is to refactor existing users to move to the new backend registration.

Differential Revision: D26261042

…terface

Enables backend preprocessing to take place outside of the backend interface.

What's new:
* A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction).
* Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction.
* A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key.
* When a BackendPreprocessFunction is used, the  PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module.


Why?:
These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface.
This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary.

What didn't change:
* Everything is backwards compatible:
** The existing "preprocess" method in PyTorchBackendInterface.
** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface.

Longer term, the plan is to refactor existing users to move to the new backend registration.

Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/)

[ghstack-poisoned]
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Feb 5, 2021

💊 CI failures summary and remediations

As of commit e9dcc34 (more details on the Dr. CI page):


  • 8/8 failures possibly* introduced in this PR
    • 2/8 non-CircleCI failure(s)

🕵️ 4 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See CircleCI build pytorch_linux_xenial_py3_6_gcc5_4_build (1/4)

Step: "(Optional) Merge target branch" (full log | diagnosis details | 🔁 rerun)

Automatic merge failed; fix conflicts and then commit the result.
CONFLICT (add/add): Merge conflict in .circleci/docker/common/install_rocm.sh
Auto-merging .circleci/docker/common/install_rocm.sh
CONFLICT (add/add): Merge conflict in .circleci/docker/common/install_conda.sh
Auto-merging .circleci/docker/common/install_conda.sh
CONFLICT (add/add): Merge conflict in .circleci/config.yml
Auto-merging .circleci/config.yml
CONFLICT (add/add): Merge conflict in .circleci/cimodel/data/dimensions.py
Auto-merging .circleci/cimodel/data/dimensions.py
CONFLICT (add/add): Merge conflict in .circleci/README.md
Auto-merging .circleci/README.md
Automatic merge failed; fix conflicts and then commit the result.


Exited with code exit status 1

See CircleCI build pytorch_xla_linux_bionic_py3_6_clang9_build (2/4)

Step: "(Optional) Merge target branch" (full log | diagnosis details | 🔁 rerun)

Automatic merge failed; fix conflicts and then commit the result.
CONFLICT (add/add): Merge conflict in .circleci/docker/common/install_rocm.sh
Auto-merging .circleci/docker/common/install_rocm.sh
CONFLICT (add/add): Merge conflict in .circleci/docker/common/install_conda.sh
Auto-merging .circleci/docker/common/install_conda.sh
CONFLICT (add/add): Merge conflict in .circleci/config.yml
Auto-merging .circleci/config.yml
CONFLICT (add/add): Merge conflict in .circleci/cimodel/data/dimensions.py
Auto-merging .circleci/cimodel/data/dimensions.py
CONFLICT (add/add): Merge conflict in .circleci/README.md
Auto-merging .circleci/README.md
Automatic merge failed; fix conflicts and then commit the result.


Exited with code exit status 1

See CircleCI build pytorch_windows_vs2019_py36_cuda10.1_build (3/4)

Step: "Install Cuda" (full log | diagnosis details | 🔁 rerun)

ls: cannot access '/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/bin/nvcc.exe': No such file or directory

Folders: 11
Files: 130
Size:       907512
Compressed: 111420
+ mkdir -p 'C:/Program Files/NVIDIA Corporation/NvToolsExt'
+ cp -r NvToolsExt/bin NvToolsExt/docs NvToolsExt/include NvToolsExt/lib NvToolsExt/samples 'C:/Program Files/NVIDIA Corporation/NvToolsExt/'
+ export 'NVTOOLSEXT_PATH=C:\Program Files\NVIDIA Corporation\NvToolsExt\'
+ NVTOOLSEXT_PATH='C:\Program Files\NVIDIA Corporation\NvToolsExt\'
+ ls '/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/bin/nvcc.exe'
ls: cannot access '/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/bin/nvcc.exe': No such file or directory
+ echo 'CUDA installation failed'
CUDA installation failed
+ mkdir -p /c/w/build-results
+ 7z a 'c:\w\build-results\cuda_install_logs.7z' cuda_install_logs

7-Zip 19.00 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21

Scanning the drive:
1 folder, 2 files, 3721951 bytes (3635 KiB)

See CircleCI build pytorch_linux_xenial_cuda9_2_cudnn7_py3_gcc5_4_build (4/4)

Step: "(Optional) Merge target branch" (full log | diagnosis details | 🔁 rerun)

Automatic merge failed; fix conflicts and then commit the result.
CONFLICT (add/add): Merge conflict in .circleci/docker/common/install_rocm.sh
Auto-merging .circleci/docker/common/install_rocm.sh
CONFLICT (add/add): Merge conflict in .circleci/docker/common/install_conda.sh
Auto-merging .circleci/docker/common/install_conda.sh
CONFLICT (add/add): Merge conflict in .circleci/config.yml
Auto-merging .circleci/config.yml
CONFLICT (add/add): Merge conflict in .circleci/cimodel/data/dimensions.py
Auto-merging .circleci/cimodel/data/dimensions.py
CONFLICT (add/add): Merge conflict in .circleci/README.md
Auto-merging .circleci/README.md
Automatic merge failed; fix conflicts and then commit the result.


Exited with code exit status 1


2 failures not recognized by patterns:

Job Step Action
CircleCI pytorch_linux_bionic_py3_8_gcc9_coverage_test2 Run tests 🔁 rerun
CircleCI pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test2 Run tests 🔁 rerun

Extra GitHub checks: 1 failed


ci.pytorch.org: 1 failed


This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

@facebook-github-bot facebook-github-bot added cla signed oncall: jit Add this issue/PR to JIT oncall triage queue labels Feb 5, 2021
… backend interface"

Enables backend preprocessing to take place outside of the backend interface.

What's new:
* A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction).
* Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction.
* A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key.
* When a BackendPreprocessFunction is used, the  PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module.


Why?:
These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface.
This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary.

What didn't change:
* Everything is backwards compatible:
** The existing "preprocess" method in PyTorchBackendInterface.
** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface.

Longer term, the plan is to refactor existing users to move to the new backend registration.

Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/)

[ghstack-poisoned]
… backend interface"

Enables backend preprocessing to take place outside of the backend interface.

What's new:
* A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction).
* Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction.
* A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key.
* When a BackendPreprocessFunction is used, the  PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module.


Why?:
These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface.
This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary.

What didn't change:
* Everything is backwards compatible:
** The existing "preprocess" method in PyTorchBackendInterface.
** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface.

Longer term, the plan is to refactor existing users to move to the new backend registration.

Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/)

[ghstack-poisoned]
… backend interface"

Enables backend preprocessing to take place outside of the backend interface.

What's new:
* A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction).
* Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction.
* A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key.
* When a BackendPreprocessFunction is used, the  PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module.


Why?:
These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface.
This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary.

What didn't change:
* Everything is backwards compatible:
** The existing "preprocess" method in PyTorchBackendInterface.
** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface.

Longer term, the plan is to refactor existing users to move to the new backend registration.

Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/)

[ghstack-poisoned]
raziel pushed a commit that referenced this pull request Feb 5, 2021
…terface

Pull Request resolved: #51757

Enables backend preprocessing to take place outside of the backend interface.

What's new:
* A new definition for backend preprocessing (i.e. BackendPreprocessFunction).
* Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction.
* A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key.
* When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module.


Why?:
These changes are needed to avoid forcing backend preprocessing to be part of the LoweredModule, and in the future be able to eliminate "preprocess" from the PyTorchBackendInterface.
This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary.

What didn't change:
* Everything is backwards compatible:
** The existing "preprocess" method in PyTorchBackendInterface is still there.
** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface.

Longer term, the plan is to refactor existing users to move to the new backend registration.
ghstack-source-id: 121097330

Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/)
Copy link

@SplitInfinity SplitInfinity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I don't see any major issues, but I'd like a response to some of my suggestions and questions before I accept.

torch/csrc/jit/backends/backend.h Show resolved Hide resolved
const BackendPreprocessFunction& preprocess) {
TORCH_CHECK(
!detail::hasBackendPreprocessFunction(name),
"BackendPreprocessFunction for backend ",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"BackendPreprocessFunction for backend ",
"Preprocessing function for backend ",

@@ -67,8 +67,15 @@ class TestBackend : public PyTorchBackendInterface {
}
};

c10::IValue backendPreprocessFunction(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
c10::IValue backendPreprocessFunction(
c10::IValue preprocess(

Comment on lines 54 to 56
typedef std::function<
c10::IValue(const Module&, const c10::Dict<IValue, IValue>&)>
BackendPreprocessFunction;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
typedef std::function<
c10::IValue(const Module&, const c10::Dict<IValue, IValue>&)>
BackendPreprocessFunction;
using BackendPreprocessFunction = std::function<
c10::IValue(const Module&, const c10::Dict<IValue, IValue>&)>;

Comment on lines +58 to +64
TORCH_API void registerBackendPreprocessFunction(
const std::string& name,
const BackendPreprocessFunction& preprocess);

TORCH_API bool hasBackendPreprocessFunction(const std::string& name);

TORCH_API BackendPreprocessFunction

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these need to be declared with TORCH_API?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's right.
Just registerBackendPreprocessFunction
I removed it from the others.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Afterall, the 3 functions need to be exported with TORCH_API since:

  • backend.h uses registerBackendPreprocessFunction() and backend_init.cpp uses hasBackendPreprocessFunction() and getBackendPreprocessFunction(), AND
  • backend_detail.h/.cpp is in a separate .so (libtorch) from backend.h/backend_init.h/.cpp (libtorch_python)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for following up.

For my own knowledge, what does the macro do, exactly?

Comment on lines 193 to 202
def __create_backend(self):
self.__backend = $name()
)");
def __create_backend(self):
self.__backend = $name()
)");

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intentional formatting changes?

Copy link
Contributor Author

@raziel raziel Feb 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, or so the VSCode said so, but indeed not necessary so I reverted them.

Comment on lines 205 to 214
def __getstate__(self):
return self.__method_compile_spec, self.__processed_module
)",
def __getstate__(self):
return self.__method_compile_spec, self.__processed_module
)",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intentional formatting changes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

torch/csrc/jit/backends/backend_detail.cpp Show resolved Hide resolved
… backend interface"

Enables backend preprocessing to take place outside of the backend interface.

What's new:
* A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction).
* Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction.
* A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key.
* When a BackendPreprocessFunction is used, the  PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module.


Why?:
These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface.
This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary.

What didn't change:
* Everything is backwards compatible:
** The existing "preprocess" method in PyTorchBackendInterface.
** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface.

Longer term, the plan is to refactor existing users to move to the new backend registration.

Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/)

[ghstack-poisoned]
… backend interface"

Enables backend preprocessing to take place outside of the backend interface.

What's new:
* A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction).
* Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction.
* A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key.
* When a BackendPreprocessFunction is used, the  PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module.


Why?:
These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface.
This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary.

What didn't change:
* Everything is backwards compatible:
** The existing "preprocess" method in PyTorchBackendInterface.
** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface.

Longer term, the plan is to refactor existing users to move to the new backend registration.

Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/)

[ghstack-poisoned]
… backend interface"

Enables backend preprocessing to take place outside of the backend interface.

What's new:
* A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction).
* Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction.
* A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key.
* When a BackendPreprocessFunction is used, the  PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module.


Why?:
These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface.
This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary.

What didn't change:
* Everything is backwards compatible:
** The existing "preprocess" method in PyTorchBackendInterface.
** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface.

Longer term, the plan is to refactor existing users to move to the new backend registration.

Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/)

[ghstack-poisoned]
Copy link

@SplitInfinity SplitInfinity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Let's keep an eye on test signals from internal use cases.

… backend interface"

Enables backend preprocessing to take place outside of the backend interface.

What's new:
* A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction).
* Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction.
* A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key.
* When a BackendPreprocessFunction is used, the  PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module.


Why?:
These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface.
This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary.

What didn't change:
* Everything is backwards compatible:
** The existing "preprocess" method in PyTorchBackendInterface.
** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface.

Longer term, the plan is to refactor existing users to move to the new backend registration.

Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/)

[ghstack-poisoned]
raziel pushed a commit that referenced this pull request Feb 6, 2021
…terface

Pull Request resolved: #51757

Enables backend preprocessing to take place outside of the backend interface.

What's new:
* A new definition for backend preprocessing (i.e. BackendPreprocessFunction).
* Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction.
* A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key.
* When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module.


Why?:
These changes are needed to avoid forcing backend preprocessing to be part of the LoweredModule, and in the future be able to eliminate "preprocess" from the PyTorchBackendInterface.
This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary.

What didn't change:
* Everything is backwards compatible:
** The existing "preprocess" method in PyTorchBackendInterface is still there.
** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface.

Longer term, the plan is to refactor existing users to move to the new backend registration.
ghstack-source-id: 121176393

Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/)
… backend interface"

Enables backend preprocessing to take place outside of the backend interface.

What's new:
* A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction).
* Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction.
* A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key.
* When a BackendPreprocessFunction is used, the  PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module.


Why?:
These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface.
This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary.

What didn't change:
* Everything is backwards compatible:
** The existing "preprocess" method in PyTorchBackendInterface.
** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface.

Longer term, the plan is to refactor existing users to move to the new backend registration.

Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/)

[ghstack-poisoned]
raziel pushed a commit that referenced this pull request Feb 6, 2021
…terface

Pull Request resolved: #51757

Enables backend preprocessing to take place outside of the backend interface.

What's new:
* A new definition for backend preprocessing (i.e. BackendPreprocessFunction).
* Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction.
* A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key.
* When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module.


Why?:
These changes are needed to avoid forcing backend preprocessing to be part of the LoweredModule, and in the future be able to eliminate "preprocess" from the PyTorchBackendInterface.
This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary.

What didn't change:
* Everything is backwards compatible:
** The existing "preprocess" method in PyTorchBackendInterface is still there.
** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface.

Longer term, the plan is to refactor existing users to move to the new backend registration.
ghstack-source-id: 121190883

Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/)
@facebook-github-bot
Copy link
Contributor

This pull request has been merged in 9a964ce.

@facebook-github-bot facebook-github-bot deleted the gh/raziel/3/head branch February 9, 2021 15:14
xsacha pushed a commit to xsacha/pytorch that referenced this pull request Mar 31, 2021
…terface (pytorch#51757)

Summary:
Pull Request resolved: pytorch#51757

Enables backend preprocessing to take place outside of the backend interface.

What's new:
* A new definition for backend preprocessing (i.e. BackendPreprocessFunction).
* Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction.
* A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key.
* When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module.

Why?:
These changes are needed to avoid forcing backend preprocessing to be part of the LoweredModule, and in the future be able to eliminate "preprocess" from the PyTorchBackendInterface.
This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary.

What didn't change:
* Everything is backwards compatible:
** The existing "preprocess" method in PyTorchBackendInterface is still there.
** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface.

Longer term, the plan is to refactor existing users to move to the new backend registration.
ghstack-source-id: 121190883

Test Plan:
Updated existing tests (test_backend.py) to use the new registration mechanism.
Verified test ran and passed (in my OSS build).

Reviewed By: iseeyuan

Differential Revision: D26261042

fbshipit-source-id: 0dc378acd5f2ab60fcdc01f7373616d1db961e61
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla signed Merged oncall: jit Add this issue/PR to JIT oncall triage queue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants