-
Notifications
You must be signed in to change notification settings - Fork 22.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enables backend preprocessing to take place outside of the backend interface #51757
Conversation
…terface Enables backend preprocessing to take place outside of the backend interface. What's new: * A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction). * Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction. * A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key. * When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module. Why?: These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface. This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary. What didn't change: * Everything is backwards compatible: ** The existing "preprocess" method in PyTorchBackendInterface. ** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface. Longer term, the plan is to refactor existing users to move to the new backend registration. Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/) [ghstack-poisoned]
💊 CI failures summary and remediationsAs of commit e9dcc34 (more details on the Dr. CI page):
🕵️ 4 new failures recognized by patternsThe following CI failures do not appear to be due to upstream breakages: pytorch_linux_xenial_py3_6_gcc5_4_build (1/4)Step: "(Optional) Merge target branch" (full log | diagnosis details | 🔁 rerun)
|
Job | Step | Action |
---|---|---|
pytorch_linux_bionic_py3_8_gcc9_coverage_test2 | Run tests | 🔁 rerun |
pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test2 | Run tests | 🔁 rerun |
Extra GitHub checks: 1 failed
- Failed: GitHub Actions -
clang-tidy
ci.pytorch.org: 1 failed
This comment was automatically generated by Dr. CI (expand for details).
Follow this link to opt-out of these comments for your Pull Requests.Please report bugs/suggestions to the (internal) Dr. CI Users group.
… backend interface" Enables backend preprocessing to take place outside of the backend interface. What's new: * A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction). * Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction. * A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key. * When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module. Why?: These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface. This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary. What didn't change: * Everything is backwards compatible: ** The existing "preprocess" method in PyTorchBackendInterface. ** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface. Longer term, the plan is to refactor existing users to move to the new backend registration. Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/) [ghstack-poisoned]
… backend interface" Enables backend preprocessing to take place outside of the backend interface. What's new: * A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction). * Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction. * A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key. * When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module. Why?: These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface. This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary. What didn't change: * Everything is backwards compatible: ** The existing "preprocess" method in PyTorchBackendInterface. ** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface. Longer term, the plan is to refactor existing users to move to the new backend registration. Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/) [ghstack-poisoned]
… backend interface" Enables backend preprocessing to take place outside of the backend interface. What's new: * A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction). * Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction. * A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key. * When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module. Why?: These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface. This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary. What didn't change: * Everything is backwards compatible: ** The existing "preprocess" method in PyTorchBackendInterface. ** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface. Longer term, the plan is to refactor existing users to move to the new backend registration. Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/) [ghstack-poisoned]
…terface Pull Request resolved: #51757 Enables backend preprocessing to take place outside of the backend interface. What's new: * A new definition for backend preprocessing (i.e. BackendPreprocessFunction). * Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction. * A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key. * When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module. Why?: These changes are needed to avoid forcing backend preprocessing to be part of the LoweredModule, and in the future be able to eliminate "preprocess" from the PyTorchBackendInterface. This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary. What didn't change: * Everything is backwards compatible: ** The existing "preprocess" method in PyTorchBackendInterface is still there. ** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface. Longer term, the plan is to refactor existing users to move to the new backend registration. ghstack-source-id: 121097330 Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice! I don't see any major issues, but I'd like a response to some of my suggestions and questions before I accept.
const BackendPreprocessFunction& preprocess) { | ||
TORCH_CHECK( | ||
!detail::hasBackendPreprocessFunction(name), | ||
"BackendPreprocessFunction for backend ", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"BackendPreprocessFunction for backend ", | |
"Preprocessing function for backend ", |
test/cpp/jit/test_backend.cpp
Outdated
@@ -67,8 +67,15 @@ class TestBackend : public PyTorchBackendInterface { | |||
} | |||
}; | |||
|
|||
c10::IValue backendPreprocessFunction( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
c10::IValue backendPreprocessFunction( | |
c10::IValue preprocess( |
typedef std::function< | ||
c10::IValue(const Module&, const c10::Dict<IValue, IValue>&)> | ||
BackendPreprocessFunction; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typedef std::function< | |
c10::IValue(const Module&, const c10::Dict<IValue, IValue>&)> | |
BackendPreprocessFunction; | |
using BackendPreprocessFunction = std::function< | |
c10::IValue(const Module&, const c10::Dict<IValue, IValue>&)>; |
TORCH_API void registerBackendPreprocessFunction( | ||
const std::string& name, | ||
const BackendPreprocessFunction& preprocess); | ||
|
||
TORCH_API bool hasBackendPreprocessFunction(const std::string& name); | ||
|
||
TORCH_API BackendPreprocessFunction |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do these need to be declared with TORCH_API
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's right.
Just registerBackendPreprocessFunction
I removed it from the others.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Afterall, the 3 functions need to be exported with TORCH_API since:
- backend.h uses registerBackendPreprocessFunction() and backend_init.cpp uses hasBackendPreprocessFunction() and getBackendPreprocessFunction(), AND
- backend_detail.h/.cpp is in a separate .so (libtorch) from backend.h/backend_init.h/.cpp (libtorch_python)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for following up.
For my own knowledge, what does the macro do, exactly?
def __create_backend(self): | ||
self.__backend = $name() | ||
)"); | ||
def __create_backend(self): | ||
self.__backend = $name() | ||
)"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Intentional formatting changes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, or so the VSCode said so, but indeed not necessary so I reverted them.
def __getstate__(self): | ||
return self.__method_compile_spec, self.__processed_module | ||
)", | ||
def __getstate__(self): | ||
return self.__method_compile_spec, self.__processed_module | ||
)", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Intentional formatting changes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed.
… backend interface" Enables backend preprocessing to take place outside of the backend interface. What's new: * A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction). * Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction. * A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key. * When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module. Why?: These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface. This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary. What didn't change: * Everything is backwards compatible: ** The existing "preprocess" method in PyTorchBackendInterface. ** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface. Longer term, the plan is to refactor existing users to move to the new backend registration. Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/) [ghstack-poisoned]
… backend interface" Enables backend preprocessing to take place outside of the backend interface. What's new: * A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction). * Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction. * A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key. * When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module. Why?: These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface. This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary. What didn't change: * Everything is backwards compatible: ** The existing "preprocess" method in PyTorchBackendInterface. ** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface. Longer term, the plan is to refactor existing users to move to the new backend registration. Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/) [ghstack-poisoned]
… backend interface" Enables backend preprocessing to take place outside of the backend interface. What's new: * A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction). * Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction. * A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key. * When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module. Why?: These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface. This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary. What didn't change: * Everything is backwards compatible: ** The existing "preprocess" method in PyTorchBackendInterface. ** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface. Longer term, the plan is to refactor existing users to move to the new backend registration. Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/) [ghstack-poisoned]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice! Let's keep an eye on test signals from internal use cases.
… backend interface" Enables backend preprocessing to take place outside of the backend interface. What's new: * A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction). * Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction. * A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key. * When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module. Why?: These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface. This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary. What didn't change: * Everything is backwards compatible: ** The existing "preprocess" method in PyTorchBackendInterface. ** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface. Longer term, the plan is to refactor existing users to move to the new backend registration. Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/) [ghstack-poisoned]
…terface Pull Request resolved: #51757 Enables backend preprocessing to take place outside of the backend interface. What's new: * A new definition for backend preprocessing (i.e. BackendPreprocessFunction). * Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction. * A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key. * When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module. Why?: These changes are needed to avoid forcing backend preprocessing to be part of the LoweredModule, and in the future be able to eliminate "preprocess" from the PyTorchBackendInterface. This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary. What didn't change: * Everything is backwards compatible: ** The existing "preprocess" method in PyTorchBackendInterface is still there. ** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface. Longer term, the plan is to refactor existing users to move to the new backend registration. ghstack-source-id: 121176393 Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/)
… backend interface" Enables backend preprocessing to take place outside of the backend interface. What's new: * A new definition for backend preprocessing is defined (i.e. BackendPreprocessFunction). * Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction. * A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key. * When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module. Why?: These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface. This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary. What didn't change: * Everything is backwards compatible: ** The existing "preprocess" method in PyTorchBackendInterface. ** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface. Longer term, the plan is to refactor existing users to move to the new backend registration. Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/) [ghstack-poisoned]
…terface Pull Request resolved: #51757 Enables backend preprocessing to take place outside of the backend interface. What's new: * A new definition for backend preprocessing (i.e. BackendPreprocessFunction). * Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction. * A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key. * When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module. Why?: These changes are needed to avoid forcing backend preprocessing to be part of the LoweredModule, and in the future be able to eliminate "preprocess" from the PyTorchBackendInterface. This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary. What didn't change: * Everything is backwards compatible: ** The existing "preprocess" method in PyTorchBackendInterface is still there. ** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface. Longer term, the plan is to refactor existing users to move to the new backend registration. ghstack-source-id: 121190883 Differential Revision: [D26261042](https://our.internmc.facebook.com/intern/diff/D26261042/)
This pull request has been merged in 9a964ce. |
…terface (pytorch#51757) Summary: Pull Request resolved: pytorch#51757 Enables backend preprocessing to take place outside of the backend interface. What's new: * A new definition for backend preprocessing (i.e. BackendPreprocessFunction). * Registration of the backend's PyTorchBackendInterface interface implementation is augmented to take the BackendPreprocessFunction. * A new registry is created to handle the BackendPreprocessFunction functions, using the backend's name as key. * When a BackendPreprocessFunction is used, the PyTorchBackendInterface's "preprocess" method is not added to the LoweredModule. Instead, the BackendPreprocessFunction is called and its output used to set the LoweredModule's __processed_module. Why?: These changes are needed to avoid forcing backend preprocessing to be part of the LoweredModule, and in the future be able to eliminate "preprocess" from the PyTorchBackendInterface. This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary. What didn't change: * Everything is backwards compatible: ** The existing "preprocess" method in PyTorchBackendInterface is still there. ** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface. Longer term, the plan is to refactor existing users to move to the new backend registration. ghstack-source-id: 121190883 Test Plan: Updated existing tests (test_backend.py) to use the new registration mechanism. Verified test ran and passed (in my OSS build). Reviewed By: iseeyuan Differential Revision: D26261042 fbshipit-source-id: 0dc378acd5f2ab60fcdc01f7373616d1db961e61
Stack from ghstack:
Enables backend preprocessing to take place outside of the backend interface.
What's new:
Why?:
These changes are needed to avoid forcing "preprocessing" part of the LoweredModule, and in the future of PyTorchBackendInterface.
This is important for Mobile use cases where "preprocess" can take the bulk of the compilation process, and thus contain code dependencies that we do not want to bring (or cannot bring) to the Mobile binary.
What didn't change:
** The existing "preprocess" method in PyTorchBackendInterface.
** When backend registration is done without the BackendPreprocessFunction, as before, things work the same way: "preprocess" is added to LoweredModule, and invoked through the module's instance of the backend interface.
Longer term, the plan is to refactor existing users to move to the new backend registration.
Differential Revision: D26261042