Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Refactor] extract block forward, backward and inference into a separate file #435

Merged
merged 6 commits into from
Aug 7, 2023

Conversation

justheuristic
Copy link
Collaborator

@justheuristic justheuristic commented Aug 4, 2023

This PR does not change any functionality. It merely moves stuff around.

Problem: handler is huge and difficult to navigate. It is about to become more complicated once we support general peft

Suggesion: move all math (forward and backward pass; grad w.r.t. prompt, etc) to a separate file. Handler handles all networking and queues, but not the math; block_methods is responsible for math only.

List of changes:

  • handler.py/_rpc_forward became block_methods/rpc_forward
  • handler.py/_rpc_backward became block_methods/rpc_backward
  • the math bits of rpc_inference were extracted into block_methods/iterate_rpc_inference

I took care to avoid any other changes

Copy link
Collaborator

@borzunov borzunov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR.

Please rename it to block_functions.py and test that pushes are successfully sent (this is not covered by test and can break due to can_push refactor). Then it's good to go!

@justheuristic justheuristic merged commit ac9b546 into main Aug 7, 2023
7 checks passed
@justheuristic justheuristic deleted the extract_block_methods branch August 7, 2023 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants