Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support FullyShardedDataParallel #24

Open
gpauloski opened this issue Mar 12, 2023 · 0 comments
Open

Support FullyShardedDataParallel #24

gpauloski opened this issue Mar 12, 2023 · 0 comments
Assignees
@gpauloski
Labels
enhancement New features or improvements to existing functionality
Milestone
v0.1.1

Comments

@gpauloski
Copy link
Owner

Describe the Request

Support FullyShardedDataParallel for training larger models or on memory constrained hardware. We'll need to consider how the user passes configuration options to FDSP as there are a lot.

Sample Code

No response

Additional Code or Information

No response
@gpauloski gpauloski added the enhancement New features or improvements to existing functionality label Mar 12, 2023
@gpauloski gpauloski self-assigned this Mar 12, 2023
@gpauloski gpauloski modified the milestones: v0.0.1, v0.0.2 Mar 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gpauloski gpauloski

Labels
enhancement New features or improvements to existing functionality
Projects
None yet
Milestone
v0.1.1
Development

No branches or pull requests
1 participant
@gpauloski