Model card issue#1125 #1129
Open
Model card issue#1125 #1129
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Updates to address [this issue](#1125) - Addition of training regime in the annotated model card to keep this doc and the template in sync. - Defined training_regime, along with examples
Sentence rephrasing
EziOzoani
requested review from
pcuenca,
osanseviero,
davanstrien and
meg-huggingface
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
pcuenca
reviewed
Nov 24, 2023
| @@ -158,6 +158,9 @@ _Write 1-2 sentences on what the training data is. Ideally this links to a Datas | |||
|
|
|||
| ## Training Procedure [optional] | |||
|
|
|||
| _When you want to know what hardware you'll need to fine-tune a model, consider the following factors: the number of parameters in the model and the training regime you plan to use._ | |||
|
|
|||
| _e.g A model with 3B parameters and fp32 precision format needs at least 48GB of GPU memory, while bf16 requires at least 24GB of memory with Amphere or higher hardware. Mixed pf16 requires at least 54GB of GPU memory._ | |||
There was a problem hiding this comment.
Suggested change
| _e.g A model with 3B parameters and fp32 precision format needs at least 48GB of GPU memory, while bf16 requires at least 24GB of memory with Amphere or higher hardware. Mixed pf16 requires at least 54GB of GPU memory._ | |
| _e.g A model with 3B parameters and fp32 precision format needs at least 48GB of GPU memory, while bf16 requires at least 24GB of memory with Ampere or higher hardware. Mixed fp16 requires at least 54GB of GPU memory._ |
There was a problem hiding this comment.
These numbers sound a bit high to me. In any case, they depend on a number of factors like optimizer choice. Should the recommended optimizer be a part of the training_regime data?
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
|
is this PR still in process, @EziOzoani? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updates to address this issue