Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add nvidia megatron models #10911

Merged
merged 43 commits into from
Apr 8, 2021
Merged

Add nvidia megatron models #10911

Changes from 1 commit
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
9da3853
Add support for NVIDIA Megatron models
jdemouth-nvidia Mar 25, 2021
f943ed0
Add support for NVIDIA Megatron GPT2 and BERT
jdemouth-nvidia Mar 25, 2021
685479d
Update src/transformers/models/megatron_bert/configuration_megatron_b…
jdemouth Mar 29, 2021
e347036
Update src/transformers/models/megatron_bert/configuration_megatron_b…
jdemouth Mar 29, 2021
0af4168
Update src/transformers/models/megatron_bert/configuration_megatron_b…
jdemouth Mar 29, 2021
435c33e
Remove model.half in tests + add "# Copied ..."
jdemouth-nvidia Mar 29, 2021
343f68d
Fix issues
jdemouth-nvidia Mar 31, 2021
6b551fa
Fix Flax/TF tests
jdemouth-nvidia Mar 31, 2021
4236f00
Fix copyright
jdemouth-nvidia Apr 1, 2021
d2c48de
Update src/transformers/models/megatron_bert/configuration_megatron_b…
jdemouth Apr 1, 2021
2f80114
Update src/transformers/models/megatron_bert/configuration_megatron_b…
jdemouth Apr 1, 2021
691466c
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
35c91b8
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
b159513
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
75dbd92
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
ba47704
Update docs/source/model_doc/megatron_bert.rst
jdemouth Apr 1, 2021
7c69cca
Update docs/source/model_doc/megatron_gpt2.rst
jdemouth Apr 1, 2021
ef5a4dd
Update src/transformers/models/megatron_bert/__init__.py
jdemouth Apr 1, 2021
934bc8d
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
e3b4c2b
Update src/transformers/models/megatron_gpt2/convert_megatron_gpt2_ch…
jdemouth Apr 1, 2021
f1efe7a
Update src/transformers/models/megatron_gpt2/convert_megatron_gpt2_ch…
jdemouth Apr 1, 2021
30164e9
Update src/transformers/models/megatron_gpt2/convert_megatron_gpt2_ch…
jdemouth Apr 1, 2021
4b4eb7c
Update src/transformers/models/megatron_bert/convert_megatron_bert_ch…
jdemouth Apr 1, 2021
d20e628
Update src/transformers/models/megatron_bert/convert_megatron_bert_ch…
jdemouth Apr 1, 2021
1b02b4e
Update src/transformers/models/megatron_bert/convert_megatron_bert_ch…
jdemouth Apr 1, 2021
5a2b555
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
19206aa
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
8c7f61b
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
1bf4b51
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
92d461d
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
93096e7
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
11072bc
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
acd1ee8
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
5f616b7
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
5e24d73
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
57ea6d3
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py
jdemouth Apr 1, 2021
74a8205
Resolve most of 'sgugger' comments
jdemouth-nvidia Apr 1, 2021
f155bc4
Fix conversion issue + Run make fix-copies/quality/docs
jdemouth-nvidia Apr 1, 2021
487c5a0
Apply suggestions from code review
LysandreJik Apr 7, 2021
f1d2538
Merge branch 'master' into add-nvidia-megatron-models
LysandreJik Apr 7, 2021
6a4367e
Causal LM & merge
LysandreJik Apr 7, 2021
bae4340
Fix init
LysandreJik Apr 7, 2021
8f7a942
Add CausalLM to last auto class
LysandreJik Apr 8, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update src/transformers/models/megatron_bert/modeling_megatron_bert.py 
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
  • Loading branch information
@jdemouth @sgugger @jdemouth-nvidia
2 people authored and jdemouth-nvidia committed Apr 1, 2021
commit 934bc8d4b09160507068201867dc4020233a7e4c
2 changes: 1 addition & 1 deletion src/transformers/models/megatron_bert/modeling_megatron_bert.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1750,7 +1750,7 @@ def __init__(self, config):
@add_start_docstrings_to_model_forward(MEGATRON_BERT_INPUTS_DOCSTRING.format("batch_size, sequence_length"))
@add_code_sample_docstrings(
tokenizer_class=_TOKENIZER_FOR_DOC,
checkpoint="nvidia/megatron-bert",
checkpoint=_CHECKPOINT_FOR_DOC,
output_type=QuestionAnsweringModelOutput,
config_class=_CONFIG_FOR_DOC,
)
Expand Down