Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add BigBirdPegasus #10991

Merged
Merged
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
84b8ab3
init bigbird pegasus
thevasudevgupta Mar 30, 2021
5c0307b
add debugging nb ; update config
thevasudevgupta Apr 2, 2021
ee47797
init conversion
thevasudevgupta Apr 2, 2021
677b72a
update conversion script
thevasudevgupta Apr 4, 2021
94861d6
complete conversion script
thevasudevgupta Apr 5, 2021
7529516
init forward()
thevasudevgupta Apr 7, 2021
2ce502b
complete forward()
thevasudevgupta Apr 10, 2021
72807a7
add tokenizer
thevasudevgupta Apr 10, 2021
0f00e43
add some slow tests
thevasudevgupta Apr 12, 2021
3bee313
commit current
thevasudevgupta Apr 13, 2021
2f72512
fix copies
thevasudevgupta Apr 13, 2021
af47e3c
add docs
thevasudevgupta Apr 13, 2021
bbe710f
add conversion script for bigbird-roberta-summarization
thevasudevgupta Apr 13, 2021
7b1ecce
remove TODO
thevasudevgupta Apr 16, 2021
1de6954
small fixups
thevasudevgupta Apr 17, 2021
fb10be0
correct tokenizer
patrickvonplaten Apr 19, 2021
c78bf6f
add bigbird core for now
thevasudevgupta Apr 22, 2021
38c7638
fix config
thevasudevgupta Apr 23, 2021
0b87a70
fix more
thevasudevgupta Apr 30, 2021
c403796
revert pegasus-tokenizer back
thevasudevgupta Apr 30, 2021
30a586a
make style
thevasudevgupta Apr 30, 2021
9618f8b
everything working for pubmed; yayygit status
thevasudevgupta Apr 30, 2021
53d99b5
complete tests finally
thevasudevgupta Apr 30, 2021
a83e681
remove bigbird pegasus tok
patrickvonplaten May 4, 2021
789d170
correct tokenizer
patrickvonplaten May 4, 2021
324cb89
correct tests
patrickvonplaten May 4, 2021
24f7931
add tokenizer files
patrickvonplaten May 4, 2021
94326e1
merge conflicts
patrickvonplaten May 4, 2021
0657b81
finish make style
patrickvonplaten May 4, 2021
0aaddf4
fix test
patrickvonplaten May 4, 2021
0c11023
update
patrickvonplaten May 4, 2021
2b23619
make style
patrickvonplaten May 4, 2021
53d3a45
fix tok utils base file
patrickvonplaten May 4, 2021
2d64dda
make fix-copies
patrickvonplaten May 4, 2021
5157815
clean a bit
thevasudevgupta May 4, 2021
cd2a5b6
small update
thevasudevgupta May 4, 2021
237c118
fix some suggestions
thevasudevgupta May 4, 2021
2d217e3
add to readme
thevasudevgupta May 4, 2021
11d9c2e
fix a bit, clean tests
thevasudevgupta May 5, 2021
b556fc9
fix more tests
thevasudevgupta May 5, 2021
a066db7
Update src/transformers/__init__.py
patrickvonplaten May 6, 2021
9c0b312
Update src/transformers/__init__.py
patrickvonplaten May 6, 2021
4d706f7
make fix-copies
patrickvonplaten May 6, 2021
9a52751
complete attn switching, auto-padding left
thevasudevgupta May 6, 2021
b13e76a
make style
patrickvonplaten May 6, 2021
97e1f71
fix auto-padding test
thevasudevgupta May 6, 2021
0ba203a
make style
thevasudevgupta May 6, 2021
0a85bb2
fix batched attention tests
patrickvonplaten May 6, 2021
89e9d5c
Merge branch 'add_bigbird_pegasus' of https://github.com/vasudevgupta…
patrickvonplaten May 6, 2021
7d2cae9
put tolerance at 1e-1 for stand-alone decoder test
patrickvonplaten May 6, 2021
34c3a63
fix docs
patrickvonplaten May 6, 2021
c229de1
fix tests
patrickvonplaten May 6, 2021
1d8d60d
correct slow tokenizer conversion
patrickvonplaten May 6, 2021
dfb74ea
Apply suggestions from code review
thevasudevgupta May 6, 2021
ca77ad3
complete remaining suggestions
thevasudevgupta May 6, 2021
2eb7563
fix test
patrickvonplaten May 7, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix config
  • Loading branch information
thevasudevgupta committed Apr 23, 2021
commit 38c76385f27e6d550b2e97295c92c4777dbe59fb
Original file line number Diff line number Diff line change
Expand Up @@ -133,15 +133,15 @@ def __init__(
init_std=0.02,
decoder_start_token_id=2,
classifier_dropout=0.0,
scale_embedding=False,
scale_embedding=True,
gradient_checkpointing=False,
pad_token_id=0,
bos_token_id=2,
eos_token_id=1,
attention_type="block_sparse", # only for encoder
block_size=64,
num_random_blocks=3,
use_bias=True,
use_bias=False,
**kwargs
):
super().__init__(
Expand Down