Skip to content

Commit

Permalink
Update longt5.mdx (huggingface#18634)
Browse files Browse the repository at this point in the history
  • Loading branch information
flozi00 authored and amyeroberts committed Aug 17, 2022
1 parent b53ef28 commit 1769f66
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/source/en/model_doc/longt5.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Tips:
- [`LongT5ForConditionalGeneration`] is an extension of [`T5ForConditionalGeneration`] exchanging the traditional
encoder *self-attention* layer with efficient either *local* attention or *transient-global* (*tglobal*) attention.
- Unlike the T5 model, LongT5 does not use a task prefix. Furthermore, it uses a different pre-training objective
inspired by the pre-training of `[PegasusForConditionalGeneration]`.
inspired by the pre-training of [`PegasusForConditionalGeneration`].
- LongT5 model is designed to work efficiently and very well on long-range *sequence-to-sequence* tasks where the
input sequence exceeds commonly used 512 tokens. It is capable of handling input sequences of a length up to 16,384 tokens.
- For *Local Attention*, the sparse sliding-window local attention operation allows a given token to attend only `r`
Expand Down

0 comments on commit 1769f66

Please sign in to comment.