Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bitsandbytes - Linear8bitLt integration into transformers models #17901

Merged
merged 91 commits into from
Aug 10, 2022
Merged

bitsandbytes - Linear8bitLt integration into transformers models #17901

merged 91 commits into from
Aug 10, 2022

Commits on Jun 24, 2022

  1. first commit

    @younesbelkada
    younesbelkada committed Jun 24, 2022
    Configuration menu
    Copy the full SHA
    ed1dd12 View commit details
    Browse the repository at this point in the history

  2. correct replace function

    @younesbelkada
    younesbelkada committed Jun 24, 2022
    Configuration menu
    Copy the full SHA
    b9d0da6 View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2022

  1. add final changes 

    - works like charm!
- cannot implement tests yet
- tested
    @younesbelkada
    younesbelkada committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    dd9a464 View commit details
    Browse the repository at this point in the history

  2. clean up a bit

    @younesbelkada
    younesbelkada committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    35e1534 View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2022

  1. add bitsandbytes dependencies

    @younesbelkada
    younesbelkada committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    d01822b View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2022

  1. working version 

    - added import function
- added bitsandbytes utils file
    @younesbelkada
    younesbelkada committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    839c9cd View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of https://github.com/huggingface/transformers in… 

    …to integration-8bit
    @younesbelkada
    younesbelkada committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    93a5ac6 View commit details
    Browse the repository at this point in the history

  3. small fix

    @younesbelkada
    younesbelkada committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    42a6845 View commit details
    Browse the repository at this point in the history

  4. small fix 

    - fix import issue
    @younesbelkada
    younesbelkada committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    97f64f8 View commit details
    Browse the repository at this point in the history

  5. fix import issues

    @younesbelkada
    younesbelkada committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    a1fe7fc View commit details
    Browse the repository at this point in the history

  6. Apply suggestions from code review 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @younesbelkada @sgugger
    younesbelkada and sgugger authored Jul 12, 2022
    Configuration menu
    Copy the full SHA
    05739e3 View commit details
    Browse the repository at this point in the history

  7. refactor a bit 

    - move bitsandbytes utils to utils
- change comments on functions
    @younesbelkada
    younesbelkada committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    7816ef9 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'integration-8bit' of https://github.com/younesbelkada/t… 

    …ransformers into integration-8bit
    @younesbelkada
    younesbelkada committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    1155549 View commit details
    Browse the repository at this point in the history

  9. reformat docstring 

    - reformat docstring on init_empty_weights_8bit
    @younesbelkada
    younesbelkada committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    b222b9a View commit details
    Browse the repository at this point in the history

  10. Update src/transformers/__init__.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @younesbelkada @sgugger
    younesbelkada and sgugger authored Jul 12, 2022
    Configuration menu
    Copy the full SHA
    32f48cd View commit details
    Browse the repository at this point in the history

  11. revert bad formatting

    @younesbelkada
    younesbelkada committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    e116e21 View commit details
    Browse the repository at this point in the history

  12. change to bitsandbytes

    @younesbelkada
    younesbelkada committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    39c46a0 View commit details
    Browse the repository at this point in the history

  13. refactor a bit 

    - remove init8bit since it is useless
    @younesbelkada
    younesbelkada committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    b92c25c View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2022

  1. more refactoring 

    - fixed init empty weights issue
- added threshold param
    @younesbelkada
    younesbelkada committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    3779f5d View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2022

  1. small hack to make it work

    @younesbelkada
    younesbelkada committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    b41c250 View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2022

  1. Merge branch 'main' into integration-8bit

    @younesbelkada
    younesbelkada authored Jul 20, 2022
    Configuration menu
    Copy the full SHA
    311dcbf View commit details
    Browse the repository at this point in the history

  2. Update src/transformers/modeling_utils.py

    @younesbelkada
    younesbelkada authored Jul 20, 2022
    Configuration menu
    Copy the full SHA
    c91a58e View commit details
    Browse the repository at this point in the history

  3. Update src/transformers/modeling_utils.py

    @younesbelkada
    younesbelkada authored Jul 20, 2022
    Configuration menu
    Copy the full SHA
    db16cf8 View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2022

  1. revmoe the small hack

    @younesbelkada
    younesbelkada committed Jul 21, 2022
    Configuration menu
    Copy the full SHA
    be6ce29 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'integration-8bit' of https://github.com/younesbelkada/t… 

    …ransformers into integration-8bit
    @younesbelkada
    younesbelkada committed Jul 21, 2022
    Configuration menu
    Copy the full SHA
    848d64d View commit details
    Browse the repository at this point in the history

  3. modify utils file

    @younesbelkada
    younesbelkada committed Jul 21, 2022
    Configuration menu
    Copy the full SHA
    15a81e0 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2022

  1. make style + refactor a bit

    @younesbelkada
    younesbelkada committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    514758d View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2022

  1. create correctly device map

    @younesbelkada
    younesbelkada committed Jul 25, 2022
    Configuration menu
    Copy the full SHA
    a09e055 View commit details
    Browse the repository at this point in the history

  2. add correct dtype for device map creation

    @younesbelkada
    younesbelkada committed Jul 25, 2022
    Configuration menu
    Copy the full SHA
    387aa1e View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2022

  1. Apply suggestions from code review 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @younesbelkada @sgugger
    younesbelkada and sgugger authored Jul 27, 2022
    Configuration menu
    Copy the full SHA
    7199292 View commit details
    Browse the repository at this point in the history

  2. apply suggestions 

    - remove with torch.grad
- do not rely on Python bool magic!
    @younesbelkada
    younesbelkada committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    a4c19c1 View commit details
    Browse the repository at this point in the history

  3. add docstring 

     - add docstring for new kwargs
    @younesbelkada
    younesbelkada committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    a5cd157 View commit details
    Browse the repository at this point in the history

  4. add docstring 

    - comment `replace_8bit_linear` function
- fix weird formatting
    @younesbelkada
    younesbelkada committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    9bd326b View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2022

  1. - added more documentation 

    - added new utility function for memory footprint tracking
- colab demo to add
    @younesbelkada
    younesbelkada committed Jul 28, 2022
    Configuration menu
    Copy the full SHA
    6e4fee6 View commit details
    Browse the repository at this point in the history

  2. few modifs 

    - typo doc
- force cast into float16 when load_in_8bit is enabled
    @younesbelkada
    younesbelkada committed Jul 28, 2022
    Configuration menu
    Copy the full SHA
    9e121b7 View commit details
    Browse the repository at this point in the history

  3. added colab link

    @younesbelkada
    younesbelkada committed Jul 28, 2022
    Configuration menu
    Copy the full SHA
    a2ac688 View commit details
    Browse the repository at this point in the history

  4. add test architecture + docstring a bit

    @younesbelkada
    younesbelkada committed Jul 28, 2022
    Configuration menu
    Copy the full SHA
    ac370b9 View commit details
    Browse the repository at this point in the history

  5. refactor a bit testing class

    @younesbelkada
    younesbelkada committed Jul 28, 2022
    Configuration menu
    Copy the full SHA
    27e9486 View commit details
    Browse the repository at this point in the history

  6. make style + refactor a bit

    @younesbelkada
    younesbelkada committed Jul 28, 2022
    Configuration menu
    Copy the full SHA
    a0db982 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2022

  1. enhance checks 

    - add more checks
- start writing saving test
    @younesbelkada
    younesbelkada committed Jul 29, 2022
    Configuration menu
    Copy the full SHA
    21bd590 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'integration-8bit' of https://github.com/younesbelkada/t… 

    …ransformers into integration-8bit
    @younesbelkada
    younesbelkada committed Jul 29, 2022
    Configuration menu
    Copy the full SHA
    f1fcf77 View commit details
    Browse the repository at this point in the history

  3. clean up a bit

    @younesbelkada
    younesbelkada committed Jul 29, 2022
    Configuration menu
    Copy the full SHA
    9b81c67 View commit details
    Browse the repository at this point in the history

  4. male style

    @younesbelkada
    younesbelkada committed Jul 29, 2022
    Configuration menu
    Copy the full SHA
    659f427 View commit details
    Browse the repository at this point in the history

  5. add more details on doc

    @younesbelkada
    younesbelkada committed Jul 29, 2022
    Configuration menu
    Copy the full SHA
    147683e View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2022

  1. add more tests 

    - still needs to fix 2 tests
    @younesbelkada
    younesbelkada committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    c2e1918 View commit details
    Browse the repository at this point in the history

  2. replace by "or" 

    - could not fix it from GitHub GUI

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @younesbelkada @sgugger
    younesbelkada and sgugger committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    f6eb945 View commit details
    Browse the repository at this point in the history

  3. refactor a bit testing code + add readme

    @younesbelkada
    younesbelkada committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    ceff43e View commit details
    Browse the repository at this point in the history

  4. make style

    @younesbelkada
    younesbelkada committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    55cec55 View commit details
    Browse the repository at this point in the history

  5. fix import issue

    @younesbelkada
    younesbelkada committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    67bf4fb View commit details
    Browse the repository at this point in the history

  6. Update src/transformers/modeling_utils.py 

    Co-authored-by: Michael Benayoun <mickbenayoun@gmail.com>
    @younesbelkada @michaelbenayoun
    younesbelkada and michaelbenayoun authored Aug 1, 2022
    Configuration menu
    Copy the full SHA
    56e9147 View commit details
    Browse the repository at this point in the history

  7. add few comments

    @younesbelkada
    younesbelkada committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    5a03a86 View commit details
    Browse the repository at this point in the history

  8. add more doctring + make style

    @younesbelkada
    younesbelkada committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    7abb914 View commit details
    Browse the repository at this point in the history

  9. more docstring

    @younesbelkada
    younesbelkada committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    961e57e View commit details
    Browse the repository at this point in the history

  10. raise error when loaded in 8bit

    @younesbelkada
    younesbelkada committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    1326a42 View commit details
    Browse the repository at this point in the history

  11. make style

    @younesbelkada
    younesbelkada committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    9a0051b View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2022

  1. add warning if loaded on CPU

    @younesbelkada
    younesbelkada committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    59f9a5a View commit details
    Browse the repository at this point in the history

  2. add small sanity check

    @younesbelkada
    younesbelkada committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    418690b View commit details
    Browse the repository at this point in the history

  3. fix small comment

    @younesbelkada
    younesbelkada committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    6d93424 View commit details
    Browse the repository at this point in the history

  4. add bitsandbytes on dockerfile

    @younesbelkada
    younesbelkada committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    d428d8d View commit details
    Browse the repository at this point in the history

  5. Improve documentation 

    - improve documentation from comments
    @younesbelkada
    younesbelkada committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    0324f4b View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into integration-8bit

    @younesbelkada
    younesbelkada authored Aug 2, 2022
    Configuration menu
    Copy the full SHA
    af229c4 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2022

  1. add few comments

    @younesbelkada
    younesbelkada committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    70ad8cb View commit details
    Browse the repository at this point in the history

  2. slow tests pass on the VM but not on the CI VM

    @younesbelkada
    younesbelkada committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    1eedb90 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'integration-8bit' of https://github.com/younesbelkada/t… 

    …ransformers into integration-8bit
    @younesbelkada
    younesbelkada committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    163ef77 View commit details
    Browse the repository at this point in the history

  4. Fix merge conflict

    @younesbelkada
    younesbelkada committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    8100d03 View commit details
    Browse the repository at this point in the history

  5. make style

    @younesbelkada
    younesbelkada committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    c9589f6 View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2022

  1. another test should pass on a multi gpu setup

    @younesbelkada
    younesbelkada committed Aug 4, 2022
    Configuration menu
    Copy the full SHA
    eb9a26d View commit details
    Browse the repository at this point in the history

  2. fix bad import in testing file

    @younesbelkada
    younesbelkada committed Aug 4, 2022
    Configuration menu
    Copy the full SHA
    838e2a9 View commit details
    Browse the repository at this point in the history

  3. Fix slow tests 

    - remove dummy batches
- no more CUDA illegal memory errors
    @younesbelkada
    younesbelkada committed Aug 4, 2022
    Configuration menu
    Copy the full SHA
    c4a1e9b View commit details
    Browse the repository at this point in the history

  4. odify dockerfile

    @younesbelkada
    younesbelkada committed Aug 4, 2022
    Configuration menu
    Copy the full SHA
    31fce94 View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2022

  1. Update docs/source/en/main_classes/model.mdx

    @younesbelkada
    younesbelkada authored Aug 5, 2022
    Configuration menu
    Copy the full SHA
    3e4a2a4 View commit details
    Browse the repository at this point in the history

  2. Update Dockerfile

    @younesbelkada
    younesbelkada authored Aug 5, 2022
    Configuration menu
    Copy the full SHA
    fdf37b3 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2022

  1. Update model.mdx

    @younesbelkada
    younesbelkada authored Aug 7, 2022
    Configuration menu
    Copy the full SHA
    53e0b2e View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2022

  1. Update Dockerfile

    @younesbelkada
    younesbelkada authored Aug 8, 2022
    Configuration menu
    Copy the full SHA
    91364c4 View commit details
    Browse the repository at this point in the history

  2. Apply suggestions from code review

    @younesbelkada
    younesbelkada authored Aug 8, 2022
    Configuration menu
    Copy the full SHA
    5ea8976 View commit details
    Browse the repository at this point in the history

  3. few modifications 

    - lm head can stay on disk/cpu
- change model name so that test pass
    @younesbelkada
    younesbelkada committed Aug 8, 2022
    Configuration menu
    Copy the full SHA
    8b72d08 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into integration-8bit

    @younesbelkada
    younesbelkada committed Aug 8, 2022
    Configuration menu
    Copy the full SHA
    32a4863 View commit details
    Browse the repository at this point in the history

  5. change test value 

    - change test value to the correct output
- torch bmm changed to baddmm in bloom modeling when merging
    @younesbelkada
    younesbelkada committed Aug 8, 2022
    Configuration menu
    Copy the full SHA
    c6c139f View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2022

  1. Merge remote-tracking branch 'upstream/main' into integration-8bit

    @younesbelkada
    younesbelkada committed Aug 9, 2022
    Configuration menu
    Copy the full SHA
    3d3224f View commit details
    Browse the repository at this point in the history

  2. modify installation guidelines

    @younesbelkada
    younesbelkada committed Aug 9, 2022
    Configuration menu
    Copy the full SHA
    0d5bc2b View commit details
    Browse the repository at this point in the history

  3. Apply suggestions from code review 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @younesbelkada @sgugger
    younesbelkada and sgugger authored Aug 9, 2022
    Configuration menu
    Copy the full SHA
    bc8f332 View commit details
    Browse the repository at this point in the history

  4. Apply suggestions from code review 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @younesbelkada @sgugger
    younesbelkada and sgugger authored Aug 9, 2022
    Configuration menu
    Copy the full SHA
    5adcadc View commit details
    Browse the repository at this point in the history

  5. Apply suggestions from code review 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @younesbelkada @sgugger
    younesbelkada and sgugger authored Aug 9, 2022
    Configuration menu
    Copy the full SHA
    bb00e7a View commit details
    Browse the repository at this point in the history

  6. replace nby name

    @younesbelkada
    younesbelkada committed Aug 9, 2022
    Configuration menu
    Copy the full SHA
    630b4f7 View commit details
    Browse the repository at this point in the history

  7. merge load_in_8bit and low_cpu_mem_usage

    @younesbelkada
    younesbelkada committed Aug 9, 2022
    Configuration menu
    Copy the full SHA
    a925641 View commit details
    Browse the repository at this point in the history

  8. first try - keep the lm head in full precision

    @younesbelkada
    younesbelkada committed Aug 9, 2022
    Configuration menu
    Copy the full SHA
    279b8c4 View commit details
    Browse the repository at this point in the history

  9. better check 

    - check the attribute `base_model_prefix` instead of computing the number of parameters
    @younesbelkada
    younesbelkada committed Aug 9, 2022
    Configuration menu
    Copy the full SHA
    e49a2ea View commit details
    Browse the repository at this point in the history

  10. added more tests

    @younesbelkada
    younesbelkada committed Aug 9, 2022
    Configuration menu
    Copy the full SHA
    5718d78 View commit details
    Browse the repository at this point in the history

  11. Update src/transformers/utils/bitsandbytes.py 

    Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
    @younesbelkada @sgugger
    younesbelkada and sgugger authored Aug 9, 2022
    Configuration menu
    Copy the full SHA
    a40667a View commit details
    Browse the repository at this point in the history

  12. Merge branch 'integration-8bit' of https://github.com/younesbelkada/t… 

    …ransformers into integration-8bit
    @younesbelkada
    younesbelkada committed Aug 9, 2022
    Configuration menu
    Copy the full SHA
    61faa28 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2022

  1. improve documentation 

    - fix typos for installation
- change title in the documentation
    @younesbelkada
    younesbelkada committed Aug 10, 2022
    Configuration menu
    Copy the full SHA
    36fbbd2 View commit details
    Browse the repository at this point in the history