Skip to content

Windows compile of bitsandbytes for use in text-generation-webui.

License

Notifications You must be signed in to change notification settings

jllllll/bitsandbytes-windows-webui

Repository files navigation

bitsandbytes-windows-webui

Wheels will be published in the releases section going forward --->

This is due to git file size limits restricting what can be uploaded in the main repo.
The wheels seen in the repo above may be outdated and are only kept to not break current setups.


Windows compile of bitsandbytes for use in text-generation-webui.
Compiled for CUDA 11.1-11.8.
0.38.1+ additionally includes support for 12.0 and 12.1.
0.39.1+ additionally includes support for 12.2.

0.35.X and 0.39.0 does not support cards with compute less than 7.0.
Other versions do.

Credit to Adrian Popescu for the initial conversion to Windows at: https://github.com/acpopescu/bitsandbytes/tree/cmake_windows

The latest version available from here can be installed with the following command:

python -m pip install bitsandbytes --prefer-binary --extra-index-url=https://jllllll.github.io/bitsandbytes-windows-webui

You may need to change the command if the latest version is not yet available in this repo:

python -m pip install scipy
python -m pip install bitsandbytes --index-url=https://jllllll.github.io/bitsandbytes-windows-webui

You can install a specific version with:

python -m pip install bitsandbytes==<version> --prefer-binary --extra-index-url=https://jllllll.github.io/bitsandbytes-windows-webui

List of available versions:

python -m pip index versions bitsandbytes --index-url=https://jllllll.github.io/bitsandbytes-windows-webui

Wheels can be manually downloaded from: https://github.com/jllllll/bitsandbytes-windows-webui/releases/tag/wheels

All wheels are compiled using GitHub Actions.

About

Windows compile of bitsandbytes for use in text-generation-webui.

Resources

License

Stars

Watchers

Forks

Packages

No packages published