New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support wildcard for source files in convert utilities #399

Merged

guillaume-be merged 2 commits into main from convert_wildcard_support

Jul 2, 2023

Owner

guillaume-be commented Jun 25, 2023

Support wildcard for source file name in the conversion tool. Inputs now supported:
- single file
- list of files
- single pattern
- list of patterns
Weights are now offloaded to the numpy output file at the end of each pytorch file parsed to limit memory footprint

The matching files will be opened and appended to a single Rust-compatible set of weights (.ot).
Fixes #342

guillaume-be added 2 commits

June 25, 2023 09:31


          Support wildcard for source files

a1e4bea


          - Support offloading at end of every file for conversion

20878ee

guillaume-be merged commit 107fb21 into main

11 checks passed

guillaume-be deleted the convert_wildcard_support branch

July 2, 2023 07:36

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment