You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
import io
import torch
from safetensors.torch import save_file as safetensors_save
data = {}
tensor_shape = (64, 128, 256)
num_tensors = 16
for i in range(num_tensors):
data[str(i)] = torch.randn(*tensor_shape)
stream = io.BytesIO()
out = safetensors_save(data, stream)
Error:
TypeError: argument 'filename': expected str, bytes or os.PathLike object, not BytesIO
Expected behavior
I've expected serialized checkpoint bytes to be written to buffer.
It would be convenient to do so for:
Further uploading from file object.
Use custom file object, representing database file.
There are only options to serialize checkpoint and get bytes (which is not convenient when checkpoint is large and it is materialized only on master process), and to serialize checkpoint to specified filename.
The text was updated successfully, but these errors were encountered:
System Info
safetensors==0.4.5
torch==2.4.0+cu124
Information
Reproduction
Error:
Expected behavior
I've expected serialized checkpoint bytes to be written to buffer.
It would be convenient to do so for:
There are only options to serialize checkpoint and get bytes (which is not convenient when checkpoint is large and it is materialized only on master process), and to serialize checkpoint to specified filename.
The text was updated successfully, but these errors were encountered: