Skip to content
This repository has been archived by the owner on Feb 25, 2022. It is now read-only.

Support conversion to pytorch for GPT Neo #186

Closed
sink-chan opened this issue Mar 31, 2021 · 2 comments
Closed

Support conversion to pytorch for GPT Neo #186

sink-chan opened this issue Mar 31, 2021 · 2 comments
Labels
feature request A feature that isn't implemented yet.

Comments

@sink-chan
Copy link

Now that this PR has been merged (huggingface/transformers#10848) it should be technically possible to convert a GPT Neo model to Pytorch. However, the configs published with the models do not seem to be compatible with the new scripts.

This may be as simple as updating the README with instructions on which config to use when attempting the conversion.
@sink-chan sink-chan added the feature request A feature that isn't implemented yet. label Mar 31, 2021
@StellaAthena
Copy link
Member

This is much more than "technically possible," the PyTorch checkpoints we released on HuggingFace were generated using their conversion code. What goes wrong when you run it?

TBH, this might be better settled on their repo than ours.

@StellaAthena
Copy link
Member

Closing due to lack of activity.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
feature request A feature that isn't implemented yet.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@StellaAthena @sink-chan