convert_gpt.py error on converting new pre-trained models

[sink-chan]

While attempting to convert one of the new pre-trained models (https://the-eye.eu/public/AI/gptneo-release/GPT3_XL/) to pytorch, I get the following error:

torch.nn.modules.module.ModuleAttributeError: 'GPT2LMHeadModel' object has no attribute 'set_output_embeddings'

I've run "pip3 install -r requirements.txt" for the required python libraries.

Running the script without the flag results in the script completing without error, but the resulting model does not produce reasonable output. See comment for more.

[StellaAthena]

This is related to #174 but may not be the same thing.

@sink-chan have you tried converting both models?

[sink-chan]

@StellaAthena No, I've only tried it on the smaller of the two models.

[sink-chan]

Interestingly enough, using the configs in the (now deleted?) scripts folder works on both 1.3B and 2.7B models BUT the resulting pytorch models output pure nonsense:

manifest and I hop or the hand back it)]. Mid. You did Mrs in the sand as ansh houses and you have

When run with the --lm_head flag, the following error is encountered:

torch.nn.modules.module.ModuleAttributeError: 'GPT2LMHeadModel' object has no attribute 'set_output_embeddings'

I'll update the bug accordingly.

[StellaAthena]

Interestingly enough, using the configs in the (now deleted?) scripts folder works on both 1.3B and 2.7B models BUT the resulting pytorch models output pure nonsense:

manifest and I hop or the hand back it)]. Mid. You did Mrs in the sand as ansh houses and you have

When run with the --lm_head flag, the following error is encountered:

torch.nn.modules.module.ModuleAttributeError: 'GPT2LMHeadModel' object has no attribute 'set_output_embeddings'

I'll update the bug accordingly.

The scripts repository was deleted precisely because it’s completely broken. It attempts to convert the TF checkpoint into a pytorch file compatible with the HuggingFace transformers package. We are working with HF to fix this, see huggingface/transformers#10848