napoler / reformer-chinese Public

Notifications You must be signed in to change notification settings
Fork 3
Star 16

reformer-pytorch中文版本，简单高效的生成模型。类似GPT2的效果

www.terrychan.org/reformer-chinese/

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 98 Commits
cache		cache
docs		docs
notebook		notebook
reformer_chinese		reformer_chinese
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
demo.py		demo.py
generate.py		generate.py
generate_sent.py		generate_sent.py
generate_test.py		generate_test.py
pre_process_data.py		pre_process_data.py
requirements.txt		requirements.txt
run.sh		run.sh
save_mode_cpu.py		save_mode_cpu.py
t.py		t.py
train.py		train.py
train_sent.py		train_sent.py

Repository files navigation

reformer-chinese

reformer-pytorch中文版本

训练

python3 train.py --raw --epochs 2 --min_length 64 --batch_size 10 --stride 768 --num_pieces 10 --output_dir /kaggle/model/ --pretrained_model /kaggle/working/model/ --tokenizer_path /kaggle/working/model/vocab.txt --model_config /kaggle/working/model/config.json

python3 train.py --epochs 2 --device cpu --batch_size 4 --gradient_accumulation 2 --lr 5e-05

python3 train.py --epochs 2 --device cpu --batch_size 4 --gradient_accumulation 2 --lr 0.01 --num_pieces 100

python3 train.py --raw --epochs 2 --device cpu --batch_size 4 --gradient_accumulation 2 --lr 0.001 --num_pieces 10 --dim 128 --depth 12 --full_attn_thres 128

python3 train.py --epochs 2 --device cpu --batch_size 4 --gradient_accumulation 2 --lr 0.001 --num_pieces 10 --dim 128 --depth 12 --full_attn_thres 128