Merge branch 'master' of https://github.com/yifding/Distributed_BERT …

…into bert_fine_tuning

README.md

@@ -69,6 +69,18 @@ Running BERT on nodes with 4 GPUs each.
 
 ## Notice and tips
 > loading BERT data takes a while. 
+
+## Known issues
+- [ ] currently not supporting continue training
+- [ ] mnist datasets download does not support multiple GPUs
+
+## future patch
+- [ ] bert processing pipeline not included
+- [ ] interface of datasets/transformers not included
+- [ ] hetseq not supporting download from pip
+- [ ] evaluation separate/combined not included
+- [ ] fp16 support
+
 ## License
 this repository is MIT-licensed. It is created based on [fairseq](https://github.com/pytorch/fairseq), 
 [NVIDIA-BERT](https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/LanguageModeling/BERT), and