-
Notifications
You must be signed in to change notification settings - Fork 6
ItzikMalkiel/MTAdam
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
The implementation of MTAdam can be found in the file mtadam.py. An example of using the MTAdam optimizer can be found in main.py. The main.py file uses MTAdam and compares it to Adam with optimal and suboptimal weights. The main.py file executes a sequence of MNIST experiments. As can be seen in the file, the use of MTAdam is as simple as using Adam, and requires the following steps: (a) initiating the MTAdam optimizer (in a similar way to Adam). (b) keeping the multi-term loss objective decomposed as a sequence of single terms, and sending the sequence as an argument to MTAdam.step(). (c) avoid calling the function loss.backward(), since it is done internally in MTAdam.step(). Dependencies: Pytorch version >= 1.0 tqdm torchvision Files: mtadam.py Main.py If you find the code useful in your research, please consider citing the paper: @article{malkiel2020mtadam, title={MTAdam: Automatic Balancing of Multiple Training Loss Terms}, author={Malkiel, Itzik and Wolf, Lior}, journal={arXiv preprint arXiv:2006.14683}, year={2020} }
About
MTAdam: Automatic Balancing of Multiple Training Loss Terms
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published