Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save input cfg file #14

Merged
merged 22 commits into from
Mar 19, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
2df22ff
Fix formatting errors in master
akainth015 Feb 17, 2021
9379aab
Update pre-commit steps and isort configuration
akainth015 Feb 17, 2021
a7f0801
Update CI test command
akainth015 Feb 17, 2021
75e5ca8
Restore original setup.cfg because known third party is not deprecated
akainth015 Feb 17, 2021
1dde0d2
Remove newline before vissl imports
akainth015 Feb 20, 2021
ee15b38
Allow MLP to be specified in one param with any combination of add-ons
akainth015 Feb 25, 2021
346bb1f
Update model configuration files
akainth015 Feb 27, 2021
3655fcd
Revert changes to tests
akainth015 Mar 1, 2021
e211182
Fix MLP network generation
akainth015 Mar 1, 2021
aea3782
Add stdout.json logging, file is not created
akainth015 Feb 23, 2021
c8d918d
Fix issues with unit tests, file is created
akainth015 Feb 25, 2021
3f5e5a7
Convert prior logging to JSON and keep same key names
akainth015 Feb 27, 2021
5939bc6
Revert changes to tests
akainth015 Mar 1, 2021
a01e4e2
Remove unused imports
akainth015 Mar 1, 2021
d0cf82b
Revert changes to test_tasks.py
akainth015 Mar 2, 2021
4d2f827
added function for saving cfg to yaml file
grace-omotoso Mar 5, 2021
27c71fb
added option for saving .yaml files
grace-omotoso Mar 5, 2021
5fbb0d5
added function for saving cfg to yaml file
grace-omotoso Mar 5, 2021
d3bcc07
added option for saving .yaml files
grace-omotoso Mar 5, 2021
fdd013a
Open CFG YAML file in regular mode
akainth015 Mar 16, 2021
994d1b1
removed unused import
grace-omotoso Mar 17, 2021
20a2515
saved attributes to disc after assertion
grace-omotoso Mar 17, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -92,3 +92,4 @@ website/pages/tutorials/*
*-checkpoint.ipynb
**/.ipynb_checkpoints
**/.ipynb_checkpoints/**
train_config.yaml
6 changes: 6 additions & 0 deletions vissl/utils/hydra_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

from omegaconf import DictConfig, OmegaConf
from vissl.config import check_cfg_version
from vissl.utils.io import save_file


class AttrDict(dict):
Expand Down Expand Up @@ -86,6 +87,10 @@ def __dir__(self):
return self.keys()


def save_attrdict_to_disk(cfg, filename="train_config.yaml"):
save_file(cfg, filename)


def convert_to_attrdict(cfg: DictConfig, cmdline_args: List[Any] = None):
"""
Given the user input Hydra Config, and some command line input options
Expand Down Expand Up @@ -117,6 +122,7 @@ def convert_to_attrdict(cfg: DictConfig, cmdline_args: List[Any] = None):
# assert the config and infer
config = cfg.config
assert_hydra_conf(config)
save_attrdict_to_disk(cfg)
return cfg, config


Expand Down
6 changes: 6 additions & 0 deletions vissl/utils/io.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
from urllib.parse import urlparse

import numpy as np
import yaml
from fvcore.common.download import download
from fvcore.common.file_io import PathManager, file_lock
from vissl.utils.slurm import get_slurm_dir
Expand Down Expand Up @@ -65,6 +66,11 @@ def save_file(data, filename):
with PathManager.open(filename, "a") as fopen:
fopen.write(json.dumps(data, sort_keys=True) + "\n")
fopen.flush()
elif file_ext == ".yaml":
with PathManager.open(filename, "w") as fopen:
dump = yaml.dump(data)
fopen.write(dump)
fopen.flush()
else:
raise Exception(f"Saving {file_ext} is not supported yet")
logging.info(f"Saved data to file: {filename}")
Expand Down