QCSchema coordination, testing, and export #237

loriab · 2020-09-04T06:53:57Z

Description

This reconciles some details of models that had gotten out of sync with qcsk. It makes the schema generatable from the models. It dumps some examples of schema from the existing test cases. It patches up some mypy errors. Finally, it starts setting up some coordination with qcsk so that PRs to qcel (on upstream) that propose to change the schema get notified over in qcsk, addressing MolSSI/QCSchema#68 . This coordinates with MolSSI/QCSchema#77

Changelog description

export models to JSON Schema with make qcschema
generate example json from tests and test against exported schema with pytest --validate qcelemental/
mypy fixes incl. dynamic provenance. bump pydantic to v1.5
testing configuration and GHA coordination with QCSchema repo
material changes to models
description string and formatting changes to models
a0 as Bohr abbreviation. test basis sets as str or float
some GHA details aren't polished yet.
- to have permissions to push a diff branch to qcsk, model-changing PRs here need to be on upstream, not forks
- need to be able to detect no-change PRs so they're still allowed from forks
- I turned off CI checks to non-master to avoid duplicate checks, and now the Required ones aren't satisfied

Status

Code base linted
Ready to go

codecov · 2020-09-04T06:55:41Z

Codecov Report

Merging #237 (b970ecb) into master (217e650) will decrease coverage by 0.24%.
The diff coverage is 89.50%.

loriab · 2020-09-08T23:25:08Z

@dgasmith, a few changes to qcel that you should probably glance over to see if they ring any alarm bells. particularly the first bullet.

any subtle reason exclude_none and exclude_defaults weren't options before? https://github.com/MolSSI/QCElemental/pull/237/files#diff-7a9af1eb4f18dc820783bf210d2cfc95
prov version and routine default going to string https://github.com/MolSSI/QCElemental/pull/237/files#diff-0baf84b1436ced317794f35442380396R22-R26
model.basis can now be full basis https://github.com/MolSSI/QCElemental/pull/237/files#diff-0baf84b1436ced317794f35442380396R44
prov and error are moving to default_factory https://github.com/MolSSI/QCElemental/pull/237/files#diff-c1df0b88317a0c0331bea59a1793059cR274, https://github.com/MolSSI/QCElemental/pull/237/files#diff-f178293574dc070baaac6713e52806beR495
time to drop old mp2 name unless that'd be trouble for qcf https://github.com/MolSSI/QCElemental/pull/237/files#diff-f178293574dc070baaac6713e52806beR108
I added a0 to pint https://github.com/MolSSI/QCElemental/pull/237/files#diff-a0d915622c0e7d268558ba8b1f5a3216R52 b/c a_0^2 seems harder to separate than a0^2. I'd like Eh in addition to E_h, too; both are in psi. thoughts?
it's a little sad that a straight .json() from qcel won't validate so need to exclude the Nones. any troubling implications you see? https://github.com/MolSSI/QCElemental/pull/237/files#diff-37eecce1557470bdbfa6da71286b3090R66

dgasmith

I don't think exclude_defaults was an option at the time.
Thats fine, but I did like in the past where it forced you to add these.
Sounds good, but will require a lot of checks inside QCEngine.
Very much needed.
The mp2 issues isn't linking at the moment, I cannot see what you are referring to.
The aliases are fine, though I'm not sure E_h will work as expected. I don't remember if there are odd rules around this.
The .json link doesn't appear to be working.

dgasmith · 2020-09-09T14:56:26Z

setup.py

@@ -31,7 +31,7 @@
        package_data={'': [os.path.join('qcelemental', 'data', '*.json')]},
        setup_requires=[] + pytest_runner,
        python_requires='>=3.6',
-        install_requires=['numpy >= 1.12.0', 'pint >= 0.10.0', 'pydantic >= 1.0.0'],
+        install_requires=["numpy >= 1.12.0", "pint >= 0.10.0", "pydantic >= 1.5.0"],


Pydantic 1.6.0 has a bad regression with nested models, recommend skipping. Fixed in 1.6.1.

dgasmith · 2020-09-09T15:01:15Z

qcelemental/models/__init__.py

+        AtomicResultProperties,
+        BasisSet,
+        Molecule,
+        Provenance,


Optimization?

Optimization was never added over at qcsk, so I was aiming for a 1:1 changeset. Agree that it'd be good to get OptimizationInput/Result formalized over there soon.

loriab · 2020-09-09T15:52:19Z

I don't think exclude_defaults was an option at the time.

✔️

Thats fine, but I did like in the past where it forced you to add these.

I don't think it was, though. It's been None, not .... https://github.com/MolSSI/QCElemental/blob/master/qcelemental/models/common_models.py#L23-L25 The latter I would have left alone as being consistent with qcsk's all three of creator/version/routine being required. Glad to switch to ... instead of "".

Sounds good, but will require a lot of checks inside QCEngine.

yes, indeed

Very much needed.

✔️

The mp2 issues isn't linking at the moment, I cannot see what you are referring to.

Maybe this works better https://github.com/MolSSI/QCElemental/blob/master/qcelemental/models/results.py#L77-L79, mp2_total_correlation_energy

The aliases are fine, though I'm not sure E_h will work as expected. I don't remember if there are odd rules around this.

Ok. I haven't explored Datum with these aliases either, so was trying for nominal consistency

The .json link doesn't appear to be working.

fp.write(instance.json(exclude_unset=True, exclude_none=True)) aka https://github.com/MolSSI/QCElemental/pull/237/files#diff-37eecce1557470bdbfa6da71286b3090R66

dgasmith

Thanks for the links, it all looks great!

loriab · 2020-09-09T17:24:07Z

It turns out a0/a_0/Eh/E_h are already in pint c. v0.10 hgrecco/pint#811

python -c "exec(\"import pathlib, qcelemental\nfor md in qcelemental.models.qcschema_models():\n\tmfile = (pathlib.Path('qcschema') / md.__name__).with_suffix('.schema')\n\twith open(mfile, 'w') as fp:\n\t\tfp.write(md.schema_json(indent=None))\")" python -c "exec(\"import json, pathlib, pydantic, qcelemental\nwith open((pathlib.Path('qcschema') / 'QCSchema').with_suffix('.schema'), 'w') as fp:\n\tjson.dump(pydantic.schema.schema(qcelemental.models.qcschema_models(), title='QCSchema'), fp, indent=4)\")"

…ma pytest --validate

This was referenced Sep 4, 2020

provenance default for schema #233

Closed

Schemalint3 #236

Closed

GitHub Action btwn QCEl and QCSk #218

Closed

qcschema generating, testing, and reconciliation #235

Closed

loriab force-pushed the schemalint13 branch from 1d77ee1 to aadfdf0 Compare September 8, 2020 18:48

dgasmith reviewed Sep 9, 2020

View reviewed changes

dgasmith approved these changes Sep 9, 2020

View reviewed changes

loriab added 9 commits January 29, 2021 12:43

qcsk: generate example json from tests and test against exported sche…

16e2b7b

…ma pytest --validate

qcsk: mypy fixes incl. dynamic provenance. bump pydantic to v1.5

d3fb4e5

qcsk: testing configuration and GHA coordination with QCSchema repo

75758a1

qcsk: material changes to models

1573804

qcsk: description string and formatting changes to models

1bf66dc

qcsk: a0 as Bohr abbreviation. test basis sets as str or float

10742c8

patch-ups after review

81a2a0c

rebase and put off QCSchema.yml GHZ

4d7df19

loriab force-pushed the schemalint13 branch from b8957f1 to 4d7df19 Compare January 29, 2021 18:52

CI on py39 and changelog

b970ecb

loriab merged commit 7fb730e into MolSSI:master Jan 30, 2021

loriab deleted the schemalint13 branch January 30, 2021 03:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

QCSchema coordination, testing, and export #237

QCSchema coordination, testing, and export #237

loriab commented Sep 4, 2020 •

edited

Loading

codecov bot commented Sep 4, 2020 •

edited

Loading

loriab commented Sep 8, 2020

dgasmith left a comment

dgasmith Sep 9, 2020

dgasmith Sep 9, 2020

loriab Sep 9, 2020

loriab commented Sep 9, 2020

dgasmith left a comment

loriab commented Sep 9, 2020

QCSchema coordination, testing, and export #237

QCSchema coordination, testing, and export #237

Conversation

loriab commented Sep 4, 2020 • edited Loading

Description

Changelog description

Status

codecov bot commented Sep 4, 2020 • edited Loading

Codecov Report

loriab commented Sep 8, 2020

dgasmith left a comment

Choose a reason for hiding this comment

dgasmith Sep 9, 2020

Choose a reason for hiding this comment

dgasmith Sep 9, 2020

Choose a reason for hiding this comment

loriab Sep 9, 2020

Choose a reason for hiding this comment

loriab commented Sep 9, 2020

dgasmith left a comment

Choose a reason for hiding this comment

loriab commented Sep 9, 2020

loriab commented Sep 4, 2020 •

edited

Loading

codecov bot commented Sep 4, 2020 •

edited

Loading