Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dynamic topology phase IV #481

Open
wants to merge 35 commits into
base: master
Choose a base branch
from

Conversation

tokusanya
Copy link
Collaborator

Functionality for reduction operations on groups

Copy link
Member

@gsjaardema gsjaardema left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. A few questions/comments/suggestions.

packages/seacas/libraries/ioss/src/Ioss_Region.C Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_Region.C Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_Region.C Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_Region.C Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_Region.C Outdated Show resolved Hide resolved
Copy link
Member

@gsjaardema gsjaardema left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lots of good stuff here. I didn't make it through everything yet, but tried to get a good overview. I think it looks good overall and is evolving toward where I think we want to go...

packages/seacas/libraries/ioss/src/Ioss_DatabaseIO.h Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_DatabaseIO.h Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_DatabaseIO.h Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_DatabaseIO.h Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_DatabaseIO.h Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_DynamicTopology.C Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_DynamicTopology.C Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_DynamicTopology.C Outdated Show resolved Hide resolved
Copy link
Member

@gsjaardema gsjaardema left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few suggestions for changes. Nothing required that I remember... Looks good although lots of code, so not sure I fully covered all of it...

packages/seacas/libraries/ioss/src/Ioss_ChangeSetFactory.h Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_ChangeSet.C Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_ChangeSet.C Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_ChangeSet.C Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_ChangeSet.h Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_DynamicTopology.C Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_DynamicTopology.C Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/Ioss_DynamicTopology.h Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/exodus/Ioex_ChangeSet.h Outdated Show resolved Hide resolved
packages/seacas/libraries/ioss/src/exodus/Ioex_ChangeSet.h Outdated Show resolved Hide resolved
tokusanya and others added 7 commits September 11, 2024 14:25
Bumps [step-security/harden-runner](https://github.com/step-security/harden-runner) from 2.9.1 to 2.10.0.
- [Release notes](https://github.com/step-security/harden-runner/releases)
- [Commits](step-security/harden-runner@5c7944e...446798f)

---
updated-dependencies:
- dependency-name: step-security/harden-runner
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* EXODUS: Add support for lossy compression via netCDF quantize method

* IOSS: Add support for lossy compression specifying NSD for netCDF quantize method

* EXODUS: Update version date

* EXODUS: Add some error checking on nsd values

* EXODUS: Check if Quantization supported; include in config output

* EXODUS: Cleaning up new compression support

* EXODUS: Fix error handling code

* EXODUS: Better error checking on compression

* IOSS: Support for zstd compression

* IOSS: Handle building with no netCDF-4 capability

* IOSS: Change gold files to netcdf-3 so work without hdf5-supported netCDF

* CI: See if can figure out intel build failure

* CI: Fix hdf5 version warning message

* CI: Turn on plugins; support new syntax

* EXODUS: Only flush stderr if written to

Signed-off-by: Greg Sjaardema <gdsjaar@sandia.gov>

* Tweak compression additions

* CI: Fix intel build script syntax error

* EXODUS: Support new netCDF with changed _FillValue

* IOSS: Fix so latest fmt version works

* IOSS: Catalyst API 2 (sandialabs#463)

* IOSS: Catalyst API 2

Added support to database for using my_processor and
processor_count IOSS properties when operating in
serial parallel mode and reading Conduit files.

* IOSS: Catalyst API 2

Changed Conduit serial parallel test to use
files from the one MPI rank can.ex2 test.

---------

Co-authored-by: Greg Sjaardema <gsjaardema@gmail.com>

* Fix export symbols (sandialabs#465)

* IOSS: Catalyst API 2 (sandialabs#466)

Create the cell_ids and cell_node_ids fields for
a StructuredBlock when asked for by a reading
application, if these fields are not already stored in
the Conduit data. Uses the get_cell_ids() and
get_cell_node_ids() methods on StructuredBlock.

* CI: Remove hdf5-1.8 testing

* IOSS: Catalyst API 2 (sandialabs#463)

* IOSS: Catalyst API 2

Added support to database for using my_processor and
processor_count IOSS properties when operating in
serial parallel mode and reading Conduit files.

* IOSS: Catalyst API 2

Changed Conduit serial parallel test to use
files from the one MPI rank can.ex2 test.

---------

Co-authored-by: Greg Sjaardema <gsjaardema@gmail.com>

* NEM_SPREAD: Add missing include file

* CI: Enable netCDF quantize option

* CI: Support newest netCDF; fmt

* Enable new compression options

* EXODUS: Fix non-hdf5 build

* EXODUS: Do not commit the kluge to repository

* IOSS: See if can fix spack ci build

* CI: Spack build should not run on PR

* CI: Spack build on master only [ci skip]

* CI: Fix spack workflow syntax [ci skip]

* IOSS: Add progress output to chain/face generation

* IOSS: Reduce Face memory

* EXODUS: Better control over exodus verbosity

* EXODUS:, IOSS:  Add bzip2 compression support

* Further testing of non-lossy compression with plugins

* CI: Update cmake-sems to work with current sems

* Default netcdf should be 4.9.2

Signed-off-by: Greg Sjaardema <gsjaardema@gmail.com>

* Clean up copyright dates

Signed-off-by: Greg Sjaardema <gsjaardema@gmail.com>

* Clean up copyright dates

Signed-off-by: Greg Sjaardema <gsjaardema@gmail.com>

* accidentally added to pr; removed

Signed-off-by: Greg Sjaardema <gsjaardema@gmail.com>

---------

Signed-off-by: Greg Sjaardema <gdsjaar@sandia.gov>
Signed-off-by: Greg Sjaardema <gsjaardema@gmail.com>
Co-authored-by: Greg Sjaardema <gdsjaar@sandia.gov>
Co-authored-by: tjotaha <tjotaha@sandia.gov>
Co-authored-by: Spiros Tsalikis <spiros.tsalikis@kitware.com>
@tokusanya tokusanya marked this pull request as ready for review September 30, 2024 20:28
@tokusanya
Copy link
Collaborator Author

@gdsjaar I have removed the draft status and marked it for review ... let me know if there is still anything required

@gsjaardema gsjaardema assigned gsjaardema and tokusanya and unassigned gsjaardema Oct 1, 2024
Copy link
Member

@gsjaardema gsjaardema left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I have some questions about the lock/no-lock functions.

I'm fine with looking at this after merging or as part of this PR whichever you prefer.

@@ -43,17 +43,39 @@ int file_exists( const Ioss::ParallelUtils &util,
return file.parallel_exists(util.communicator(), message);
}

bool internal_decomp_specified(const Ioss::PropertyManager& props)
std::string get_decomposition_property(const Ioss::PropertyManager &properties,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks good. At some point we maybe move to the Ioss::Utils class to avoid the code dupication, but good for now.

Signed-off-by: Greg Sjaardema <gsjaardema@gmail.com>
I think this is the correct change (deleting this block of code) since `handle_output_file` is returning a  `m_exodusFilePtr` and sets`m_groupName`, so that is redundant with calling `ex_get_group_id` and even  worse, `ex_get_group_id` is looking for a child group of itself...

I'm not sure whether the code in this block is needed for an input file though...
@gsjaardema
Copy link
Member

@tokusanya Can you take a look at my latest commit and see if it makes sense. (The ifdef'd code would be completely removed in the final; just left it there so easier to see what was there...

@tokusanya
Copy link
Collaborator Author

@gdsjaar I think I see what is happening ... the code I put in to automatically open up the first child group in finalize_file_open() and handle_output_file() may have been interfering with those calls you #ifdef'd out

@gsjaardema
Copy link
Member

@gdsjaar I think I see what is happening ... the code I put in to automatically open up the first child group in finalize_file_open() and handle_output_file() may have been interfering with those calls you #ifdef'd out

Yes, and when I put in code that zero'd out the file pointer if the change group call failed, it then caught the error which was always happening, but just using the old file pointer...

@gsjaardema
Copy link
Member

@tokusanya I am good with you merging whenever you feel that it is ready. You should have permission to merge.

@tokusanya
Copy link
Collaborator Author

@tokusanya I am good with you merging whenever you feel that it is ready. You should have permission to merge.

Ok, I'm working on one last issue that might involve a quick commit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants