Remove Tetra as an option for Extruded meshes #888

bartgol · 2023-01-14T00:50:05Z

This PR removes Tetra as an option for Extruded meshes. Hence, if the basal element shape is Triangle, the volume element will always be a Wedge.

This is somewhat needed in the DOF manager PR, where it has already been assumed. Having 2+ elements in each layer makes retrieval of dofs a bit more complicated.

I ran all the tests (except pyAlbany), and got some failures in FO_GIS tests, with responses/sensitivities being a bit off. I tried to increase quadrature degree and make NOX tolerance stricted, but it only improves a bit. I suspect that our current tolerances were not tight enough to make the results "converged", so there's no hope to get a match. I'm going to verify this next week, by taking a test in master, and bumping up quadrature degree while making NOX tolerances stricter. This way, I should get an "exact" response/sensitivity from master, which should match the ones I get in this branch.

@mperego I am assuming that, provided that the quadrature/tolerances are strict enough, all the responses at the top/bottom with Tets (master) should match the values with Wedges (branch). I feel like the FE approx on the top/bottom should be the same for Tets/Wedges, but I am not expert enough in what Intrepid2 does with Wedges. Is my assumption correct? If not, then it's possible we cannot get an exact match with current master, no matter how strict the convergence is.

…e basal shape

mperego · 2023-01-16T21:25:19Z

@bartgol I don't think we can expect that using wedges will produce the same results as using tets. The discretizations are different (piecewise linear for tets, piecewise bilinear for wedges), so they would converge to the same solution only as the mesh size goes to zero. If you do not see large differences between the tets and wedges solution, I would just re-bless the tests.

bartgol · 2023-01-16T21:52:39Z

@bartgol I don't think we can expect that using wedges will produce the same results as using tets. The discretizations are different (piecewise linear for tets, piecewise bilinear for wedges), so they would converge to the same solution only as the mesh size goes to zero. If you do not see large differences between the tets and wedges solution, I would just re-bless the tests.

The differences are all like the following:

Response Test 0: 1.098837995820e+08 != 1.076679182380e+08 (rel 1.000000000000e-04 abs 1.000000000000e-04)
Sensitivity (0,0), Test 0,0: 1.826738903461e+07 != 1.867976080160e+07 (rel 1.000000000000e-04 abs 1.000000000000e-04)
---
Response Test 0: 1.089173687130e+08 != 1.066976120250e+08 (rel 1.000000000000e-06 abs 1.000000000000e-06)
Sensitivity (0,0), Test 0,0: 1.720834249957e+07 != 1.753719284188e+07 (rel 1.000000000000e-06 abs 1.000000000000e-06)
----
Response Test 0: 1.111601023583e+08 != 1.091294526860e+08 (rel 1.000000000000e-04 abs 1.000000000000e-04)
Sensitivity (0,0), Test 0,0: 1.849596738867e+07 != 1.882621076480e+07 (rel 1.000000000000e-04 abs 1.000000000000e-04)
---
Piro Analysis Test Two Norm: 9.69190977257 != 9.66706 (rel 1e-05 abs 1e-05)
---
Piro Analysis Test Two Norm: 90.6074319631 != 85.2913 (rel 1e-05 abs 1e-05)

They are all of the order of 2-3%, except for the analysis tests, which I assume are more sensitive to convergence, since there are two nonlinear solvers (NOX+ROL). If you think this kind of diffs could be attributed simply to the difference of discretizations, I can bless them.

mperego · 2023-01-16T21:54:21Z

Yes, they look reasonable. I would bless them.

…tests

tests/landIce/FO_GIS/input_fo_gis_analysis_stiffening.yaml

bartgol · 2023-01-17T20:42:30Z

tests/landIce/FO_GIS/input_fo_gis_beta_smb.yaml

    Relative Tolerance: 1.00000000000000004e-04
  Regression For Response 1:
    Absolute Tolerance: 1.00000000000000004e-04
-    Test Value: 3.907406428441e+03
+    Test Value: 6.422658118363e+03


@mperego This is very different though. It's almost a 2x. Is it worrisome?

This is a lot, at the same time I think that the flux div evaluator, used for the SMB response has been tested more for wedges than for tets. Maybe you can check if increasing the cubature rule makes any difference.

tests/landIce/FO_GIS/input_fo_gis_analysis_stiffening.yaml

tests/landIce/FO_GIS/input_fo_gis_analysis_stiffening_memT.yaml

mperego · 2023-01-17T22:31:59Z

tests/landIce/FO_GIS/input_fo_gis_beta_smb.yaml

    Relative Tolerance: 1.00000000000000004e-04
  Regression For Response 1:
    Absolute Tolerance: 1.00000000000000004e-04
-    Test Value: 3.907406428441e+03
+    Test Value: 6.422658118363e+03


This is a lot, at the same time I think that the flux div evaluator, used for the SMB response has been tested more for wedges than for tets. Maybe you can check if increasing the cubature rule makes any difference.

mperego · 2023-01-17T22:34:06Z

src/disc/Albany_DiscretizationFactory.cpp

-              "Tetrahedron" ? 3 : 1; // number of 3D cells in one extruded 2D cell
-          const int num_cells_per_column = num_cells_per_column_per_layer * disc_params->get<int>("NumLayers");
-          int basal_ws_size = extruded_ws_size / num_cells_per_column;
+          int basal_ws_size = extruded_ws_size / disc_params->get<int>("NumLayers");


This might create issues with the MPAS mesh struct. Need to check this branch with MPAS before you merge.

mperego · 2023-01-21T01:45:47Z

I created a PR in Compass to switch the tests using Tets to use Wedges.
@jewatkins when are you planning to do the same with the performance tests?
Note that when you switch to wedges, you need to set the Cubature Degree in the Discretization list to 4 (or more) and you'll have to fix the regression values.

jewatkins · 2023-01-21T02:07:26Z

I created a PR in Compass to switch the tests using Tets to use Wedges. @jewatkins when are you planning to do the same with the performance tests? Note that when you switch to wedges, you need to set the Cubature Degree in the Discretization list to 4 (or more) and you'll have to fix the regression values.

I think I need to wait until blake is back up. blake has more tests than weaver so I can't get the new regression values without blake. If this is a blocker, I could try setting up performance tests on attaway and get the new values there? We should probably do that anyways if we want to run large simulations there.

bartgol · 2023-01-21T02:38:49Z

Can't we take care of blake performance tests once we get it back? We're pretty convinced to switch to Wedge anyways, so I think we can adjust the performance regression tests later.

jewatkins · 2023-01-23T20:23:12Z

okay maybe what I'll go ahead and do is update the weaver performance tests to use wedge once nightly albany is building again. Then this should be safe to push. Once blake is back up, I'll do the same for the rest of the tests.

mperego

I think we can merge this. Both Compass and Ali_perf_tests moved to Wedges.

bartgol · 2023-02-01T20:50:31Z

Running one last round of tests on mappy, to ensure the conflicts resolution during master merge didn't change things.

jewatkins · 2023-02-01T20:52:20Z

FYI there's been a couple of pretty large regressions in performance on blake. One around 10/21 and another while blake was down (12/12 was the last point). I changed the test names to no longer have "tet" or "wdg" so it won't be captured in future plots (plots are constructed based on test name). But I saved the old plot. I'll try to investigate a bit.

…a-extruded-meshes

bartgol · 2023-02-01T22:27:58Z

All tests pass on mappy. Merging.

These tests are only enabled when ENABLE_MESH_DEPENDS_ON_PARAMETERS is ON.

bartgol added 2 commits January 13, 2023 17:32

Remove Tetra as an option for Extruded meshes

12163fe

Without Tetra for extruded meshes, we can infer element shape from th…

3e0891f

…e basal shape

bartgol self-assigned this Jan 14, 2023

bartgol added 2 commits January 13, 2023 17:56

Remove now unused exo mesh square_tri.exo

24e28fc

Remove another unused mesh

7d0e00d

bartgol added 3 commits January 17, 2023 13:34

Disambiguate exo output files in FO_GIS folder

ba1a492

Removed unused FO_GIS yaml files

a4a8db0

Bless change in regression values due to elem shape change in FO_GIS …

43ff592

…tests

bartgol commented Jan 17, 2023

View reviewed changes

tests/landIce/FO_GIS/input_fo_gis_analysis_stiffening.yaml Outdated Show resolved Hide resolved

bartgol commented Jan 17, 2023

View reviewed changes

mperego reviewed Jan 17, 2023

View reviewed changes

bartgol added 2 commits January 17, 2023 16:22

With wedges, use cub degree 4 in volume, 3 on lateralside

b612e27

Re-bless some tests due to cub degree changes

f7d8d5d

This was referenced Jan 25, 2023

Remove all tet cases sandialabs/ali-perf-tests#6

Merged

Remove SFAD4 and SFAD8 builds #900

Merged

Remove all tet cases sandialabs/ali-perf-data#1

Merged

mperego approved these changes Feb 1, 2023

View reviewed changes

Merge remote-tracking branch 'origin/master' into bartgol/remove-tetr…

cd026d6

…a-extruded-meshes

bartgol force-pushed the bartgol/remove-tetra-extruded-meshes branch from 759acb1 to cd026d6 Compare February 1, 2023 20:55

bartgol merged commit 6ac26c4 into master Feb 1, 2023

bartgol deleted the bartgol/remove-tetra-extruded-meshes branch February 1, 2023 22:28

jewatkins mentioned this pull request Feb 2, 2023

Remove "Element Shape: Wedge" from input files sandialabs/ali-perf-tests#8

Merged

mperego added a commit that referenced this pull request Feb 6, 2023

Fix test values after switching to Wedges (left over from #888)

c3995d3

These tests are only enabled when ENABLE_MESH_DEPENDS_ON_PARAMETERS is ON.

mperego mentioned this pull request Feb 6, 2023

Seg Fault when trying to use new MueLu preconditioners for Enthalpy problems #862

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove Tetra as an option for Extruded meshes #888

Remove Tetra as an option for Extruded meshes #888

bartgol commented Jan 14, 2023

mperego commented Jan 16, 2023

bartgol commented Jan 16, 2023

mperego commented Jan 16, 2023

bartgol Jan 17, 2023

mperego Jan 17, 2023

mperego Jan 17, 2023

mperego Jan 17, 2023

mperego commented Jan 21, 2023

jewatkins commented Jan 21, 2023

bartgol commented Jan 21, 2023

jewatkins commented Jan 23, 2023

mperego left a comment

bartgol commented Feb 1, 2023

jewatkins commented Feb 1, 2023

bartgol commented Feb 1, 2023

Remove Tetra as an option for Extruded meshes #888

Remove Tetra as an option for Extruded meshes #888

Conversation

bartgol commented Jan 14, 2023

mperego commented Jan 16, 2023

bartgol commented Jan 16, 2023

mperego commented Jan 16, 2023

bartgol Jan 17, 2023

Choose a reason for hiding this comment

mperego Jan 17, 2023

Choose a reason for hiding this comment

mperego Jan 17, 2023

Choose a reason for hiding this comment

mperego Jan 17, 2023

Choose a reason for hiding this comment

mperego commented Jan 21, 2023

jewatkins commented Jan 21, 2023

bartgol commented Jan 21, 2023

jewatkins commented Jan 23, 2023

mperego left a comment

Choose a reason for hiding this comment

bartgol commented Feb 1, 2023

jewatkins commented Feb 1, 2023

bartgol commented Feb 1, 2023