Problem running exoplanet example script

[youngr1960]

I have installed Mirage inside a singularity container with all its dependencies, I believe, and the required references files. The container is using the latest stable Debian with Python 3.9. Installation of the Python environment was done using Conda and the installed version of Mirage is 2.2.1. When I run the example script NIRISS_SOSS_exoplanet_example.ipynb to test the installation and/or configuration I continuously get the error(s) below. Has anybody seen this error(s) before and can provide a hint on how to fix it?

2022-03-31 15:06:31,431 - stpipe - INFO - Working on dark file: /opt/data/mirage_data/niriss/darks/raw/NISNIRISSDARK-153451235_11_496_SE_2015-12-11T16h05m20_dms_uncal.fits
2022-03-31 15:06:33,210 - mirage.utils.read_fits.read_datamodel - INFO - Zeroframe in /opt/data/mirage_data/niriss/darks/raw/NISNIRISSDARK-153451235_11_496_SE_2015-12-11T16h05m20_dms_uncal.fits is all zeros. Returning None.
2022-03-31 15:06:33,210 - stpipe - INFO - Zeroframe in /opt/data/mirage_data/niriss/darks/raw/NISNIRISSDARK-153451235_11_496_SE_2015-12-11T16h05m20_dms_uncal.fits is all zeros. Returning None.
2022-03-31 15:06:33,657 - mirage.dark.dark_prep - ERROR - WARNING: cannot transform input None integration into output NISRAPID integration.
2022-03-31 15:06:33,657 - stpipe - ERROR - WARNING: cannot transform input None integration into output NISRAPID integration.
2022-03-31 15:06:33,659 - root - CRITICAL - Traceback (most recent call last):
File "/opt/miniconda3/lib/python3.9/site-packages/mirage/logging/logging_functions.py", line 111, in wrapped
func(*args, **kwargs)
File "/opt/miniconda3/lib/python3.9/site-packages/mirage/dark/dark_prep.py", line 836, in prepare
self.get_base_dark(filename)
File "/opt/miniconda3/lib/python3.9/site-packages/mirage/dark/dark_prep.py", line 454, in get_base_dark
raise ValueError(("Only RAPID, NISRAPID, or FGSRAPID inputs can be translated to a "
ValueError: Only RAPID, NISRAPID, or FGSRAPID inputs can be translated to a different readout pattern

2022-03-31 15:06:33,659 - stpipe - CRITICAL - Traceback (most recent call last):
File "/opt/miniconda3/lib/python3.9/site-packages/mirage/logging/logging_functions.py", line 111, in wrapped
func(*args, **kwargs)
File "/opt/miniconda3/lib/python3.9/site-packages/mirage/dark/dark_prep.py", line 836, in prepare
self.get_base_dark(filename)
File "/opt/miniconda3/lib/python3.9/site-packages/mirage/dark/dark_prep.py", line 454, in get_base_dark
raise ValueError(("Only RAPID, NISRAPID, or FGSRAPID inputs can be translated to a "
ValueError: Only RAPID, NISRAPID, or FGSRAPID inputs can be translated to a different readout pattern

2022-03-31 15:06:33,659 - root - CRITICAL - CRASHED
2022-03-31 15:06:33,659 - stpipe - CRITICAL - CRASHED

Thank you
Richard Young
HPC Systems Engineer
USQ

[hover2pi]

Double check the yaml file has a valid readout pattern. If it's empty or invalid it may return None, which is what your error says.

If that doesn't work, I can take a look. Any thoughts on this @bhilbert4 ?

[bhilbert4]

That seems right to me. Also check that either the dark or linearized_darkfile entry is pointing to a valid dark current file.

[youngr1960]

I will apologise but my knowledge of Mirage is very limited, basically I am attempting to get it running for other users. After following the installation instructions, I have download NIRISS_SOSS_exoplanet_example.ipynb and then the example files from niriss_soss_example_data directory, i.e. niriss_soss_substrip256_clear.yaml and WASP-43_Jmag_8.npy. Checking the niriss_soss_substrip256_clear.yaml file there is the following:

Readout:
readpatt: NISRAPID #Readout pattern (RAPID, BRIGHT2, etc)
ngroup: 6 #Number of groups in integration
nint: 320 #Number of integrations per exposure
nframe: 1
nskip: 0
resets_bet_ints: 1 #Number of detector resets between integrations
array_name: NIS_SUBSTRIP256 #Name of array from pysiaf (NIS_SOSSFULL, NIS_SUBSTRIP256, NIS_SUBSTRIP96)
filter: CLEAR #Filter of simulated data (F090W, F322W2, etc)
pupil: GR700XD #Pupil element for simulated data (CLEAR, GRISMC, etc)

The file mentioned in the yaml file and the error message both do exist in the required paths. The error message was only the last part of the output from running the ipynb file, except for a message about exoctk version unknown but it is installed. The paths in the ipynb file where changed to look for files in the current working directory. As I am working inside a container I have confirmed all the required paths have been set correctly, e.g. MIRAGE_DATA. The next two lines in the yaml file does mention:

Reffiles: #Set to None or leave blank if you wish to skip that step
dark: $MIRAGE_DATA/niriss/darks/raw/NISNIRISSDARK-172500017_15_496_SE_2017-09-07T05h28m22_dms_uncal.fits #Dark current integration used as the base
linearized_darkfile: None # Linearized dark ramp to use as input. Supercedes dark above

Thanks

[hover2pi]

I'll take a look and get back to you ASAP.

[hover2pi]

Hmm @youngr1960 , I'm having trouble reproducing the issue as it runs fine on my machine. Did you install from source or did you use pip? Perhaps try reinstalling? Do you have the same trouble when you use an environment with Python 3.8?

[youngr1960]

Joe I rebuilt a new container which uses python3.8, however this produces the same error. Looking through the dependencies both containers have the following. During the installation, when exoctk is installed, it downgrades some of the dependencies, e.g. astropy and pysiaf, which then have to be upgraded after exoctk is installed. Do you know which packages does the checking of the fits file?, as maybe exoctk downgrades a dependency(ies). asdf>=2.11.0 astropy>=5.0 astroquery>=0.4.6 bokeh>=2.4.2 batman-package>=2.4.8 crds>=11.12.0 grismconf>=1.32 h5py>=3.6.0 hotsoss>=0.1.7 ipython>=8.2.0 jupyter>=1.0.0 jwst>=1.4.6 jwst-backgrounds>=1.1.2 lxml>=4.8.0 matplotlib>=3.5.1 nircam_gsim>=1.60 numpy>=1.22.3 photutils>=1.4.0 pysiaf==0.15.0 pytest>=7.1.1 scipy>=1.8.0 sphinx>=4.5.0 synphot>=1.1.1 webbpsf>=1.0.0 pyyaml>=6.0

…

________________________________________ From: Joe Filippazzo ***@***.***> Sent: Tuesday, 12 April 2022 7:52 AM To: spacetelescope/mirage Cc: Richard Young; Mention Subject: Re: [spacetelescope/mirage] Problem running exoplanet example script (Issue #780) Hmm @youngr1960<https://github.com/youngr1960> , I'm having trouble reproducing the issue as it runs fine on my machine. Did you install from source or did you use pip? Perhaps try reinstalling? Do you have the same trouble when you use an environment with Python 3.8? — Reply to this email directly, view it on GitHub<#780 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AQSWENTG5NN4BGBWYNLBUETVESNJNANCNFSM5SEGPPMQ>. You are receiving this because you were mentioned.Message ID: ***@***.***>

__________________________________________________________________ This email (including any attached files) is confidential and is for the intended recipient(s) only. If you received this email by mistake, please, as a courtesy, tell the sender, then delete this email. The views and opinions are the originator's and do not necessarily reflect those of the University of Southern Queensland. Although all reasonable precautions were taken to ensure that this email contained no viruses at the time it was sent we accept no liability for any losses arising from its receipt. The University of Southern Queensland is a registered provider of education with the Australian Government. (CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081)

[hover2pi]

Yeah, actually exoctk could be the problem. Let me check it out. Thanks for your patience!