Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Geoserver importer NO_CRS task status doesn't trigger the CRS selection step #9016

Closed
giohappy opened this issue Mar 31, 2022 · 10 comments
Closed
Assignees
Labels
3.3.x 4.0.x major A high priority issue which might affect a lot of people or large parts of the codebase

Comments

@giohappy
Copy link
Contributor

When an invalid .prj file is sent to Geoserver the importer blocks in a PENDING status with the first task in NO_CRS status.
GeoNode's upload keeps returning the PENDING status but with an empty resume_url, which is expected to be populated to let the client redirect to the CRS selection (as with the srs step).

The result is a stale import that can only be deleted with the delete button (delete_url).

This was tested on master with this shepafile.
I suspect it also affects 3.3.x.

@giohappy giohappy added major A high priority issue which might affect a lot of people or large parts of the codebase 3.3.x 4.0.x labels Mar 31, 2022
@giohappy giohappy changed the title Geoserver importer NO_CRS tast stauts doesn't trigger the upload srs step Geoserver importer NO_CRS task stauts doesn't trigger the upload srs step Mar 31, 2022
@giohappy giohappy changed the title Geoserver importer NO_CRS task stauts doesn't trigger the upload srs step Geoserver importer NO_CRS task status doesn't trigger the upload srs step Mar 31, 2022
@giohappy giohappy changed the title Geoserver importer NO_CRS task status doesn't trigger the upload srs step Geoserver importer NO_CRS task status doesn't trigger the CRS selection step Mar 31, 2022
afabiani pushed a commit that referenced this issue Apr 1, 2022
github-actions bot pushed a commit that referenced this issue Apr 6, 2022
#9021)

* [Fixes #9016] Geoserver importer NO_CRS task status doesn't trigger the CRS selection step

* [CircleCI] Fix tests

* [CircleCI] Fix tests

* [Minor

* [Minor] Improve syntax and tests

* [Minor] Improve syntax and tests

* [CircleCI] Fix tests

* [CircleCI] Fix tests

* [CircleCI] Fix tests

* [CircleCI] Fix tests

* [CircleCI] Fix tests

* [Pep8] Flake8 issues fixes

Co-authored-by: mattiagiupponi <51856725+mattiagiupponi@users.noreply.github.com>
afabiani pushed a commit that referenced this issue Apr 6, 2022
#9021) (#9046)

* [Fixes #9016] Geoserver importer NO_CRS task status doesn't trigger the CRS selection step

* [CircleCI] Fix tests

* [CircleCI] Fix tests

* [Minor

* [Minor] Improve syntax and tests

* [Minor] Improve syntax and tests

* [CircleCI] Fix tests

* [CircleCI] Fix tests

* [CircleCI] Fix tests

* [CircleCI] Fix tests

* [CircleCI] Fix tests

* [Pep8] Flake8 issues fixes

Co-authored-by: mattiagiupponi <51856725+mattiagiupponi@users.noreply.github.com>

Co-authored-by: Alessio Fabiani <alessio.fabiani@geo-solutions.it>
Co-authored-by: mattiagiupponi <51856725+mattiagiupponi@users.noreply.github.com>
afabiani pushed a commit that referenced this issue Apr 6, 2022
afabiani pushed a commit that referenced this issue Apr 6, 2022
#9057)

* [Backport 3.3.x][Fixes #9016] Geoserver importer NO_CRS task status doesn't trigger the CRS selection step

* [CircleCI] Fix test cases
@giohappy
Copy link
Contributor Author

The problem is still there @afabiani. It can be reproduced on master with this sample tiff

@giohappy giohappy reopened this Apr 13, 2022
@afabiani
Copy link
Member

@giohappy I don't think this one is related to this issue, or at least it does not depend from the Importer... I'll check
This is the error I get

Could not find any valid Time Regex for the Mosaic files.
Traceback (most recent call last):
  File "/mnt/c/Work/Code/geonode/geonode/upload/views.py", line 705, in view
    resp = _steps[step](req, upload_session)
  File "/mnt/c/Work/Code/geonode/geonode/upload/views.py", line 161, in save_step_view
    import_session, upload = save_step(
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/mnt/c/Work/Code/geonode/geonode/upload/upload.py", line 427, in save_step
    raise e
  File "/mnt/c/Work/Code/geonode/geonode/upload/upload.py", line 340, in save_step
    target_store, files_to_upload = utils.import_imagemosaic_granules(
  File "/mnt/c/Work/Code/geonode/geonode/upload/utils.py", line 722, in import_imagemosaic_granules
    raise GeneralUploadException(detail=_("Could not find any valid Time Regex for the Mosaic files."))
geonode.upload.api.exceptions.GeneralUploadException: Could not find any valid Time Regex for the Mosaic files.
Internal Server Error: /api/v2/uploads/upload
"POST /api/v2/uploads/upload HTTP/1.1" 500 114

while uploading a dataset without a valid CRS resulting in

image

@afabiani
Copy link
Member

I guess this is a side-effect of this one

image

@giohappy
Copy link
Contributor Author

@afabiani I've tested the sample I attached both locally and master. Both cases I get the progress bar stuck and the importer session task[0] with NO_CRS.

@afabiani
Copy link
Member

we should explictly send "mosaic=False" from the upload form

@giohappy
Copy link
Contributor Author

No, we should assume mosaic False by default, not the reverse. Also considering that support for mosaic is a not well defined status right now.

@giohappy
Copy link
Contributor Author

giohappy commented Apr 13, 2022

Found the difference. You were trying to upload the zip file, which triggers the mosaic workflow, because of the field.
I will open an issue for that, but when the tiff is loaded directly the problme of the progress stuck can be reproduced

EDIT: issue created #9097

@afabiani
Copy link
Member

About the raster files, currently the workflow does not envisage passing through the srs_step, this is why it is not blocking...

@giohappy
Copy link
Contributor Author

@afabiani either we support it, or we return an error in case the importer is stuck with an invalid CRS.

@afabiani
Copy link
Member

yes, I agree @giohappy

github-actions bot pushed a commit that referenced this issue Apr 14, 2022
…he CRS selection step (#9099)

* [Fixes #9016] Geoserver importer NO_CRS task status doesn't trigger the CRS selection step

* [Fixes #9016] Geoserver importer NO_CRS task status doesn't trigger the CRS selection step

* [Fixes #9016] Geoserver importer NO_CRS task status doesn't trigger the CRS selection step

* [CircleCI] Fix Tests

* [CircleCI] Fix Docker Build

* [CircleCI] Fix Tests

* [CircleCI] Fix Tests
afabiani pushed a commit that referenced this issue Apr 14, 2022
…he CRS selection step (#9099) (#9103)

* [Fixes #9016] Geoserver importer NO_CRS task status doesn't trigger the CRS selection step

* [Fixes #9016] Geoserver importer NO_CRS task status doesn't trigger the CRS selection step

* [Fixes #9016] Geoserver importer NO_CRS task status doesn't trigger the CRS selection step

* [CircleCI] Fix Tests

* [CircleCI] Fix Docker Build

* [CircleCI] Fix Tests

* [CircleCI] Fix Tests

Co-authored-by: Alessio Fabiani <alessio.fabiani@geo-solutions.it>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.3.x 4.0.x major A high priority issue which might affect a lot of people or large parts of the codebase
Projects
None yet
Development

No branches or pull requests

2 participants