-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Import volumes from Pangaea #95
Comments
We noticed that some datasets in Pangaea may be easily used as a remote volume. Take this one for example. You can download a CSV with all image filenames and the volume URL. There is even location data for each image. Another example is this where the URL is also usable for a remote volume. These images are loaded from tape so first there is a redirect to a "please wait" page and then the download is initiated. This works automatically, too. When I request an image with cURL I get the HTML response first. If I wait a few seconds and then request the same URL again, I get the image file. We can probably implement a dialog where users can create new volumes from Pangaea. They only have to insert the dataset URL and BIIGLE does the rest. |
Make sure to import the DOI of the dataset as well (biigle/volumes#38). |
This would be a very good feature with high application potential! |
Also import these image metadata fields. |
I just sent my second message to the PANGAEA guys via their contact form. Hopefully they'll answer at some point. |
The PANGAEA people finally answered. They said that they can't change the existing behavior of a returned code 503 and a periodic retry until an image is fetched from tape. If we want to make BIIGLE compatible with this, we would need to handle URLs from PANGAEA as a special case, both in the (video) annotation tool and in the file cache package. |
Apparently PANGAEA is not interested in becoming a central image and video repository. Continued in #207. |
We had another discussion with the people of PANGAEA. The possibility to receive a 503 response remains but it should be possible to make BIIGLE compatible at the following locations:
|
We could provide a function to easily import volumes from Pangaea as remote volumes. These volumes can keep a reference to their source and all the metadata stored in Pangaea (as we don't want to store all that in Biigle ourselves).
The text was updated successfully, but these errors were encountered: