Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GDAL, Nan Calculations, and HDFS for loading images #296

Closed
wants to merge 1 commit into from
Closed

GDAL, Nan Calculations, and HDFS for loading images #296

wants to merge 1 commit into from

Conversation

SteveAIS
Copy link

@SteveAIS SteveAIS commented May 3, 2016

Adding changes to use GDAL for loading TIF images.

Adding changes to allow nan calculations (i.e. nanmean, nanmin, nanmax)

Adding changes to allow reading from HDFS when loading images.

Adding changes to allow nan calculations (i.e. nanmean, nanmin, nanmax)

Adding changes to allow reading from HDFS when loading images.
@freeman-lab
Copy link
Member

@stevevarner thanks for putting this together! A few top-level comments that will hopefully make it easier to incorporate these changes

  • it'd be ideal if you could split this into three separate PRs for the three separate changes (GDAL, HDFS, nan)
  • it'd be great to write new tests for each new piece of functionality, you can look at the existing tests as a guide
  • there's code in the commit that adds the crop method back in, but it's not mentioned in the PR or used in any of the other changes, maybe this was included by accident?
  • the gdal related code seem to have some hardcoded paths, maybe that was included by accident too?

thanks!

@techchrj
Copy link

techchrj commented May 4, 2016

@freeman-lab

In reference to gdal:

you use FileFromMemBuffer() to create a virtual file and open it with the buffer

Here's the C API for it :
http://gdal.org/cpl__vsi_8h.html#1ecb3a78729ecea4d2ce22065a605244

And here's an extract of a test in the Python autotest suite.

content = open('data/byte.tif', mode='rb').read()

# Create in-memory file and initialize it with the content 
gdal.FileFromMemBuffer('/vsimem/tiffinmem', content)

# Open the in-memory file
ds = gdal.Open('/vsimem/tiffinmem')

# do something with the dataset

# Close the dataset
ds = None

# Release memory associated to the in-memory file
gdal.Unlink('/vsimem/tiffinmem')

@SteveAIS SteveAIS closed this May 4, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants