Stephan's sprintbattical #12

shoyer · 2014-02-14T21:23:09Z

No description provided.

Now O(n) instead of O(n^2), even though we do have to iterate through every element of an array in Python (gasp!).

Refactored the `broadcast_variables` function and fixed a bug in `variable.transpose` (see the new test case).

Added better test that uses string labels instead of just integers.

I think these names are much more straightforward. The only annoying aspect is that "array" is the name of a built-in module, which conflicts with naming the module "Array" is in "array".

We should probably remove Array.aggregate to reduce confusion, but for now I'll keep aggregate for error checks.

ebrevdo · 2014-02-14T22:55:58Z

Looks like a great change! I'm seeing some failing tests:

~/dev/scidata (DataView)$ python setup.py test
running test
running egg_info
writing requirements to src/xray.egg-info/requires.txt
writing src/xray.egg-info/PKG-INFO
writing top-level names to src/xray.egg-info/top_level.txt
writing dependency_links to src/xray.egg-info/dependency_links.txt
reading manifest file 'src/xray.egg-info/SOURCES.txt'
writing manifest file 'src/xray.egg-info/SOURCES.txt'
running build_ext
test_1d_math (test.test_array.TestArray) ... ok
test_aggregate (test.test_array.TestArray) ... ERROR
test_array_interface (test.test_array.TestArray) ... ok
test_broadcasting_failures (test.test_array.TestArray) ... ok
test_broadcasting_math (test.test_array.TestArray) ... ok
test_collapse (test.test_array.TestArray) ... ok
test_data (test.test_array.TestArray) ... ok
test_from_stack (test.test_array.TestArray) ... ok
test_groupby (test.test_array.TestArray) ... ERROR
test_indexed_by (test.test_array.TestArray) ... ok
test_inplace_math (test.test_array.TestArray) ... ok
test_items (test.test_array.TestArray) ... ok
test_properties (test.test_array.TestArray) ... ok
test_repr (test.test_array.TestArray) ... ok
test_transpose (test.test_array.TestArray) ... ok
test_attributes (test.test_dataset.DataTest) ... SKIP: attribute checks are not yet backend specific
test_coordinate (test.test_dataset.DataTest) ... ok
test_copy (test.test_dataset.DataTest) ... ok
test_dimension (test.test_dataset.DataTest) ... ok
test_getitem (test.test_dataset.DataTest) ... ERROR
test_indexed_by (test.test_dataset.DataTest) ... ok
test_init (test.test_dataset.DataTest) ... ok
test_iterator (test.test_dataset.DataTest) ... ok
test_labeled_by (test.test_dataset.DataTest) ... ERROR
test_merge (test.test_dataset.DataTest) ... ok
test_rename (test.test_dataset.DataTest) ... ok
test_repr (test.test_dataset.DataTest) ... ok
test_select (test.test_dataset.DataTest) ... ok
test_setitem (test.test_dataset.DataTest) ... ok
test_to_dataframe (test.test_dataset.DataTest) ... ok
test_unselect (test.test_dataset.DataTest) ... SKIP: need to write this test
test_variable (test.test_dataset.DataTest) ... ok
test_variable_indexing (test.test_dataset.DataTest) ... ok
test_write_store (test.test_dataset.DataTest) ... ok
test_attributes (test.test_dataset.NetCDF4DataTest) ... SKIP: attribute checks are not yet backend specific
test_coordinate (test.test_dataset.NetCDF4DataTest) ... ok
test_copy (test.test_dataset.NetCDF4DataTest) ... ok
test_dimension (test.test_dataset.NetCDF4DataTest) ... ok
test_dump_and_open_dataset (test.test_dataset.NetCDF4DataTest) ... ok
test_getitem (test.test_dataset.NetCDF4DataTest) ... ERROR
test_indexed_by (test.test_dataset.NetCDF4DataTest) ... ok
test_init (test.test_dataset.NetCDF4DataTest) ... ok
test_iterator (test.test_dataset.NetCDF4DataTest) ... ok
test_labeled_by (test.test_dataset.NetCDF4DataTest) ... ERROR
test_merge (test.test_dataset.NetCDF4DataTest) ... ok
test_rename (test.test_dataset.NetCDF4DataTest) ... ok
test_repr (test.test_dataset.NetCDF4DataTest) ... ok
test_select (test.test_dataset.NetCDF4DataTest) ... ok
test_setitem (test.test_dataset.NetCDF4DataTest) ... ok
test_to_dataframe (test.test_dataset.NetCDF4DataTest) ... ok
test_unselect (test.test_dataset.NetCDF4DataTest) ... SKIP: need to write this test
test_variable (test.test_dataset.NetCDF4DataTest) ... ok
test_variable_indexing (test.test_dataset.NetCDF4DataTest) ... ok
test_write_store (test.test_dataset.NetCDF4DataTest) ... ok
test_attributes (test.test_dataset.ScipyDataTest) ... SKIP: attribute checks are not yet backend specific
test_coordinate (test.test_dataset.ScipyDataTest) ... ok
test_copy (test.test_dataset.ScipyDataTest) ... ok
test_dimension (test.test_dataset.ScipyDataTest) ... ok
test_dump_and_open_dataset (test.test_dataset.ScipyDataTest) ... FAIL
test_getitem (test.test_dataset.ScipyDataTest) ... ERROR
test_indexed_by (test.test_dataset.ScipyDataTest) ... ok
test_init (test.test_dataset.ScipyDataTest) ... ok
test_iterator (test.test_dataset.ScipyDataTest) ... ok
test_labeled_by (test.test_dataset.ScipyDataTest) ... ERROR
test_merge (test.test_dataset.ScipyDataTest) ... ok
test_rename (test.test_dataset.ScipyDataTest) ... ok
test_repr (test.test_dataset.ScipyDataTest) ... ok
test_select (test.test_dataset.ScipyDataTest) ... ok
test_setitem (test.test_dataset.ScipyDataTest) ... ok
test_to_dataframe (test.test_dataset.ScipyDataTest) ... ok
test_unselect (test.test_dataset.ScipyDataTest) ... SKIP: need to write this test
test_variable (test.test_dataset.ScipyDataTest) ... ok
test_variable_indexing (test.test_dataset.ScipyDataTest) ... ok
test_write_store (test.test_dataset.ScipyDataTest) ... ok
test.test_dataset.create_test_data ... ok
test_aggregate (test.test_dataset_array.TestDatasetArray) ... FAIL
test_array_interface (test.test_dataset_array.TestDatasetArray) ... ok
test_collapse (test.test_dataset_array.TestDatasetArray) ... ok
test_dataset_getitem (test.test_dataset_array.TestDatasetArray) ... ok
test_from_stack (test.test_dataset_array.TestDatasetArray) ... ok
test_groupby (test.test_dataset_array.TestDatasetArray) ... FAIL
test_indexed_by (test.test_dataset_array.TestDatasetArray) ... ok
test_inplace_math (test.test_dataset_array.TestDatasetArray) ... ok
test_intersection (test.test_dataset_array.TestDatasetArray) ... FAIL
test_item_math (test.test_dataset_array.TestDatasetArray) ... ok
test_items (test.test_dataset_array.TestDatasetArray) ... ok
test_iteration (test.test_dataset_array.TestDatasetArray) ... ok
test_labeled_by (test.test_dataset_array.TestDatasetArray) ... FAIL
test_loc (test.test_dataset_array.TestDatasetArray) ... FAIL
test_math (test.test_dataset_array.TestDatasetArray) ... ok
test_properties (test.test_dataset_array.TestDatasetArray) ... ok
test_refocus (test.test_dataset_array.TestDatasetArray) ... ok
test_renamed (test.test_dataset_array.TestDatasetArray) ... ok
test_frozen (test.test_utils.TestDictionaries) ... ok
test_ordered_dict_intersection (test.test_utils.TestDictionaries) ... ok
test_safe (test.test_utils.TestDictionaries) ... ok
test_unsafe (test.test_utils.TestDictionaries) ... ok
test_expanded_indexer (test.test_utils.TestIndexers) ... ok
test_orthogonal_indexer (test.test_utils.TestIndexers) ... ok
test (test.test_utils.TestNum2DatetimeIndex) ... ERROR

ERROR: test_aggregate (test.test_array.TestArray)

Traceback (most recent call last):
File "/Users/ebrevdo/dev/scidata/test/test_array.py", line 240, in test_aggregate
self.assertVarEqual(expected_unique, actual_unique)
File "/Users/ebrevdo/dev/scidata/test/init.py", line 10, in assertVarEqual
self.assertTrue(utils.variable_equal(v1, v2))
File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 132, in variable_equal
return np.array_equal(data1, data2)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/numeric.py", line 1977, in array_equal
return bool(logical_and.reduce(equal(a1,a2).ravel()))
AttributeError: 'NotImplementedType' object has no attribute 'ravel'

ERROR: test_groupby (test.test_array.TestArray)

Traceback (most recent call last):
File "/Users/ebrevdo/dev/scidata/test/test_array.py", line 220, in test_groupby
self.assertVarEqual(expected_unique, grouped.unique_coord)
File "/Users/ebrevdo/dev/scidata/test/init.py", line 10, in assertVarEqual
self.assertTrue(utils.variable_equal(v1, v2))
File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 132, in variable_equal
return np.array_equal(data1, data2)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/numeric.py", line 1977, in array_equal
return bool(logical_and.reduce(equal(a1,a2).ravel()))
AttributeError: 'NotImplementedType' object has no attribute 'ravel'

ERROR: test_getitem (test.test_dataset.DataTest)

Traceback (most recent call last):
File "/Users/ebrevdo/dev/scidata/test/test_dataset.py", line 325, in test_getitem
{'units': 'days since 2000-01-01'})
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 464, in create_variable
return self.add_variable(name, v)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 534, in add_variable
return self.set_variable(name, var)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 595, in set_variable
self.indices.build_index(name)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 109, in build_index
self.cache[key] = self.dataset._create_index(key)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 224, in _create_index
attr.get('calendar'))
File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 106, in num2datetimeindex
dates = first_time_delta * num_delta + np.datetime64(first_dates[0])
TypeError: ufunc 'multiply' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'

ERROR: test_labeled_by (test.test_dataset.DataTest)

Traceback (most recent call last):
File "/Users/ebrevdo/dev/scidata/test/test_dataset.py", line 234, in test_labeled_by
{'units': 'days since 2000-01-01'})
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 464, in create_variable
return self.add_variable(name, v)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 534, in add_variable
return self.set_variable(name, var)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 595, in set_variable
self.indices.build_index(name)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 109, in build_index
self.cache[key] = self.dataset._create_index(key)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 224, in _create_index
attr.get('calendar'))
File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 106, in num2datetimeindex
dates = first_time_delta * num_delta + np.datetime64(first_dates[0])
TypeError: ufunc 'multiply' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'

ERROR: test_getitem (test.test_dataset.NetCDF4DataTest)

Traceback (most recent call last):
File "/Users/ebrevdo/dev/scidata/test/test_dataset.py", line 325, in test_getitem
{'units': 'days since 2000-01-01'})
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 464, in create_variable
return self.add_variable(name, v)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 534, in add_variable
return self.set_variable(name, var)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 595, in set_variable
self.indices.build_index(name)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 109, in build_index
self.cache[key] = self.dataset._create_index(key)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 224, in _create_index
attr.get('calendar'))
File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 106, in num2datetimeindex
dates = first_time_delta * num_delta + np.datetime64(first_dates[0])
TypeError: ufunc 'multiply' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'

ERROR: test_labeled_by (test.test_dataset.NetCDF4DataTest)

Traceback (most recent call last):
File "/Users/ebrevdo/dev/scidata/test/test_dataset.py", line 234, in test_labeled_by
{'units': 'days since 2000-01-01'})
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 464, in create_variable
return self.add_variable(name, v)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 534, in add_variable
return self.set_variable(name, var)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 595, in set_variable
self.indices.build_index(name)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 109, in build_index
self.cache[key] = self.dataset._create_index(key)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 224, in _create_index
attr.get('calendar'))
File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 106, in num2datetimeindex
dates = first_time_delta * num_delta + np.datetime64(first_dates[0])
TypeError: ufunc 'multiply' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'

ERROR: test_getitem (test.test_dataset.ScipyDataTest)

Traceback (most recent call last):
File "/Users/ebrevdo/dev/scidata/test/test_dataset.py", line 325, in test_getitem
{'units': 'days since 2000-01-01'})
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 464, in create_variable
return self.add_variable(name, v)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 534, in add_variable
return self.set_variable(name, var)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 595, in set_variable
self.indices.build_index(name)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 109, in build_index
self.cache[key] = self.dataset._create_index(key)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 224, in _create_index
attr.get('calendar'))
File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 106, in num2datetimeindex
dates = first_time_delta * num_delta + np.datetime64(first_dates[0])
TypeError: ufunc 'multiply' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'

ERROR: test_labeled_by (test.test_dataset.ScipyDataTest)

Traceback (most recent call last):
File "/Users/ebrevdo/dev/scidata/test/test_dataset.py", line 234, in test_labeled_by
{'units': 'days since 2000-01-01'})
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 464, in create_variable
return self.add_variable(name, v)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 534, in add_variable
return self.set_variable(name, var)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 595, in set_variable
self.indices.build_index(name)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 109, in build_index
self.cache[key] = self.dataset._create_index(key)
File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 224, in _create_index
attr.get('calendar'))
File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 106, in num2datetimeindex
dates = first_time_delta * num_delta + np.datetime64(first_dates[0])
TypeError: ufunc 'multiply' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'

ERROR: test (test.test_utils.TestNum2DatetimeIndex)

Traceback (most recent call last):
File "/Users/ebrevdo/dev/scidata/test/test_utils.py", line 68, in test
actual = utils.num2datetimeindex(num_dates, units, calendar)
File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 106, in num2datetimeindex
dates = first_time_delta * num_delta + np.datetime64(first_dates[0])
TypeError: ufunc 'multiply' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'

FAIL: test_dump_and_open_dataset (test.test_dataset.ScipyDataTest)

Traceback (most recent call last):
File "/Users/ebrevdo/dev/scidata/test/test_dataset.py", line 404, in test_dump_and_open_dataset
self.assertEquals(expected, actual)
AssertionError: <xray.Dataset (time: 1000, @dim1: 100, @dim2: 50, @dim3: 10): var1 var2 var3> != <xray.Dataset (@dim2: 50, @dim3: 10, @dim1: 100, time: 1000): var1 var3 var2>

FAIL: test_aggregate (test.test_dataset_array.TestDatasetArray)

Traceback (most recent call last):
File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 213, in test_aggregate
self.assertViewEqual(expected, actual)
File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 9, in assertViewEqual
self.assertEqual(dv1.dataset, dv2.dataset)
AssertionError: <xray.Dataset (@x: 10, @abc: 3): foo> != <xray.Dataset (@x: 10, @abc: 3): foo>

FAIL: test_groupby (test.test_dataset_array.TestDatasetArray)

Traceback (most recent call last):
File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 189, in test_groupby
grouped.collapse(np.sum, dimension=None))
File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 9, in assertViewEqual
self.assertEqual(dv1.dataset, dv2.dataset)
AssertionError: <xray.Dataset (@abc: 3): foo> != <xray.Dataset (@abc: 3): foo>

FAIL: test_intersection (test.test_dataset_array.TestDatasetArray)

Traceback (most recent call last):
File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 240, in test_intersection
self.assertViewEqual(dv1, self.dv[:5])
File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 9, in assertViewEqual
self.assertEqual(dv1.dataset, dv2.dataset)
AssertionError: <xray.Dataset (@x: 5, @y: 20): foo> != <xray.Dataset (@x: 5, @y: 20): foo>

FAIL: test_labeled_by (test.test_dataset_array.TestDatasetArray)

Traceback (most recent call last):
File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 75, in test_labeled_by
self.assertViewEqual(self.dv, self.dv.labeled_by(x=slice(None)))
File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 9, in assertViewEqual
self.assertEqual(dv1.dataset, dv2.dataset)
AssertionError: <xray.Dataset (@x: 10, @y: 20): foo> != <xray.Dataset (@x: 10, @y: 20): foo>

FAIL: test_loc (test.test_dataset_array.TestDatasetArray)

Traceback (most recent call last):
File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 81, in test_loc
self.assertViewEqual(self.dv[:3], self.dv.loc[:'c'])
File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 9, in assertViewEqual
self.assertEqual(dv1.dataset, dv2.dataset)
AssertionError: <xray.Dataset (@x: 3, @y: 20): foo> != <xray.Dataset (@x: 3, @y: 20): foo>

Ran 100 tests in 1.248s

FAILED (failures=6, errors=9, skipped=6)

shoyer · 2014-02-15T01:49:31Z

Thanks for @ebrevdo for taking a look! I'm pretty sure the issue here is the numpy version. I am running numpy 1.8 on my machine. It looks like we need at least numpy 1.7 for timedelta math [1], and it appears that array_equal can only compare string arrays in numpy 1.8 [2]. It's not insurmountable to work around either of these issues, but for I will increment the required version of numpy to 1.8.

[1] http://pandas.pydata.org/pandas-docs/stable/timeseries.html#numpy-1-7-compatibility
[2] numpy/numpy#2686

We need this for our current use of np.array_equal.

ebrevdo · 2014-02-15T02:39:21Z

Thanks for looking at that. I'll do a more thorough evaluation over the
weekend!

On Fri, Feb 14, 2014 at 5:49 PM, Stephan Hoyer notifications@github.comwrote:

Thanks for @ebrevdo https://github.com/ebrevdo for taking a look! I'm
pretty sure the issue here is the numpy version. I am running numpy 1.8 on
my machine. It looks like we need at least numpy 1.7 for timedelta math
[1], and it appears that array_equal can only compare string arrays in
numpy 1.8 [2]. It's not insurmountable to work around either of these
issues, but for I will increment the required version of numpy to 1.8.

[1]
http://pandas.pydata.org/pandas-docs/stable/timeseries.html#numpy-1-7-compatibility
[2] numpy/numpy#2686 numpy/numpy#2686

Reply to this email directly or view it on GitHubhttps://github.com//pull/12#issuecomment-35144276
.

Implements most of GitHub issue #13.

akleeman · 2014-02-21T00:20:34Z

src/xray/array_.py

@@ -0,0 +1,572 @@
+import functools


As we just discussed lets rename this to xarray.py.

akleeman · 2014-02-21T00:36:39Z

This is all great. I've been experimenting with this branch and the majority of it is running fine. Given that this project is still under heavy development and rather than bloating this pull request, lets go ahead and merge it into master and iterate on top of it.

Stephan's sprintbattical

shoyer · 2014-02-21T01:06:55Z

FYI -- I just pushed a commit renaming "Array" to "XArray" to master. There
are probably still a few lingering references to clean up...

On Thu, Feb 20, 2014 at 4:36 PM, akleeman notifications@github.com wrote:

Merged #12 #12.

Reply to this email directly or view it on GitHubhttps://github.com//pull/12
.

shoyer added 30 commits February 14, 2014 16:21

Tweaks to Variable and Dataset repr

920fd42

Add DataView and indexing

e330438

Orthogonal indexing and cached indices

63e2b15

Added utils.num2datetimeindex and collapse operators

0192e25

Simpler Dataset.views + working non-trivial indices + aggregated_by

900b61e

Refactored backends for more consistency/flexibility

0fbea90

Added virtual variables

8c5ba8f

Added utils.Frozen

fdfd4ce

Moved data.py to dataset.py

0e87356

Fixed iterator and removed broken methods

0354aa0

Renamed views & loc_views to indexed_by and labeled_by

f626652

More flexible __getitem__ and __setitem__

883f0fe

Patched in more numpy methods

c4794c5

Faster aggregate

d517bd1

Now O(n) instead of O(n^2), even though we do have to iterate through every element of an array in Python (gasp!).

More array interface tests for ufuncs

ba7083d

Fixed DataView.from_stack

dcdabee

Initial docs with Sphinx

697f135

to_dataframe method implements pandas.DataFrame export

941e833

Refactored the `broadcast_variables` function and fixed a bug in `variable.transpose` (see the new test case).

New README.md

d6c0b82

Fixed intersection

fb3289d

Added better test that uses string labels instead of just integers.

Renamed DataView.replace_focus to DataView.refocus

b5ff8d9

README edits

3c5856f

Indexing bug fixes

828c6dd

Fully tranverse dataset graphs with Dataset.select

ef5ac51

DataView.from_stack can concatenate along existing dimensions, too

a4b6ad9

Added variable.T as shortcut for variable.transpose()

f683b07

Added Variable.apply and DataView.apply

49edf3d

Revised and extended new README

1e2c47a

Renamed "Variable" -> "Array" and "DataView" -> "DatasetArray"

4cd1361

I think these names are much more straightforward. The only annoying aspect is that "array" is the name of a built-in module, which conflicts with naming the module "Array" is in "array".

Array.groupby

ad9a913

We should probably remove Array.aggregate to reduce confusion, but for now I'll keep aggregate for error checks.

Renamed package from 'scidata' to 'xray'

f08e2eb

Updated required numpy version to 1.8

cf3d6e2

We need this for our current use of np.array_equal.

shoyer mentioned this pull request Feb 15, 2014

Simplified Dataset API #13

Closed

shoyer and others added 13 commits February 15, 2014 14:05

Added TODO notes

0bd4c29

Simplified Dataset

0af583e

Implements most of GitHub issue #13.

Added utils.datetimeindex2num

dd81b0f

Reworked virtual variables

782a933

Fix performance regression in array_._as_compatible_data

e8738db

Speedup orthogonal indexing

508e16f

Documentation and name cleanup

0547123

Better tests for GroupBy; bug fixes

039bdd0

Removed aggregate and iteartor (they are replaced by groupby)

fdba77f

to_dataframe() no longer creates a large empty array

1ab3f4d

added DatasetArray.to_series() method

d8abfd3

added unused context argument to __array_wrap__

3f5bea2

Update setup.py

9488463

akleeman reviewed Feb 21, 2014
View reviewed changes

src/xray/array_.py

@@ -0,0 +1,572 @@

import functools

Copy link

Contributor

akleeman Feb 21, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we just discussed lets rename this to xarray.py.

akleeman added a commit that referenced this pull request Feb 21, 2014

Merge pull request #12 from akleeman/DataView

6e0b12c

Stephan's sprintbattical

akleeman merged commit 6e0b12c into master Feb 21, 2014

shoyer deleted the DataView branch February 26, 2014 09:56

spencerkclark mentioned this pull request Nov 11, 2021

Time dtype encoding defaulting to int64 when writing netcdf or zarr #3942

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stephan's sprintbattical #12

Stephan's sprintbattical #12

shoyer commented Feb 14, 2014

ebrevdo commented Feb 14, 2014

shoyer commented Feb 15, 2014

ebrevdo commented Feb 15, 2014

akleeman Feb 21, 2014

akleeman commented Feb 21, 2014

shoyer commented Feb 21, 2014

Stephan's sprintbattical #12

Stephan's sprintbattical #12

Conversation

shoyer commented Feb 14, 2014

ebrevdo commented Feb 14, 2014

ERROR: test_aggregate (test.test_array.TestArray)

ERROR: test_groupby (test.test_array.TestArray)

ERROR: test_getitem (test.test_dataset.DataTest)

ERROR: test_labeled_by (test.test_dataset.DataTest)

ERROR: test_getitem (test.test_dataset.NetCDF4DataTest)

ERROR: test_labeled_by (test.test_dataset.NetCDF4DataTest)

ERROR: test_getitem (test.test_dataset.ScipyDataTest)

ERROR: test_labeled_by (test.test_dataset.ScipyDataTest)

ERROR: test (test.test_utils.TestNum2DatetimeIndex)

FAIL: test_dump_and_open_dataset (test.test_dataset.ScipyDataTest)

FAIL: test_aggregate (test.test_dataset_array.TestDatasetArray)

FAIL: test_groupby (test.test_dataset_array.TestDatasetArray)

FAIL: test_intersection (test.test_dataset_array.TestDatasetArray)

FAIL: test_labeled_by (test.test_dataset_array.TestDatasetArray)

FAIL: test_loc (test.test_dataset_array.TestDatasetArray)

shoyer commented Feb 15, 2014

ebrevdo commented Feb 15, 2014

akleeman Feb 21, 2014

Choose a reason for hiding this comment

akleeman commented Feb 21, 2014

shoyer commented Feb 21, 2014