HDF5 Output issues with save_interval

[kyleabeauchamp]

I encountered this bug with my own model (choderalab/gbff#10), but the following code reproduces it with the example models in pymc. Removing the save_interval flag fixes the issue.

import pymc
from pymc.examples import disaster_model

sampler = pymc.MCMC(disaster_model, db='hdf5', dbname="./test.h5")
sampler.sample(100, save_interval=1)

Burn-in interval complete

Sampling finished normally.
Warning, unable to save state.
Error message:
Traceback (most recent call last):
  File "/home/kyleb/opt/lib/python2.7/site-packages/pymc/Model.py", line 777, in save_state
    self.db.savestate(self.get_state())
  File "/home/kyleb/opt/lib/python2.7/site-packages/pymc/database/hdf5.py", line 503, in savestate
    cur_chain._state_[0] = state
  File "/home/kyleb/opt/lib/python2.7/site-packages/tables/vlarray.py", line 785, in __setitem__
    self._assign_values(coords, value)
  File "/home/kyleb/opt/lib/python2.7/site-packages/tables/vlarray.py", line 718, in _assign_values
    nparr[:], exc))
ValueError: Value parameter:
'array([128,   2, 125, 113,   1,  40,  85,  11, 115, 116, 111,  99, 104,
        97, 115, 116, 105,  99, 115, 113,   2, 125, 113,   3,  40,  85,
        10, 101,  97, 114, 108, 121,  95, 109, 101,  97, 110, 113,   4,
        99, 110, 117, 109, 112, 121,  46,  99, 111, 114, 101,  46, 109,
       117, 108, 116, 105,  97, 114, 114,  97, 121,  10,  95, 114, 101,
        99, 111, 110, 115, 116, 114, 117,  99, 116,  10, 113,   5,  99,
       110, 117, 109, 112, 121,  10, 110, 100,  97, 114, 114,  97, 121,
        10, 113,   6,  75,   0, 133,  85,   1,  98, 135,  82, 113,   7,
        40,  75,   1,  41,  99, 110, 117, 109, 112, 121,  10, 100, 116,
       121, 112, 101,  10, 113,   8,  85,   2, 102,  56,  75,   0,  75,
         1, 135,  82, 113,   9,  40,  75,   3,  85,   1,  60,  78,  78,
        78,  74, 255, 255, 255, 255,  74, 255, 255, 255, 255,  75,   0,
       116,  98, 137,  85,   8,  78, 134,  53, 111, 148,  87, 198,  63,
       116,  98,  85,  11, 115, 119, 105, 116,  99, 104, 112, 111, 105,
       110, 116, 113,  10, 104,   5, 104,   6,  75,   0, 133,  85,   1,
        98, 135,  82, 113,  11,  40,  75,   1,  41, 104,   8,  85,   2,
       105,  56,  75,   0,  75,   1, 135,  82, 113,  12,  40,  75,   3,
        85,   1,  60,  78,  78,  78,  74, 255, 255, 255, 255,  74, 255,
       255, 255, 255,  75,   0, 116,  98, 137,  85,   8,   0,   0,   0,
         0,   0,   0,   0,   0, 116,  98,  85,   9, 108,  97, 116, 101,
        95, 109, 101,  97, 110, 113,  13, 104,   5, 104,   6,  75,   0,
       133,  85,   1,  98, 135,  82, 113,  14,  40,  75,   1,  41, 104,
         9, 137,  85,   8,  14, 242, 100, 236, 209, 133, 253,  63, 116,
        98, 117,  85,  12, 115, 116, 101, 112,  95, 109, 101, 116, 104,
       111, 100, 115, 113,  15, 125, 113,  16,  40,  85,  20,  77, 101,
       116, 114, 111, 112, 111, 108, 105, 115,  95, 108,  97, 116, 101,
        95, 109, 101,  97, 110, 113,  17, 125, 113,  18,  40,  85,  21,
        97, 100,  97, 112, 116, 105, 118, 101,  95, 115,  99,  97, 108,
       101,  95, 102,  97,  99, 116, 111, 114, 113,  19,  71,  63, 240,
         0,   0,   0,   0,   0,   0,  85,  21, 112, 114, 111, 112, 111,
       115,  97, 108,  95, 100, 105, 115, 116, 114, 105,  98, 117, 116,
       105, 111, 110, 113,  20,  85,   6,  78, 111, 114, 109,  97, 108,
       113,  21,  85,  11, 112, 114, 111, 112, 111, 115,  97, 108,  95,
       115, 100, 113,  22,  99, 110, 117, 109, 112, 121,  46,  99, 111,
       114, 101,  46, 109, 117, 108, 116, 105,  97, 114, 114,  97, 121,
        10, 115,  99,  97, 108,  97, 114,  10, 113,  23, 104,   9,  85,
         8, 164,  23, 214,   1, 218, 171, 217,  63, 134,  82, 113,  24,
        85,   8,  97,  99,  99, 101, 112, 116, 101, 100, 113,  25,  71,
        64,  65, 128,   0,   0,   0,   0,   0,  85,  22,  99, 104, 101,
        99, 107,  95,  98, 101, 102, 111, 114, 101,  95,  97,  99,  99,
       101, 112, 116, 105, 110, 103, 113,  26, 136,  85,   8, 114, 101,
       106, 101,  99, 116, 101, 100, 113,  27,  71,  64,  80,  64,   0,
         0,   0,   0,   0, 117,  85,  30,  68, 105, 115,  99, 114, 101,
       116, 101,  77, 101, 116, 114, 111, 112, 111, 108, 105, 115,  95,
       115, 119, 105, 116,  99, 104, 112, 111, 105, 110, 116, 113,  28,
       125, 113,  29,  40, 104,  19,  71,  63, 240,   0,   0,   0,   0,
         0,   0, 104,  20,  85,   7,  80, 111, 105, 115, 115, 111, 110,
       113,  30, 104,  22, 104,  23, 104,   9,  85,   8,   0,   0,   0,
         0,   0,   0, 240,  63, 134,  82, 113,  31, 104,  25,  71,  64,
        69, 128,   0,   0,   0,   0,   0, 104,  26, 136, 104,  27,  71,
        64,  76, 128,   0,   0,   0,   0,   0, 117,  85,  21,  77, 101,
       116, 114, 111, 112, 111, 108, 105, 115,  95, 101,  97, 114, 108,
       121,  95, 109, 101,  97, 110, 113,  32, 125, 113,  33,  40, 104,
        19,  71,  63, 240,   0,   0,   0,   0,   0,   0, 104,  20, 104,
        21, 104,  22, 104,  23, 104,   9,  85,   8, 124,  57, 143, 169,
        18, 149, 144,  63, 134,  82, 113,  34, 104,  25,  71,  64,  88,
       192,   0,   0,   0,   0,   0, 104,  26, 136, 104,  27,  71,  63,
       240,   0,   0,   0,   0,   0,   0, 117, 117,  85,   7, 115,  97,
       109, 112, 108, 101, 114, 113,  35, 125, 113,  36,  40,  85,   6,
       115, 116,  97, 116, 117, 115, 113,  37,  85,   5, 114, 101,  97,
       100, 121, 113,  38,  85,   5,  95, 105, 116, 101, 114, 113,  39,
       104,  23, 104,  12,  85,   8, 100,   0,   0,   0,   0,   0,   0,
         0, 134,  82, 113,  40,  85,  14,  95, 116, 117, 110, 101,  95,
       105, 110, 116, 101, 114, 118,  97, 108, 113,  41,  77, 232,   3,
        85,  12,  95, 116, 117, 110, 101, 100,  95,  99, 111, 117, 110,
       116, 113,  42,  75,   0,  85,  16,  95, 116, 117, 110, 101,  95,
       116, 104, 114, 111, 117, 103, 104, 111, 117, 116, 113,  43, 136,
        85,  16,  95,  98, 117, 114, 110,  95, 116, 105, 108, 108,  95,
       116, 117, 110, 101, 100, 113,  44, 137,  85,  13,  95,  99, 117,
       114, 114, 101, 110, 116,  95, 105, 116, 101, 114, 113,  45,  75,
       100,  85,   5,  95,  98, 117, 114, 110, 113,  46,  75,   0,  85,
         5,  95, 116, 104, 105, 110, 113,  47,  75,   1, 117, 117,  46], dtype=uint8)'
cannot be converted into an array object compliant vlarray[0] row: 
'array([128,   2, 125, 113,   1,  40,  85,  11, 115, 116, 111,  99, 104,
        97, 115, 116, 105,  99, 115, 113,   2, 125, 113,   3,  40,  85,
        10, 101,  97, 114, 108, 121,  95, 109, 101,  97, 110, 113,   4,
        99, 110, 117, 109, 112, 121,  46,  99, 111, 114, 101,  46, 109,
       117, 108, 116, 105,  97, 114, 114,  97, 121,  10,  95, 114, 101,
        99, 111, 110, 115, 116, 114, 117,  99, 116,  10, 113,   5,  99,
       110, 117, 109, 112, 121,  10, 110, 100,  97, 114, 114,  97, 121,
        10, 113,   6,  75,   0, 133,  85,   1,  98, 135,  82, 113,   7,
        40,  75,   1,  41,  99, 110, 117, 109, 112, 121,  10, 100, 116,
       121, 112, 101,  10, 113,   8,  85,   2, 102,  56,  75,   0,  75,
         1, 135,  82, 113,   9,  40,  75,   3,  85,   1,  60,  78,  78,
        78,  74, 255, 255, 255, 255,  74, 255, 255, 255, 255,  75,   0,
       116,  98, 137,  85,   8,  78, 134,  53, 111, 148,  87, 198,  63,
       116,  98,  85,  11, 115, 119, 105, 116,  99, 104, 112, 111, 105,
       110, 116, 113,  10, 104,   5, 104,   6,  75,   0, 133,  85,   1,
        98, 135,  82, 113,  11,  40,  75,   1,  41, 104,   8,  85,   2,
       105,  56,  75,   0,  75,   1, 135,  82, 113,  12,  40,  75,   3,
        85,   1,  60,  78,  78,  78,  74, 255, 255, 255, 255,  74, 255,
       255, 255, 255,  75,   0, 116,  98, 137,  85,   8,   0,   0,   0,
         0,   0,   0,   0,   0, 116,  98,  85,   9, 108,  97, 116, 101,
        95, 109, 101,  97, 110, 113,  13, 104,   5, 104,   6,  75,   0,
       133,  85,   1,  98, 135,  82, 113,  14,  40,  75,   1,  41, 104,
         9, 137,  85,   8,  14, 242, 100, 236, 209, 133, 253,  63, 116,
        98, 117,  85,  12, 115, 116, 101, 112,  95, 109, 101, 116, 104,
       111, 100, 115, 113,  15, 125, 113,  16,  40,  85,  20,  77, 101,
       116, 114, 111, 112, 111, 108, 105, 115,  95, 108,  97, 116, 101,
        95, 109, 101,  97, 110, 113,  17, 125, 113,  18,  40,  85,  21,
        97, 100,  97, 112, 116, 105, 118, 101,  95, 115,  99,  97, 108,
       101,  95, 102,  97,  99, 116, 111, 114, 113,  19,  71,  63, 240,
         0,   0,   0,   0,   0,   0,  85,  21, 112, 114, 111, 112, 111,
       115,  97, 108,  95, 100, 105, 115, 116, 114, 105,  98, 117, 116,
       105, 111, 110, 113,  20,  85,   6,  78, 111, 114, 109,  97, 108,
       113,  21,  85,  11, 112, 114, 111, 112, 111, 115,  97, 108,  95,
       115, 100, 113,  22,  99, 110, 117, 109, 112, 121,  46,  99, 111,
       114, 101,  46, 109, 117, 108, 116, 105,  97, 114, 114,  97, 121,
        10, 115,  99,  97, 108,  97, 114,  10, 113,  23, 104,   9,  85,
         8, 164,  23, 214,   1, 218, 171, 217,  63, 134,  82, 113,  24,
        85,   8,  97,  99,  99, 101, 112, 116, 101, 100, 113,  25,  71,
        64,  65, 128,   0,   0,   0,   0,   0,  85,  22,  99, 104, 101,
        99, 107,  95,  98, 101, 102, 111, 114, 101,  95,  97,  99,  99,
       101, 112, 116, 105, 110, 103, 113,  26, 136,  85,   8, 114, 101,
       106, 101,  99, 116, 101, 100, 113,  27,  71,  64,  80,  64,   0,
         0,   0,   0,   0, 117,  85,  30,  68, 105, 115,  99, 114, 101,
       116, 101,  77, 101, 116, 114, 111, 112, 111, 108, 105, 115,  95,
       115, 119, 105, 116,  99, 104, 112, 111, 105, 110, 116, 113,  28,
       125, 113,  29,  40, 104,  19,  71,  63, 240,   0,   0,   0,   0,
         0,   0, 104,  20,  85,   7,  80, 111, 105, 115, 115, 111, 110,
       113,  30, 104,  22, 104,  23, 104,   9,  85,   8,   0,   0,   0,
         0,   0,   0, 240,  63, 134,  82, 113,  31, 104,  25,  71,  64,
        69, 128,   0,   0,   0,   0,   0, 104,  26, 136, 104,  27,  71,
        64,  76, 128,   0,   0,   0,   0,   0, 117,  85,  21,  77, 101,
       116, 114, 111, 112, 111, 108, 105, 115,  95, 101,  97, 114, 108,
       121,  95, 109, 101,  97, 110, 113,  32, 125, 113,  33,  40, 104,
        19,  71,  63, 240,   0,   0,   0,   0,   0,   0, 104,  20, 104,
        21, 104,  22, 104,  23, 104,   9,  85,   8, 124,  57, 143, 169,
        18, 149, 144,  63, 134,  82, 113,  34, 104,  25,  71,  64,  88,
       192,   0,   0,   0,   0,   0, 104,  26, 136, 104,  27,  71,  63,
       240,   0,   0,   0,   0,   0,   0, 117, 117,  85,   7, 115,  97,
       109, 112, 108, 101, 114, 113,  35, 125, 113,  36,  40,  85,   6,
       115, 116,  97, 116, 117, 115, 113,  37,  85,   7, 114, 117, 110,
       110, 105, 110, 103, 113,  38,  85,   5,  95, 105, 116, 101, 114,
       113,  39, 104,  23, 104,  12,  85,   8, 100,   0,   0,   0,   0,
         0,   0,   0, 134,  82, 113,  40,  85,  14,  95, 116, 117, 110,
       101,  95, 105, 110, 116, 101, 114, 118,  97, 108, 113,  41,  77,
       232,   3,  85,  12,  95, 116, 117, 110, 101, 100,  95,  99, 111,
       117, 110, 116, 113,  42,  75,   0,  85,  16,  95, 116, 117, 110,
       101,  95, 116, 104, 114, 111, 117, 103, 104, 111, 117, 116, 113,
        43, 136,  85,  16,  95,  98, 117, 114, 110,  95, 116, 105, 108,
       108,  95, 116, 117, 110, 101, 100, 113,  44, 137,  85,  13,  95,
        99, 117, 114, 114, 101, 110, 116,  95, 105, 116, 101, 114, 113,
        45,  75,  99,  85,   5,  95,  98, 117, 114, 110, 113,  46,  75,
         0,  85,   5,  95, 116, 104, 105, 110, 113,  47,  75,   1, 117,
       117,  46], dtype=uint8)'
The error was: <could not broadcast input array from shape (936) into shape (938)>

[kyleabeauchamp]

System details: anaconda python2.7, pytables 3.1.1, Ubuntu 14.04. I've tried both pymc 2.3.4 and latest git.