"Cannot read %d bytes, data too short" error causes ruuvitag-sensors to stop working

[jcalais]

This is probably more a feature, but causes me significant issues in use. After leaving my data collector running for a while, I will get the following kinds of errors from two Ruuvitag sensors that I've acquired new from verkkokauppa.com in Finland:

Invalid advertisement data: 1E1107DC00240EE5A9E093F3A3B50100406E0B0952757576692042333634AC04 Traceback (most recent call last): File "/home/pi/web/ruuvi/ruuvitest/venv/lib/python3.5/site-packages/ruuvitag_sensor/data_formats.py", line 62, in convert_data cdata, data = _dechunk(data) File "/home/pi/web/ruuvi/ruuvitest/venv/lib/python3.5/site-packages/ruuvitag_sensor/data_formats.py", line 21, in _dechunk raise ValueError("Cannot read %d bytes, data too short: %s" % (dlen, raw)) ValueError: Cannot read 52 bytes, data too short: 34AC

I've today updated both to the newest firmware version, but the issue persists.

I've worked around the issue by commenting the following line in the source:

https://github.com/ttu/ruuvitag-sensor/blob/master/ruuvitag_sensor/ruuvi.py#L183

What appears to be happening:

• _dechunk() in data_formats.py gets raw data to dechunk that is too short
• _dechunk() is called from convert_data() in the same file
• On line 160 in ruuvi.py convert_data() is called and if this returns a none, none response because the data was too short...
• On line 181 the else statement blacklists the ruuvitag that produced data that was too short.
• This causes that particular ruuvitag to be ignored for the rest of the session, which seems a bit harsh if short data is "normal" in the sense that periodically the bluetooth stack receives only part of the message or something.
• My sensors are admittedly quite far from the Raspberry Pi 3b listening to the ruuvitags (one in the basement, one in the green house), but on the other hand Grafana shows pretty much an uninterrupted line, so most of the messages are received.

I don't know why the choice was made to blacklist sensors that produce short data or in what circumstances short data occurs, but I fixed it by commenting out the aforementioned line 183 in ruuvi.py to prevent black-listing and now my data collector has been working without interruption. I guess it comes down to whether there is actually a poor design decision here because "short data" errors are quite common or whether something in my system is broken because short data errors shouldn't be happening: The ruuvitags, the Bluetooth stack on my Raspberry or even a hardware error on the Raspberry.

As an example, I left home for roughly 10 hours and when I came back home, I had at least 30 errors like the one I described above, so this happens fairly frequently and without my "hack", this library is all but unusable.

This is how I invoke ruuvitag-sensor:
from ruuvitag_sensor.ruuvi import RuuviTagSensor RuuviTagSensor.get_datas(handle_data, macs)
My handle_data -function just injects the received data into InfluxDB.

• OS: Debian (Raspbian on Raspberry Pi 3b)
• ruuvitag_sensor package version: 1.2.0
• RuuviTag firmware version: 3.31.1 (newest)
• Data format: 5

[ttu]

Hi and thanks for the issue!

I think this hasn't been brought up before, so it is possible that it’s non common system specific issue.

Nevertheless, we can try to build a workaround for it. Maybe when we receive too short data, we can assume that it is from RuuviTag?

Some adapters already check that data is from RuuviTag before returning it to caller, but some adapters rely on blacklisting, so we still have to blacklist macs that send non RuuviTag data.

Maybe this could be a possible fix for the issue #148