You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
from mcap_ros2.reader import read_ros2_messages
for msg in read_ros2_messages('minimal-mcap-to-trigger-memory-error.mcap'):
print(msg.ros_msg)
break
Observed behaviour
(venv) mtandy@mtandy-tr:/tmp/mcap-test$ python3 basictest.py
Traceback (most recent call last):
File "/tmp/mcap-test/basictest.py", line 4, in <module>
for msg in read_ros2_messages('demo-memoryerror.mcap'):
File "/tmp/venv/lib/python3.12/site-packages/mcap_ros2/reader.py", line 74, in read_ros2_messages
for schema, channel, message, ros2_msg in reader.iter_decoded_messages(
File "/tmp/venv/lib/python3.12/site-packages/mcap/reader.py", line 191, in iter_decoded_messages
for schema, channel, message in message_iterator:
File "/tmp/venv/lib/python3.12/site-packages/mcap/reader.py", line 278, in iter_messages
summary = self.get_summary()
^^^^^^^^^^^^^^^^^^
File "/tmp/venv/lib/python3.12/site-packages/mcap/reader.py", line 338, in get_summary
footer = next(StreamReader(self._stream, skip_magic=True).records)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/venv/lib/python3.12/site-packages/mcap/stream_reader.py", line 120, in records
record = self._read_record(opcode, length)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/venv/lib/python3.12/site-packages/mcap/stream_reader.py", line 205, in _read_record
self._stream.read(length - 9)
File "/tmp/venv/lib/python3.12/site-packages/mcap/data_stream.py", line 29, in read
data = self._stream.read(length)
^^^^^^^^^^^^^^^^^^^^^^^^^
MemoryError
### Changelog
- Added: the python MCAP library reader classes gained a new parameter,
`record_size_limit`, defaulting to 4GiB. This allows callers to limit
the size of records that their application will support. When a record
is encountered with a greater length, the reader will raise an
`InvalidRecordLength` exception. This limit can be removed by setting it
to `None`. This helps applications avoid the issue where a corrupt MCAP
can cause a `MemoryError`.
### Docs
See generated python docs in the mcap.dev preview.
### Description
<!-- Describe the problem, what has changed, and motivation behind those
changes. Pretend you are advocating for this change and the reader is
skeptical. -->
<!-- In addition to unit tests, describe any manual testing you did to
validate this change. -->
<table><tr><th>Before</th><th>After</th></tr><tr><td>
<!--before content goes here-->
</td><td>
<!--after content goes here-->
</td></tr></table>
<!-- If necessary, link relevant Linear or Github issues. Use `Fixes:
foxglove/repo#1234` to auto-close the Github issue or Fixes: FG-### for
Linear isses. -->
Fixes: #1220
Description
Steps To Reproduce
Load a truncated file, such as this one: minimal-mcap-to-trigger-memory-error.mcap.gz
Using for example this code:
Observed behaviour
4049071657446291757
bytes, having read the ASCII69048118
and the opcode is51
"unknown"Expected Behavior
mcap.exceptions.EndOfFile
or similarThe text was updated successfully, but these errors were encountered: