Detected blocking call to dsmr_parser #118919

MaVeWeb · 2024-06-05T22:41:16Z

The problem

After updating to HA 2024.6.0, I get 597 times the warning:

Detected blocking call to open with args ('/usr/local/lib/python3.12/site-packages/pytz/zoneinfo/Universal', 'rb') in /usr/local/lib/python3.12/site-packages/pytz/init.py, line 108: return open(filename, 'rb') inside the event loop Traceback

What version of Home Assistant Core has the issue?

core-2024.6.0

What was the last working version of Home Assistant Core?

core-2024.5.5

What type of installation are you running?

Home Assistant OS

Integration causing the issue

May be dsmr_parser ????

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

2024-06-05 22:53:20.785 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to open with args ('/usr/local/lib/python3.12/site-packages/pytz/zoneinfo/Africa/Abidjan', 'rb') in /usr/local/lib/python3.12/site-packages/pytz/__init__.py, line 108: return open(filename, 'rb') inside the event loop
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 672, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 639, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1988, in _run_once
    handle._run()
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.12/site-packages/serial_asyncio/__init__.py", line 120, in _read_ready
    self._protocol.data_received(data)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/clients/protocol.py", line 133, in data_received
    self.handle_telegram(telegram)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/clients/protocol.py", line 159, in handle_telegram
    parsed_telegram = self.telegram_parser.parse(telegram)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 95, in parse
    dsmr_object = object["value_parser"].parse(match)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 311, in parse
    values=self._parse(line)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 219, in _parse
    return self._parse_values(values)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 190, in _parse_values
    return [self.value_formats[i].parse(value)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 399, in parse
    value = self.coerce_type(value) if value is not None else value
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/value_types.py", line 24, in timestamp
    local_tz = pytz.timezone('Europe/Amsterdam')
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 181, in timezone
    if zone in all_timezones_set:  # noqa
  File "/usr/local/lib/python3.12/site-packages/pytz/lazy.py", line 154, in _lazy
    for i in fill_iter.pop():
  File "/usr/local/lib/python3.12/site-packages/pytz/lazy.py", line 101, in _lazy
    list.extend(self, fill_iter.pop())
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 1115, in <genexpr>
    tz for tz in _all_timezones_unchecked if resource_exists(tz))
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 121, in resource_exists
    open_resource(name).close()
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 108, in open_resource
    return open(filename, 'rb')

2024-06-05 22:53:20.789 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to open with args ('/usr/local/lib/python3.12/site-packages/pytz/zoneinfo/Africa/Accra', 'rb') in /usr/local/lib/python3.12/site-packages/pytz/__init__.py, line 108: return open(filename, 'rb') inside the event loop
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 672, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 639, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1988, in _run_once
    handle._run()
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.12/site-packages/serial_asyncio/__init__.py", line 120, in _read_ready
    self._protocol.data_received(data)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/clients/protocol.py", line 133, in data_received
    self.handle_telegram(telegram)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/clients/protocol.py", line 159, in handle_telegram
    parsed_telegram = self.telegram_parser.parse(telegram)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 95, in parse
    dsmr_object = object["value_parser"].parse(match)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 311, in parse
    values=self._parse(line)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 219, in _parse
    return self._parse_values(values)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 190, in _parse_values
    return [self.value_formats[i].parse(value)
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/parsers.py", line 399, in parse
    value = self.coerce_type(value) if value is not None else value
  File "/usr/local/lib/python3.12/site-packages/dsmr_parser/value_types.py", line 24, in timestamp
    local_tz = pytz.timezone('Europe/Amsterdam')
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 181, in timezone
    if zone in all_timezones_set:  # noqa
  File "/usr/local/lib/python3.12/site-packages/pytz/lazy.py", line 154, in _lazy
    for i in fill_iter.pop():
  File "/usr/local/lib/python3.12/site-packages/pytz/lazy.py", line 101, in _lazy
    list.extend(self, fill_iter.pop())
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 1115, in <genexpr>
    tz for tz in _all_timezones_unchecked if resource_exists(tz))
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 121, in resource_exists
    open_resource(name).close()
  File "/usr/local/lib/python3.12/site-packages/pytz/__init__.py", line 108, in open_resource
    return open(filename, 'rb')

Additional information

No response

peetersch · 2024-06-06T01:33:10Z

I have the same issue.
Returning to HA version 2024.5.5 fixed the issue.

bdraco · 2024-06-06T03:44:16Z

I have the same issue.

Returning to HA version 2024.5.5 fixed the issue.

2024.5.5 isn't capable of detecting this problem so it's still there but blissfully unaware

bdraco · 2024-06-06T03:51:49Z

https://developers.home-assistant.io/blog/2024/05/19/fix_zoneinfo_blocking_io/

elupus · 2024-06-06T09:10:12Z

This is fixed upstream ndokter/dsmr_parser#157

Needs a library bump

home-assistant · 2024-06-06T09:11:08Z

Hey there @RobBie1221, @frenck, mind taking a look at this issue as it has been labeled with an integration (dsmr) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of dsmr can trigger bot actions by commenting:

@home-assistant close Closes the issue.
@home-assistant rename Awesome new title Renames the issue.
@home-assistant reopen Reopen the issue.
@home-assistant unassign dsmr Removes the current integration label and assignees on the issue, add the integration domain after the command.
@home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
@home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

dsmr documentation
dsmr source
_{^{(message by IssueLinks)}}

bdraco · 2024-06-23T20:33:36Z

I think this is fixed upstream but the lib needs to be bumped

bdraco · 2024-06-25T01:41:37Z

Looks like there some tests that will need to be adjusted

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ERROR collecting tests/components/dsmr/test_mbus_migration.py ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
ImportError while importing test module '/Users/bdraco/home-assistant/tests/components/dsmr/test_mbus_migration.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/components/dsmr/test_mbus_migration.py:6: in <module>
    from dsmr_parser.obis_references import (
E   ImportError: cannot import name 'BELGIUM_MBUS1_DEVICE_TYPE' from 'dsmr_parser.obis_references' (/Users/bdraco/home-assistant/venv/lib/python3.12/site-packages/dsmr_parser/obis_references.py)

―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ERROR collecting tests/components/dsmr/test_sensor.py ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
ImportError while importing test module '/Users/bdraco/home-assistant/tests/components/dsmr/test_sensor.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/components/dsmr/test_sensor.py:14: in <module>
    from dsmr_parser.obis_references import (
E   ImportError: cannot import name 'BELGIUM_MBUS1_DEVICE_TYPE' from 'dsmr_parser.obis_references' (/Users/bdraco/home-assistant/venv/lib/python3.12/site-packages/dsmr_parser/obis_references.py)
collected 49 items / 2 errors

Sadly I don't have the ability to test this one so it will need to wait for someone who can do the changes

TheJulianJES added the non-thread-safe operation label Jun 5, 2024

bdraco added blocking event loop and removed non-thread-safe operation labels Jun 6, 2024

elupus added the integration: dsmr label Jun 6, 2024

home-assistant bot assigned frenck Jun 6, 2024

This was referenced Jun 7, 2024

Detected blocking call to open with args #119068

Closed

Detected blocking call #119076

Closed

bdraco added the no-stale label Jun 7, 2024

peetersch mentioned this issue Jun 23, 2024

Update manifest.json #120222

Closed

20 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Detected blocking call to dsmr_parser #118919

Detected blocking call to dsmr_parser #118919

MaVeWeb commented Jun 5, 2024

peetersch commented Jun 6, 2024 •

edited

bdraco commented Jun 6, 2024

bdraco commented Jun 6, 2024

elupus commented Jun 6, 2024

home-assistant bot commented Jun 6, 2024

bdraco commented Jun 23, 2024

bdraco commented Jun 25, 2024

Detected blocking call to dsmr_parser #118919

Detected blocking call to dsmr_parser #118919

Comments

MaVeWeb commented Jun 5, 2024

The problem

What version of Home Assistant Core has the issue?

What was the last working version of Home Assistant Core?

What type of installation are you running?

Integration causing the issue

Link to integration documentation on our website

Diagnostics information

Example YAML snippet

Anything in the logs that might be useful for us?

Additional information

peetersch commented Jun 6, 2024 • edited

bdraco commented Jun 6, 2024

bdraco commented Jun 6, 2024

elupus commented Jun 6, 2024

home-assistant bot commented Jun 6, 2024

bdraco commented Jun 23, 2024

bdraco commented Jun 25, 2024

peetersch commented Jun 6, 2024 •

edited