Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

myst-anchors ingores source_encoding in conf.py #609

Closed
ile-24556 opened this issue Aug 11, 2022 · 2 comments · Fixed by #610
Closed

myst-anchors ingores source_encoding in conf.py #609

ile-24556 opened this issue Aug 11, 2022 · 2 comments · Fixed by #610
Labels
bug Something isn't working

Comments

@ile-24556
Copy link

Describe the bug

context
When I execute myst-anchors to predict anchor links.

Source (saved with UTF-8):

# Title

## ASCII

## Ä

Built html:

<section id="title">
  <h1>Title<a class="headerlink" href="#title" title="Permalink to this heading"></a></h1>
  <section id="ascii">
    <h2>ASCII<a class="headerlink" href="#ascii" title="Permalink to this heading"></a></h2>
  </section>
  <section id="a">
    <h2>Ä<a class="headerlink" href="#a" title="Permalink to this heading"></a></h2>
  </section>
</section>

expectation
Same anchors as the built HTML are shown.

$ myst-anchors.exe source/index.md 
<h1 id="title"></h1>
<h2 id="ascii"></h2>
<h2 id="a"></h2>

bug
But instead UnicodeDecodeError is raised.

$ myst-anchors source/index.md
Traceback (most recent call last):
  File "***\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "***\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "***\venv\Scripts\myst-anchors.exe\__main__.py", line 7, in <module>
  File "***\venv\lib\site-packages\myst_parser\cli.py", line 41, in print_anchors
    text = parser.render(args.input.read())
UnicodeDecodeError: 'cp932' codec can't decode byte 0x84 in position 27: illegal multibyte sequence

problem
It seems to that myst-anchors ignores source_encoding in conf.py and uses the encoding of the locale.

Reproduce the bug

  1. Prepare Japanese Windows environment (sorry for the difficulty ...)
  2. sphinx-quickstart
  3. Add myst_parser in extensions and set source_encoding in conf.py
  4. Write index.md including non-ASCII characters
  5. Run myst-anchors source/index.md

conf.py

source_encoding = 'utf_8'

List your environment

$ python
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getdefaultlocale()
('ja_JP', 'cp932')

Windows 10 Home 21H2
myst-parser==0.18.0
Sphinx==5.1.1

@ile-24556 ile-24556 added the bug Something isn't working label Aug 11, 2022
@welcome
Copy link

welcome bot commented Aug 11, 2022

Thanks for opening your first issue here! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.

If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).

Welcome to the EBP community! 🎉

@chrisjsewell
Copy link
Member

thanks for the feedback, I added a simple fix, but let me know if this is still an issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants