Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

[Bug Fix]handle utf-16 in display data -v mode #4864

Merged
merged 1 commit into from
Nov 7, 2022
Merged

Conversation

dexterju27
Copy link
Contributor

Patch description
The current display data code will break in -v mode when emoji presented.

Traceback (most recent call last):
  File "/private/home/daju/.conda/envs/mep/bin/parlai", line 33, in <module>
    sys.exit(load_entry_point('parlai', 'console_scripts', 'parlai')())
  File "/private/home/daju/ParlAI/parlai/__main__.py", line 14, in main
    superscript_main()
  File "/private/home/daju/ParlAI/parlai/core/script.py", line 325, in superscript_main
    return SCRIPT_REGISTRY[cmd].klass._run_from_parser_and_opt(opt, parser)
  File "/private/home/daju/ParlAI/parlai/core/script.py", line 108, in _run_from_parser_and_opt
    return script.run()
  File "/private/home/daju/ParlAI/parlai/scripts/display_data.py", line 113, in run
    return display_data(self.opt)
  File "/private/home/daju/ParlAI/parlai/scripts/display_data.py", line 85, in display_data
    print(world.display() + '\n~~')
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 2079-2080: surrogates not allowed

This code fixes this issue.

Copy link
Contributor

@klshuster klshuster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! minor nit: if we start to use this in other places in the repo, might be good to have it be a general util function. but i'm fine either way

@dexterju27 dexterju27 merged commit 30d359d into main Nov 7, 2022
@dexterju27 dexterju27 deleted the fix-emojis branch November 7, 2022 22:49
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants