Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

io: Parse metadata with C engine, restrict to either CSV or TSV #812

Merged
merged 5 commits into from
Mar 31, 2023
Merged

io: Parse metadata with C engine, restrict to either CSV or TSV #812

merged 5 commits into from
Mar 31, 2023

Commits on Mar 31, 2023

  1. filter: Add test to show existing support for metadata delimiters

    @victorlin
    victorlin committed Mar 31, 2023
    Configuration menu
    Copy the full SHA
    c0b13b4 View commit details
    Browse the repository at this point in the history

  2. read_metadata: Restrict possible delimiters when reading 

    Previously, the delimiter could be anything arbitrary. However, all
Augur subcommands that use this function only advertise compatibility
with CSV and TSV. I don't think there's a good reason to support
arbitrary delimiters.
    @victorlin
    victorlin committed Mar 31, 2023
    Configuration menu
    Copy the full SHA
    74e7fe9 View commit details
    Browse the repository at this point in the history

  3. read_metadata: Use the C engine for pandas.read_csv() 

    The python engine was only used to detect the delimiter. Now that the
delimiter is detected separately, use the C engine since it is faster.
    @victorlin
    victorlin committed Mar 31, 2023
    Configuration menu
    Copy the full SHA
    a90f0a5 View commit details
    Browse the repository at this point in the history

  4. Update changelog

    @victorlin
    victorlin committed Mar 31, 2023
    Configuration menu
    Copy the full SHA
    04489f7 View commit details
    Browse the repository at this point in the history

  5. Use constant for valid delimiters 

    Avoids re-defining this list at each use case and prevents them from
getting out of sync.
    @victorlin
    victorlin committed Mar 31, 2023
    Configuration menu
    Copy the full SHA
    9f48ff2 View commit details
    Browse the repository at this point in the history