filter: --output-sequences silently allows duplicates

[victorlin]

initially reported in #810 (comment)

Current Behavior

When there are duplicate sequence records in the input --sequences, they are propagated to --output-sequences without any warning or error.

Expected behavior

augur filter should exit with an error.

How to reproduce

cat >metadata.tsv <<~~
strain	col1
SEQ1	A
SEQ2	B
~~

cat >sequences.fasta <<~~
>SEQ1
AAAA
>SEQ2
GGGG
>SEQ2
CCCC
~~

augur filter \
    --metadata metadata.tsv \
    --sequences sequences.fasta \
    --output-metadata output-metadata.tsv \
    --output-sequences output-sequences.fasta

cat output-sequences.fasta
# >SEQ1
# AAAA
# >SEQ2
# GGGG
# >SEQ2
# CCCC

Possible solution

Check for duplicates and throw an error similar to

augur/augur/filter/_run.py

Line 187 in da1c89d

raise AugurError(f"The following strains are duplicated in '{args.metadata}':\n" + "\n".join(sorted(duplicate_strains)))

Your environment: if running Nextstrain locally

• Version: augur 25.3.0