Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[egs] Speeding up i-vector training in voxceleb v1 recipe #2421

Merged
merged 8 commits into from
May 14, 2018
Merged

[egs] Speeding up i-vector training in voxceleb v1 recipe #2421

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
f5dffd3
[egs]: updating the voxceleb recipe so that it uses more of the avail…
May 4, 2018
4aa206c
[egs]: fixing comment error in mfcc.conf
May 4, 2018
cdb876d
[egs] updating voxceleb/v1/run.sh results
May 8, 2018
463d286
[egs] changing url to download voxceleb1 test set from, updating READMEs
May 10, 2018
c6d692f
[egs] fixing comment in voxceleb/v2/run.sh
May 10, 2018
a350405
[egs] adding check that ffmpeg exists in voxceleb2 data prep
May 10, 2018
6504840
[egs] subsampling the i-vector training data in voxceleb/v2, otherwis…
May 14, 2018
d38213b
[egs] merging upstream master
May 14, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 17 additions & 5 deletions egs/voxceleb/v1/run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ fi
if [ $stage -le 1 ]; then
# Make MFCCs and compute the energy-based VAD for each dataset
for name in train voxceleb1_test; do
steps/make_mfcc.sh --mfcc-config conf/mfcc.conf --nj 40 --cmd "$train_cmd" \
steps/make_mfcc.sh --write-utt2num-frames true \
--mfcc-config conf/mfcc.conf --nj 40 --cmd "$train_cmd" \
data/${name} exp/make_mfcc $mfccdir
utils/fix_data_dir.sh data/${name}
sid/compute_vad_decision.sh --nj 40 --cmd "$train_cmd" \
Expand All @@ -58,15 +59,26 @@ if [ $stage -le 2 ]; then
fi

if [ $stage -le 3 ]; then
# In this stage, we train the i-vector extractor.
#
# Note that there are well over 1 million utterances in our training set,
# and it takes an extremely long time to train the extractor on all of this.
# Also, most of those utterances are very short. Short utterances are
# harmful for training the i-vector extractor. Therefore, to reduce the
# training time and improve performance, we will only train on the 100k
# longest utterances.
utils/subset_data_dir.sh \
--utt-list <(sort -n -k 2 data/train/utt2num_frames | tail -n 100000) \
data/train data/train_100k
# Train the i-vector extractor.
sid/train_ivector_extractor.sh --cmd "$train_cmd --mem 20G" \
sid/train_ivector_extractor.sh --cmd "$train_cmd --mem 16G" \
--ivector-dim 400 --num-iters 5 \
exp/full_ubm/final.ubm data/train \
exp/full_ubm/final.ubm data/train_100k \
exp/extractor
fi

if [ $stage -le 4 ]; then
sid/extract_ivectors.sh --cmd "$train_cmd --mem 4G" --nj 40 \
sid/extract_ivectors.sh --cmd "$train_cmd --mem 4G" --nj 80 \
exp/extractor data/train \
exp/ivectors_train

Expand Down Expand Up @@ -107,5 +119,5 @@ fi
if [ $stage -le 7 ]; then
eer=`compute-eer <(local/prepare_for_eer.py $voxceleb1_trials exp/scores_voxceleb1_test) 2> /dev/null`
echo "EER: ${eer}%"
# EER: 5.53%
# EER: 5.419%
fi
2 changes: 1 addition & 1 deletion egs/voxceleb/v2/run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -203,5 +203,5 @@ if [ $stage -le 12 ]; then
echo "EER: ${eer}%"
# EER: 3.224%
# For reference, here's the ivector system from ../v1:
# EER: 5.53%
# EER: 5.419%
fi