Skip to content

DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning

Notifications You must be signed in to change notification settings

Alexander-H-Liu/dinosr

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 

Repository files navigation

Setup

  • Codebase preparation (based on fairseq)
# we use fairseq to build the model
git clone https://github.com/facebookresearch/fairseq
cd fairseq
git checkout 47e279842ac8776e3964b0e45c320ad1d2ea6096  # we recommend using the commit DinoSR was developed on
pip install --editable ./

# plug in DinoSR
cd examples
git clone https://github.com/Alexander-H-Liu/dinosr.git

Usage

  • Training

    For the list of hyper-parameters, see config file and also model attributes where default settings used in the paper are provided.

# minimal example to reproduce model
python fairseq_cli/hydra_train.py -m \
    --config-dir examples/dinosr/config/ \
    --config-name base \
    task.data=/path/to/prepared/librispeech/ \
    common.user_dir=examples/dinosr &
  • Loading pre-trained model as python object
import fairseq
import argparse
code_path = "examples/dinosr"
fairseq.utils.import_user_module(argparse.Namespace(user_dir=code_path))
ckpt_path = "/path/to/the/checkpoint.pt"
models, cfg, task = fairseq.checkpoint_utils.load_model_ensemble_and_task([ckpt_path])
model = models[0]
  • Fine-tuning pre-trained checkpoint as ASR
# minimal example for fine-tuning with 100hr data
python fairseq_cli/hydra_train.py -m \
        --config-dir examples/wav2vec/config/finetuning \
        --config-name base_100h \
        common.user_dir=examples/dinosr \
        task.data=/path/to/labeled/librispeech/ \
        model.w2v_path=/path/to/dinosr.ckpt \
        task.normalize=True

Pre-trained checkpoint

Pre-trained checkpoint without fine-tuning can be downloaded here.

About

DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages