Skip to content
This repository has been archived by the owner on Nov 11, 2023. It is now read-only.

Add multi gpu preprocess support #331

Merged
merged 10 commits into from
Jul 22, 2023
Prev Previous commit
Next Next commit
Update preprocess_hubert_f0.py
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
  • Loading branch information
Stardust-minus and github-actions[bot] committed Jul 22, 2023
commit bdb81281a23655ef459a3dc25cedd58013d7fe17
1 change: 0 additions & 1 deletion preprocess_hubert_f0.py
Original file line number Diff line number Diff line change
@@ -1,23 +1,22 @@
import argparse
import logging
import torch.multiprocessing as mp
Stardust-minus marked this conversation as resolved.
Show resolved Hide resolved
import os
import random
from concurrent.futures import ProcessPoolExecutor
from glob import glob
from random import shuffle
from utils import list_file
import librosa
Stardust-minus marked this conversation as resolved.
Show resolved Hide resolved
import numpy as np
import torch
from tqdm import tqdm

import diffusion.logger.utils as du
import utils
from diffusion.vocoder import Vocoder
from modules.mel_processing import spectrogram_torch

logging.getLogger("numba").setLevel(logging.WARNING)

Check failure on line 19 in preprocess_hubert_f0.py

View workflow job for this annotation

GitHub Actions / ruff

Ruff (I001)

preprocess_hubert_f0.py:1:1: I001 Import block is un-sorted or un-formatted

Check failure on line 19 in preprocess_hubert_f0.py

View workflow job for this annotation

GitHub Actions / ruff

Ruff (I001)

preprocess_hubert_f0.py:1:1: I001 Import block is un-sorted or un-formatted
logging.getLogger("matplotlib").setLevel(logging.WARNING)

hps = utils.get_hparams_from_file("configs/config.json")
Expand Down Expand Up @@ -153,8 +152,8 @@
if args.clean:
print("Cleaning previous preprocessed files....")

files = list_files(path, {".npy"}, recursive=True, sort=True)

Check failure on line 155 in preprocess_hubert_f0.py

View workflow job for this annotation

GitHub Actions / ruff

Ruff (F821)

preprocess_hubert_f0.py:155:17: F821 Undefined name `list_files`

Check failure on line 155 in preprocess_hubert_f0.py

View workflow job for this annotation

GitHub Actions / ruff

Ruff (F821)

preprocess_hubert_f0.py:155:28: F821 Undefined name `path`

Check failure on line 155 in preprocess_hubert_f0.py

View workflow job for this annotation

GitHub Actions / ruff

Ruff (F821)

preprocess_hubert_f0.py:155:17: F821 Undefined name `list_files`

Check failure on line 155 in preprocess_hubert_f0.py

View workflow job for this annotation

GitHub Actions / ruff

Ruff (F821)

preprocess_hubert_f0.py:155:28: F821 Undefined name `path`
for f in files:

Check failure on line 156 in preprocess_hubert_f0.py

View workflow job for this annotation

GitHub Actions / ruff

Ruff (F821)

preprocess_hubert_f0.py:156:17: F821 Undefined name `list_files`

Check failure on line 156 in preprocess_hubert_f0.py

View workflow job for this annotation

GitHub Actions / ruff

Ruff (F821)

preprocess_hubert_f0.py:156:28: F821 Undefined name `path`
f.unlink()

print("Done!")
Expand Down
Loading