Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Task] Implement ranking models integration tests #785

Open
gabrielspmoreira opened this issue Oct 4, 2022 · 0 comments
Open

[Task] Implement ranking models integration tests #785

gabrielspmoreira opened this issue Oct 4, 2022 · 0 comments
Assignees
Labels
chore Maintenance for the repository ci

Comments

@gabrielspmoreira
Copy link
Member

gabrielspmoreira commented Oct 4, 2022

Goals ⚽

Create integration tests for ranking models (based on our research scripts) to check for

  • Stability of our API (unit tests) - by using advanced options (hparams) of the APIs that might break if changed
  • Accuracy and Throughput (integration tests) - Compute accuracy and throughput metrics for training and evaluation, and log properly so that it is possible to track over time if we have a drop in those metrics.

Starting point

  • Integration tests for retrieval models implemented in Integration Tests for Retrieval models #537
  • Research scripts from the MTL research, including preprocessing of AliCCP and training/eval of ranking models for Single Task Learning (MLP, DLRM, DCN-v2, DeepFM, Wide&Deep) and Multi-Task Learning (MMoE, PLE)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore Maintenance for the repository ci
Projects
None yet
Development

No branches or pull requests

1 participant