-
Notifications
You must be signed in to change notification settings - Fork 332
Vit deit #195
Conversation
This reverts commit 8620347. "Temporarilily undeleting convit"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@growlix has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
@growlix has updated the pull request. You must reimport the pull request before landing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@growlix has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
@growlix has updated the pull request. You must reimport the pull request before landing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@growlix has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
@growlix has updated the pull request. You must reimport the pull request before landing. |
@growlix has updated the pull request. You must reimport the pull request before landing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@growlix has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
@growlix , the PR is almost good to go, some CPU tests failures that should be resolved https://app.circleci.com/pipelines/github/facebookresearch/vissl/681/workflows/fc2b7619-69f6-498e-985b-f3c6f623ecb7/jobs/1352 |
@growlix has updated the pull request. You must reimport the pull request before landing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@growlix has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
@growlix there's one error on internal tests.
|
similarly one more
|
…VisslAutoAugment because of name conflict issues
@growlix has updated the pull request. You must reimport the pull request before landing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@growlix has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
@growlix has updated the pull request. You must reimport the pull request before landing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@growlix has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
congratulations on this major milestone @growlix !!! and kudos for all the hard work!!! It's a great contribution to the research community and extremely awesome/high-quality work!! |
Summary: Make sure that KNN can be applied with big models such as FSDP models on big datasets: - Allow the feature extraction to be done separately from KNN (for faster HP search for KNN) - Allow disabling CUDA for the KNN computation, using RAM instead, to fit bigger features - Provide a different KNN implementation that is more memory friendly and avoids loading all features at the same time Rationale for memory optimisations: for features of size 30K, on a dataset such as Places205 with 2.5M images, the total memory required to load the features is 30K * 2.5M * 4 bytes = 280GB memory. Pull Request resolved: fairinternal/ssl_scaling#195 Reviewed By: iseessel Differential Revision: D31938434 Pulled By: QuentinDuval fbshipit-source-id: fc0434e687bfd965e4e8984571b96f4368ecbcd5
Let's do SSL with vision transformers!