Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resume training not available for cloud trained model/ #604

Closed
1 task done
rolurq opened this issue Mar 8, 2024 · 5 comments
Closed
1 task done

Resume training not available for cloud trained model/ #604

rolurq opened this issue Mar 8, 2024 · 5 comments
Assignees
Labels
bug Something isn't working fixed Bug is resolved

Comments

@rolurq
Copy link

rolurq commented Mar 8, 2024

Search before asking

  • I have searched the HUB issues and found no similar bug report.

HUB Component

Models, Training

Bug

I have a model that trained for 257 epochs of 300. The training was using the cloud agent but at some point i stopped training and now I can't resume.

image

I have no access to the checkpoint or to resume training in any way, which is really frustrating since it has already taken 27 hours of training.

Environment

Ultralytics HUB Version
v0.1.36
Client User Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36
Operating System
Win32
Browser Window Size
2560 x 1305
Server Timestamp
1709905304

Minimal Reproducible Example

  1. Login to Ultralytics HUB
  2. Access interrupted training model
  3. Try to resume

Additional

Bug report from #602

@rolurq rolurq added the bug Something isn't working label Mar 8, 2024
@kalenmike
Copy link
Contributor

@rolurq I can take a look into this for you, what is the model id? YOu can find it in the URL.

@kalenmike kalenmike self-assigned this Mar 8, 2024
@kalenmike kalenmike added the info needed More information is needed for action label Mar 8, 2024
@kalenmike
Copy link
Contributor

@rolurq We are able to replicate this bug in the UI and are pushing a fix now.

@kalenmike kalenmike removed the info needed More information is needed for action label Mar 8, 2024
@kalenmike
Copy link
Contributor

@rolurq The fix is now live, you may need to refresh your page and/or cache to see it reflected.

@kalenmike kalenmike added the fixed Bug is resolved label Mar 8, 2024
@rolurq
Copy link
Author

rolurq commented Mar 8, 2024

Awesome! It's working now. Thanks a lot.

@rolurq rolurq closed this as completed Mar 8, 2024
@UltralyticsAssistant
Copy link
Member

You're welcome, @rolurq! 😊 If you have any more questions or run into any issues, feel free to reach out. Happy training!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working fixed Bug is resolved
Projects
None yet
Development

No branches or pull requests

3 participants