Encountered GPU OOM when trainning.

[pigmeetsomebody]

Is training this mode consuming large memory?
When I finally run the train code, it is stopped due to GPU OOM.

The console output says this process has 23.38 GiB memory in use. Of the allocated memory 21.43 GiB is allocated by PyTorch, and 1.50 GiB is reserved by PyTorch but unallocated.
How to reduce the memory occupation when running the train code?

[alexLIUMinhao]

Was the batch size large during your training? This would likely consume a significant amount of GPU memory.