Why is 'del' called explicitly?

[developer0hye]

yolov5/train.py

Line 199 in bc48457

del ckpt, csd

@glenn-jocher

Hi jocher!

Why is 'del' called explicitly?

When I ran the below code with and without del ckpt, it printed same maximum batch size.

Can it save memory usage?

If so, can we set larger batch size?

import torch
import torchvision.models as models

def load_model(path, device):
    ckpt = torch.load(path, map_location=device)
    #Process...
    del ckpt
    return True


if __name__ == "__main__":
    torch.backends.cudnn.benchmark = True
    load_model("./yolox_x.pth", device="cuda") # dummy work
    model = models.resnet18().cuda()

    print("model is loaded!")
    batch_size = 1
    while True:
        try:
            dummy_input = torch.randn(batch_size, 3, 224, 224).cuda()
            dummy_output = model(dummy_input)
            
            dummy_output = dummy_output.sum()
            dummy_output.backward()
            
            batch_size += 1
            
            ckpt = model.state_dict()
            del ckpt
        except:
            break
    print(f"maximum batch size: {batch_size}")

Weight Link: https://github.com/Megvii-BaseDetection/YOLOX/releases/download/0.1.1rc0/yolox_x.pth

[glenn-jocher]

@developer0hye deleting a variable frees memory.

AutoBatch

You can use YOLOv5 AutoBatch (NEW) to find the best batch size for your training by passing --batch-size -1. AutoBatch will solve for a 90% CUDA memory-utilization batch-size given your training settings. AutoBatch is experimental, and only works for Single-GPU training. It may not work on all systems, and is not recommended for production use.

Good luck and let us know if you have any other questions!

[developer0hye]

@glenn-jocher

Thanks for your reply! AutoBatch seems good!

However, when I ran the code uploaded on this issue, I couldn't set more batchsize with 'del' comapred to omitting 'del'.

The results show that calling 'del' doesn't save memory usage. I am still confusing.

[glenn-jocher]

@developer0hye they're not related. Variables are typically stored in RAM. Batch size is correlated with CUDA memory.

[developer0hye]

@glenn-jocher
Thanks!