Overfitting

[johanna27045]

While setting up your neural network to on it with own data, I was wondering if you have any tactics to prevent overfitting used in your code. Is there some kind of monitoring and early stopping?

[yinbow]

If your data scale can not be enlarged, maybe you can try:
(1) A higher dropout rate helps prevent overfitting by ensuring the network doesn’t rely too much on any individual node. You can change it in the config file

DFormer/local_configs/NYUDepthv2/DFormer_Large.py

Line 25 in 935944d

C.drop_path_rate = 0.15

(2) Reduce Model Complexity: You may choose a small model, i.e., DFormer-B instead of -L.
(3) You mentioned monitoring and early stopping: Our framework can monitor the mIoU at the validation set and save the best one. It has the same effect as the early stopping
The current framework does not support the following (4)-(6). Maybe you need to implement them.
(4) More Data Augmentation: You can refer to https://github.com/VCIP-RGBD/RGBD-Pretrain/blob/main/data/auto_augment.py for more data augmentation for the RGBD data.
(5) Cross-Validation: Use cross-validation to ensure that the model’s performance is consistent across different subsets of the data.
(6) Ensemble Methods: Combine predictions from multiple models to reduce variance and improve generalization. But this will sacrifice the model's efficiency.

[johanna27045]

Okay thank you!