hellokan.ipynb returns NaN instead of formula.

[Stealeristaken]

Hi.

I was trying hellokan.ipynb file. I do some scaleup at training steps like 50 -> 150. In the end train_loss started to return NaN instead of any value.

I thought maybe it's a kernel error. So I re-downloaded the baseline hellokan.ipynb and rerun without any editing. It returned NaN once again. I will drop screenshot about problem.

[KindXiaoming]

Hi, the problem was caused by the appearance of the log function (which is unexpected behavior). This means that the pruning step is not good. Could you show the plot you have after pruning? From feedback from others, you may try model.prune(threshold=5e-2) instead of just model.prune().

[Stealeristaken]

Sorry for late response. I tried both prune options (no-specified threshold and specified threshold) result was the same i am adding pictures
specified

no-specified

[ShuleiCao]

I encountered a similar issue, but I found that increasing the step size helped.

[KindXiaoming]

@Stealeristaken, in block [8], it should again specify the threshold model = model.prune(threshold=5e-2)

[KindXiaoming]

@ShuleiCao Thanks, yes, the pruning results can depend on quite many factors. Training longer will usually end up a sparser network.