Fix exponential scheduler with bias

[andrey-churkin]

Looks like that current implementation does not work properly.
Consider the following example:

a, b, k = _init_exp(10, 0.1, 0.7)
x = a * np.exp(-k * 10) + b
print(x)

I expected that x is equal to 0.7 but I got 0.666666666666667.

[andrey-churkin]

Jenkins please retry a build

[alexsu52]

def get_b(a, k): -> def get_b(a):

[andrey-churkin]

Done.

[lzrvch]

@mkaglins @AlexKoff88 I suppose we could re-run some filter pruning benchmarks to see diff in accuracy for this scheduler once this fix is introduced?

[mkaglins]

@mkaglins @AlexKoff88 I suppose we could re-run some filter pruning benchmarks to see diff in accuracy for this scheduler once this fix is introduced?

This exponential_with_bias scheduler is not used in any officially claimed results, but we can run a comparison of exponential (that often used in Imagenet experiments) with this fixed version.

[lzrvch]

Should we make the factor parameter adjustable via config? I suppose the 1/8 value is based on what is set in the paper, not on our empirical results.

[mkaglins]

Yes, it is a good idea to set better default value based on empirical results on our models (firstly get these results). But I am not sure about the importance of this param in the config for users.