R-package lr_scheluder, clip_gradient and xavier initializer

[Lopezurrutia]

I am trying to port a model from python to R, basically the model call in python is:
model = mx.model.FeedForward(
ctx = dev,
symbol = net,
num_epoch = 35,
learning_rate = 0.01,
momentum = 0.9,
wd = 0.0001,
clip_gradient = 5,
lr_scheduler = mx.lr_scheduler.FactorScheduler(step=epoch_size * lr_factor_epoch, factor = 0.1),
initializer = mx.init.Xavier(factor_type="in", magnitude=2.34))

but I can't find in R the equivalents for lr_scheduler (and mx.lr_scheduler.FactorScheduler) , clip_gradient and mx.init.Xavier.
I guess they have not been implemented in R yet, right?

[thirdwing]

I have to say it is not implemented, although I don't want to.

Any PR will be appreciated. Otherwise, you need to wait until I finish the paper by hand.

[Lopezurrutia]

I am relatively competent at R but I don't have the knowledge in ML to implement those, so I'll wait.
If I can help providing some sort of "cheap (as in free) labour", let me know.

[Puriney]

@thirdwing I'm working on this and more will follow

[Lopezurrutia]

@Puriney @thirdwing

In mx.init.Xavier when shape has a length larger than two, fan_in is incorrect because it contains a vector instead of a number and results in an error message: "Error: expecting a single value"
fan_in should contain the product of the shape elements (except last one), this is what python code uses.
Replacing
fan_in = shape[-length(shape)]
with
fan_in = prod(shape[-length(shape)])
solves the issue.

[thirdwing]

@Gelu74 Thank you!!

Can you send a PR for this?

Besides, can you also add an example in https://github.com/dmlc/mxnet/blob/master/doc/R-package/fiveMinutesNeuralNetwork.md ?

[Lopezurrutia]

I've done the PR, #1587

I am preparing a kaggle_dsb1 image classification example in R, I can add a summary of how it works in fiveMinutesNeuralNetwork.md, or do you prefer just a regression example as the one explained by @Puriney ?

[thirdwing]

I have seen the PR. Let's wait for the testing.

A Kaggle example will be great!

If you can provide more details, I think it can also be a blog post (http://dmlc.ml/).

[Lopezurrutia]

There is nothing fancy in my kaggle example, it is just a port to R of the python code (which is actually a port from cxxnet) .done by @antinucleon https://github.com/dmlc/mxnet/tree/master/example/kaggle-ndsb1

See #1143 (also related is #1316)