[R-package] Serialization can lose boosting rounds and lead to mismatches in predict

[david-cortes]

The R package keeps a best_iteration number that it uses to serialize model objects. If one de-serializes a model that had a best iteration number lower than the actual number of iterations, further rounds after that will be lost and the predictions will not match.

I think the best solution here would be to create a new C-level function to dump the model that would not take any additional parameters, dumping instead everything in the booster, and use that to handle automated serialization of models in R/Python.

Example using code from the tests:

library(lightgbm)
data(agaricus.train, package = "lightgbm")
train <- agaricus.train

set.seed(708L)
dtrain <- lgb.Dataset(
  data = train$data
  , label = train$label
  , params = list(num_threads = 1)
)
dvalid <- lgb.Dataset(
  data = test$data
  , label = test$label
  , params = list(num_threads = 1)
)

early_stopping_rounds <- 5L
bst_auc <- lgb.train(
  params = list(
    objective = "binary"
    , metric = "auc"
    , max_depth = 3L
    , early_stopping_rounds = early_stopping_rounds
    , verbose = 1
    , num_threads = 1
  )
  , data = dtrain
  , nrounds = 100L
  , valids = list(
    "valid1" = dvalid
  )
)

pred_old <- predict(bst_auc, agaricus.train$data[1:5, ], num_iteration=99)
fname <- file.path(tempdir(), "lgb_model.Rds")
saveRDS(bst_auc, fname)
bst_new <- readRDS(fname)
lgb.restore_handle(bst_new)
pred_new <- predict(bst_new, agaricus.train$data[1:5, ], num_iteration=99)

print(pred_old - pred_new)

[1]  0.09444400 -0.06709325 -0.06576094  0.09444400 -0.06249008