Add Linear Tree Coefficients to get_leaf_output method

[bammari]

Summary

In the method get_leaf_output() the method does not return the coefficients and intercept when the booster is a linear-tree. Please add the linear_coeff and linear_const information to this output to avoid having to extract this information from a JSON when using the dump_model() method.

Motivation

This is useful because linear trees have been shown to be beneficial in several engineering applications. For example see [1].

Being able to extract these coefficients and intercept will allow for embedding these models within mixed-integer programming models and optimization problems. (e.g. see the Optimization & Machine Learning Toolkit [2]).

Description

Currently, the get_leaf_output() method returns a constant value, even if the booster is trained as a linear tree that is - lgb.Dataset(X, y, params = {linear_tree : True})

The get_leaf_output() method should return a list or tuple of the leaf's linear coefficients (slopes) corresponding to the number of features in the model as well as the intercept. (i.e. if leaf 1 returns y = mx + b, this get_leaf_output() should return m and b).

References

[1] Ammari et al. (2023) "Linear model decision trees as surrogates in optimization of engineering applications." Computers & Chemical Engineering 178, 108347

[2] https://github.com/cog-imperial/OMLT