A simple example with exogenous data

[SultanOrazbayev]

Hello! I checked the forecasting tutorial and some of the threads in this repo's discussion, but still have trouble figuring out how to run a prediction with just exogenous data.

Here's a sample example, where the time series element is absent as I'm trying to narrow down the problem in grokking sktime. This is a basic y=a+bX problem, I expect perfect forecasting accuracy for this linear model:

from pandas import DataFrame
from sklearn.linear_model import LinearRegression
from sktime.forecasting.compose import make_reduction
from sktime.pipeline import sklearn_to_sktime

df = DataFrame({"x": [0, 1, 3, 4, 5, 8, 15, 16]})

df["y1"] = 5 + 3 * df["x"]
df["y2"] = 10 + 10 * df["x"]
df["y1true"] = 5 + 3 * df["x"]
df["y2true"] = 10 + 10 * df["x"]
df.loc[6:, "y1"] = None
df.loc[6:, "y2"] = None
print(df)

mask_fit = df["y1"].notna()
Xf = df[mask_fit]["x"]
yf = df[mask_fit][["y1", "y2"]]

mask_predict = df["y1"].isna()
Xp = df[mask_predict]["x"]
yp = df[mask_predict]["y1"]

# this is not quite what I want, but I don't know what else I should use for this problem
forecaster = make_reduction(LinearRegression(), strategy="recursive", window_length=1)

forecaster.fit(y=yf, X=Xf)
print(forecaster.predict(fh=[1, 2, 3], X=Xp))
# expect perfect forecast

From the documentation for make_reduction, I understand that this is not what I want (it introduces lags of y, which is not what I am trying to achieve). I also looked into TrendForecaster and the sklearn_to_sktime utility, but they are not relevant either.

[SultanOrazbayev]

Hmm, it looks like what I want is DirectReductionForecaster:

from pandas import DataFrame
from sklearn.linear_model import LinearRegression
from sktime.forecasting.compose import DirectReductionForecaster
from sktime.pipeline import sklearn_to_sktime

df = DataFrame({"x": [0, 1, 3, 4, 5, 8, 15, 16]})

df["y1"] = 5 + 3 * df["x"]
df["y2"] = 10 + 10 * df["x"]
df["y1true"] = 5 + 3 * df["x"]
df["y2true"] = 10 + 10 * df["x"]
df.loc[6:, "y1"] = None
df.loc[6:, "y2"] = None
print(df)

mask_fit = df["y1"].notna()
Xf = df[mask_fit]["x"]
yf = df[mask_fit][["y1", "y2"]]

mask_predict = df["y1"].isna()
Xp = df[mask_predict]["x"]
yp = df[mask_predict]["y1"]

forecaster = DirectReductionForecaster(LinearRegression(), window_length=1)

forecaster.fit(y=yf, X=Xf, fh=[1,2])
print(forecaster.predict(fh=[1, 2], X=Xp))

[SultanOrazbayev]

However, I don't quite understand the purpose of fh in .fit. Is it to allow the prediction of any lagged variables (if specified in the problem), so that .predict can use those values?

[SultanOrazbayev]

Also, inspecting the forecaster, I don't understand why there are two LinearRegression objects inside forecaster.forecasters_["y1"]["forecasters"].get_fitted_params() (I expected only one, since i'm looking at forecaster for "y1" only):

{'estimators': [LinearRegression(), LinearRegression()],
 'lagger_y_to_X': {-1: ReducerTransform(lags=[0], shifted_vars_lag=-1),
  -2: ReducerTransform(lags=[0], shifted_vars_lag=-2)},
 'lagger_y_to_y': {-1: Lag(index_out='original', keep_column_names=True, lags=-1),
  -2: Lag(index_out='original', keep_column_names=True, lags=-2)},
 'transformers': None}

[fkiraly]

I don't understand why there are two LinearRegression objects inside

You requested forecasts at two horizons 1 and 2, and LinearRegression is (probably?) not capably of multioutput prediction, so one regressor is fitted per horizon. Note that this is direct not recursive, in recursive you would be applying the same fitted regressor sliding ahead, but in direct you make a direct prediction from behind the cutoff.

[fkiraly]

However, I don't quite understand the purpose of fh in fit

You need to pass fh only in one of fit or predict, but some forecasters - like direct reduction - need it in fit already. Whether a forecaster requires is in fit is annotated via the requires-fh-in-fit tag.

The direct reduction forecaster requires the fh in fit, because you need to lag the y in the training of the tabular regressors according to the fh, against X.

[SultanOrazbayev]

Thank you very much, @fkiraly !