Regression with SGTs

SGTRegressor follows the same shape-function tree idea for continuous targets, with the scikit-learn regressor API (predict, score returns R²). This notebook fits one and compares split criteria.

from sklearn.datasets import make_friedman1
from sklearn.model_selection import train_test_split
from sgtlearn import SGTRegressor

X, y = make_friedman1(n_samples=1200, noise=1.0, random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=0)

reg = SGTRegressor(max_depth=4, inner_max_depth=3, random_state=0).fit(X_tr, y_tr)
print("predictions:", reg.predict(X_te[:5]).round(2))
print(f"R^2  train={reg.score(X_tr, y_tr):.3f}  test={reg.score(X_te, y_te):.3f}")

predictions: [18.16  6.38 12.12  9.77 14.17]
R^2  train=0.766  test=0.585

Split criterion: squared_error vs absolute_error

squared_error (default) optimizes mean squared error; absolute_error optimizes mean absolute error, which is more robust to outliers but skips the coordinate-descent refinement, so it is typically faster and less tuned.

for crit in ("squared_error", "absolute_error"):
    r = SGTRegressor(max_depth=4, inner_max_depth=3, criterion=crit, random_state=0).fit(X_tr, y_tr)
    print(f"{crit:16s}: R^2 test={r.score(X_te, y_te):.3f}")

squared_error   : R^2 test=0.585
absolute_error  : R^2 test=0.264

Forests regress too

RandomSGForestRegressor bags regression SGTs and averages their outputs — the regression analogue of SG forests.

from sgtlearn import RandomSGForestRegressor
rf = RandomSGForestRegressor(
    n_estimators=50, max_depth=4, max_features="sqrt", random_state=0, n_jobs=-1
).fit(X_tr, y_tr)
print(f"forest R^2 test={rf.score(X_te, y_te):.3f}")

forest R^2 test=0.711

See structure vs. accuracy (the same depth knobs apply) and TAO (refines regressors too).