Benchmarks¶

Every model family on the Kaggle Ames training data (1,460 sales), with each model's default configuration. Two views of each model:

Held-out test: the notebook's split (80/20, seed 42; 292 test houses). The linear rows reproduce the published notebook results to within about 5%; see docs/DECISIONS.md D1.
5-fold CV: cross-validation over all 1,460 houses, mean ± standard deviation. Less dependent on which houses land in one split, so this is the column to compare models by.

Regression: predicting the sale price¶

Model	alpha	Test MAE	Test RMSE	Test R²	Test MAPE	5-fold CV RMSE
linear	-	$21,140	$31,791	0.8682	13.20%	$34,316 ± 10,474
ridge	10	$19,006	$30,751	0.8767	11.57%	$33,279 ± 11,554
lasso	300	$18,550	$31,581	0.8700	11.16%	$33,667 ± 12,177
gbm	-	$16,826	$29,462	0.8868	10.02%	$29,586 ± 6,396
gbm + engineered features	-	$16,857	$29,472	0.8868	9.87%	$28,881 ± 6,407

The threshold is the median sale price of the training rows ($165,000 on the notebook split).

Model	alpha	Test accuracy	Precision	Recall	F1	Test ROC-AUC	5-fold CV ROC-AUC
linear	-	94.52%	92.54%	95.38%	93.94%	0.9822	0.9730 ± 0.0070
ridge	30	95.21%	93.28%	96.15%	94.70%	0.9851	0.9779 ± 0.0062
lasso	3	94.52%	92.54%	95.38%	93.94%	0.9869	0.9780 ± 0.0060
gbm	-	95.21%	92.03%	97.69%	94.78%	0.9887	0.9771 ± 0.0051
gbm + engineered features	-	95.21%	92.03%	97.69%	94.78%	0.9870	0.9781 ± 0.0049

alpha is the regularisation strength; for classification C = 1 / alpha. Defaults were chosen by 5-fold cross-validation on the training split only (see Design decisions).
On a single split, small differences are noise: the CV standard deviations show how much a score moves between folds.
Engineered features (total area, ages, bathrooms, porch area) help gradient boosting. For the linear models they are almost exact linear combinations of existing columns, so they change little.

Generated on 2026-11-01 with amesml 0.1.0, Python 3.12.16, scikit-learn 1.9.1, pandas 3.1.0, NumPy 2.5.4.

Regenerate with make benchmark or python scripts/benchmark.py.