A football model has an impressive historical record. Before treating that record as evidence of forecasting skill, ask a simpler question: could it have produced those predictions with the information available at the time?
Football prediction models need more than a headline accuracy figure. A useful evaluation explains when forecasts were made, which matches were included and how the method was tested on results it had not already seen.
What does a backtest actually show?
A backtest applies a defined prediction process to historical matches and compares its outputs with recorded outcomes. It can help reveal weaknesses and compare approaches, but its usefulness depends on how closely it reproduces the original forecasting conditions.
Rebuilding predictions today is different from preserving forecasts published before kickoff. Both can contribute evidence, but the report should identify which kind of record it contains. A reconstructed forecast should never be presented as a timestamped prediction that readers could have seen at the time.
Check when every input became available
Data leakage occurs when an evaluation uses information that would not have been available for the prediction being tested. It can make historical results look stronger than performance on genuinely new cases. The scikit-learn guidance on leakage explains this problem and the importance of separating training and evaluation data.
The following football examples are illustrative checks, not findings about a particular service:
| Input | Question to ask |
|---|---|
| Team form | Does the average exclude the match being predicted and all later fixtures? |
| League position | Is it the table available before that fixture, rather than the final season table? |
| Starting lineup | Had it been announced by the stated forecast time? |
| Odds | Was that exact price available when the forecast was issued? |
For example, a forecast supposedly issued at noon cannot use a lineup first announced at 18:45. A later model version may legitimately use that lineup, but it represents a different publication time.
Test later matches after earlier ones
Football forecasts arrive in sequence. An evaluation should preserve that direction: learn from earlier information, then assess later fixtures. Randomly mixing dates can allow future information to influence what is presented as a historical forecast.
For time-ordered evaluation, scikit-learn documents splits that train on earlier observations and test on later ones. Football datasets also need appropriate date grouping: matches at the same forecasting cutoff should not accidentally feed information into one another.
In a hypothetical study, a developer could build a method using 2023–24 fixtures, choose settings using 2024–25, then evaluate the frozen approach on 2025–26. These dates describe an example design, not an OptaTips result.
If the developer repeatedly changes the method after inspecting that final season, the season has become part of development. Fresh later evidence is needed to assess the revised version.
Compare models on the same task
Suppose two hypothetical methods predict regulation-time home wins, draws and away wins for the same 100 matches. Method A gets 60 results right; Method B gets 58. That gives observed accuracies of 60% and 58% for this sample. It does not establish a reliable long-term advantage.
Now suppose Method B’s published record contains only 20 selected matches. Its percentage answers a different question. Compare both methods on their shared fixtures, or explicitly report selection coverage and exclusions. Do not quietly compare a selective record with an all-fixture record.
A simple benchmark, such as always selecting the home team, can provide context on the same test set. Choose it in advance rather than searching afterward for whichever comparison makes the model look strongest.
Look beyond correct picks
A model assigning 51% and one assigning 90% to an event can both select the same outcome. Accuracy alone does not describe the quality of those probability estimates.
Probability scoring rules and calibration checks provide additional information. Brier score and log loss evaluate probabilistic forecasts, while a calibration plot compares groups of predicted probabilities with observed frequencies. A lower Brier score alone does not prove better calibration; scikit-learn explains that distinction.
For a plain-language explanation of percentages, read what a 60% football prediction means. No performance metric is calculated for OptaTips in this article.
Five questions to ask before trusting the record
- What exact outcome and settlement period were predicted?
- Were all inputs available by the stated publication time?
- Were the final evaluation matches kept separate from model development?
- Are comparison methods tested on equivalent fixtures and selection rules?
- Does the report disclose its full period, sample, exclusions and model changes?
For your own review, retain the fixture, forecast timestamp, model version, original probabilities and eventual result. Separate revised forecasts from original ones. A complete record lets readers examine the claim; a screenshot of selected successes does not provide the same evidence.
Featured image: AI-generated editorial illustration, not actual model performance.
For adults aged 18 and over. This guide explains evaluation methods. Historical performance does not guarantee future results or profit.
Comments 0
No comments yet.