About Personal Predictions
This is a web version of a spreadsheet that has been kept since 2020: a list of yes/no claims about the coming year, each assigned a probability when written, then graded once the year is over. The site stores the claims, scores them, and draws the two charts the spreadsheet was built to produce.
Prediction sets
A set is one list of predictions — usually a year, but nothing is tied to a calendar. A set has a name, a URL slug, the date the predictions were written, and the date they were graded. Predictions within a set are grouped into categories.
Predictions
Each prediction has:
- A claim, phrased so that it is either true or false at grading time.
- A probability from 0 to 100%, or none yet — a claim can be written down before it is priced.
- A resolution: happened, did not happen, voided, or not yet graded. Voided means the claim turned out not to be gradeable — it is excluded from all scoring rather than counted as a miss.
- An optional note and source link.
Brier score
The Brier score is the mean squared error of the forecasts: for each graded prediction, square the gap between the stated probability and the outcome (1 if it happened, 0 if it did not), then average. Lower is better. Answering 50% to everything scores 0.25.
Only predictions that have both a probability and a happened/did-not-happen resolution are scored. Voided and ungraded rows are excluded from the average entirely, rather than counted in the denominator.
The base-rate baseline shown next to it is what the same set would have scored if every prediction had been given the set's own overall hit rate instead of an individual probability.
Calibration
The calibration chart groups graded predictions by the probability stated, and plots what fraction of each group actually happened. A forecaster whose 70% predictions happen 70% of the time sits on the dashed diagonal. Points below the line mean overconfidence; above means underconfidence. Dot size shows how many predictions are in each bucket.
Predictions below 50% are folded into their complement before grouping: a claim given 30% that did not happen is counted as a 70% claim that did. The two are the same forecast written in opposite directions, so counting them separately would split every bucket across two mirrored curves. Folding affects the calibration chart only — the Brier score is unchanged by it, and the distribution chart shows probabilities exactly as written.
Admin codes and privacy
There are no accounts. Each set has an admin code, chosen when the set is created, which is required to edit it. A set marked private also requires that code to read: it is hidden from the set list and its data is not served without the code.
The set list shows public sets by default. Entering an admin code switches it to show only the sets that code administers — private and public alike — so the list is scoped to yours rather than mixed in with everyone else's.
Codes are stored as plain text on the server and can be rotated from a set's settings. Do not reuse a password. Anyone can create a set; new sets are private by default.
Editing
Unlocking a set makes its table editable in place — claim text, probability, resolution and notes all save as you change them. Alongside that:
- Paste takes a block of predictions, one per line, and previews how each line will be read before adding them. A number at the end of a line is treated as the probability.
- Grade pending filters the table down to predictions that still need a resolution.
- Export CSV produces a flat file of every prediction in the set.