winning

A generalization of factor multinomial probit models that scales to millions of variables

This package prices the winner of many: N competitors whose performances can be correlated through common factors, and for the independent and factor grammars can follow any smooth noise distribution (the block, nested and tree kernels are Gaussian, and say so rather than silently ignoring a base). One pass over a shared survival field gives all N win probabilities, and the same field answers removal and photo-finish queries; complete all-pairs tables of those have quadratic output size and are priced per query. Calibration runs in the other direction, from observed shares back to abilities. Factor probit, Luce's softmax and the finite-temperature softmin are special cases of one parameterized race.

Quick start

from winning import race_probabilities, calibrate_abilities, removal_shares

p = race_probabilities(mu, V=V, D=D)     # all N shares, one field pass
mu_hat = calibrate_abilities(p, V=V, D=D)  # abilities back from shares
q = removal_shares(mu, V=V, D=D)         # P(j wins | i removed), all pairs

Max-wins semantics for the probit literature live in winning.probit (shares, calibrate_utilities, supplied-covariance fitting); the density-agnostic engine in winning.thurstone; the paper-faithful kernels in winning.factor.

One race, five covariance grammars

Every model in the package is the same Gaussian min-race $Y = \mu + \text{noise}$; the grammars are declarative descriptions of the noise covariance that admit $O(N)$-per-lattice-point evaluation. Pass any of them as structure= to the front-door verbs.

from winning import (Independent, Factor, Blocks, Nested, Tree,
                     race_probabilities, polish_race)

tree = Tree.from_linkage(Z)                    # floored cophenetic correlation
race_probabilities(mu, structure=Factor(V, D))        # Σ = VV' + diag(D)
race_probabilities(mu, structure=Blocks(cluster, loading, D))
race_probabilities(mu, structure=Nested(cluster, loading, D, coupling, gamma))
race_probabilities(mu, structure=tree)
polish_race(p0=w, structure=tree, name_caps=0.05)     # nearest race under caps

Tree.from_linkage builds the race whose implied correlation is the floored cophenetic correlation of a hierarchical clustering (merge height maps through $\rho=\max(1-2h^2,0)$; negative cophenetic correlation has no tree representation) — so a portfolio's HRP weights can be inverted, re-priced and polished along the dendrogram they came from. Analytic fixed-grid Jacobians (rows sum to zero; off-diagonals are photo-finish tie densities) ship for the factor, block and nested grammars, with preconditioned Newton inversion throughout. An arbitrary dense $\Sigma$ is priced by fitting the grammar family (seriated blocks under global factors) and promoting the residual to extra factor dimensions — one seeded, reproducible call that beats million-draw Monte Carlo on wall clock at $n = 2000$ and prices tail probabilities Monte Carlo cannot see.

Languages

The numpy implementation is the spec. The compiled kernels (fastrace, thin pyo3 bindings over the pure-rust winning crate) accelerate every path — 232× on the classic calibration, input-bound to ten million runners on the block field — and the base-R package mirrors the complete API with no compiled dependency. All ports are parity-locked: parity/vectors.json embeds inputs and outputs from the reference, and every language replays the same twenty-two scenarios — nineteen to within 1e-7 (most to 1e-10), the three fit- or optimizer-mediated ones to between 5e-4 and 5e-3.

pip install winning          # pure python (numpy/scipy)
pip install winning[fast]    # + compiled kernels (abi3 wheels)
remotes::install_github("microprediction/winning", subdir = "r/winning")

Random test covariances come from randomcov: fifteen named ensembles, because “a random covariance matrix” means nothing until the measure is named.

Papers

• Scalable inversion of contests with correlated performances, including softmax and multinomial probit (2026, arXiv:2609.01133, SSRN working paper). All shares of a correlated race in one O(QNL) pass, graph-Laplacian derivatives applied matrix-free, and inversion of observed shares back to utilities — forward pricing benchmarked to ten million alternatives, inversion to one million. Softmax is the Gumbel special case of the same lattice. Ships as winning.factor; every experiment is a committed script under research/experiments/. Supersedes the earlier share-calibration draft.
• Exact posterior probability of optimality for factor-Gaussian beliefs (2026, draft). The probability that an alternative is truly best, declared intractable in reinforcement learning and approximated there from marginal means and variances alone, is exact and linear-time whenever the posterior covariance is factor-plus-diagonal — and two standard observation processes, common random numbers and per-arm Gaussian observations, keep it in that form at every update. Measured consequences: stopping on the exact vector holds the nominal error at between a third and two-thirds of the replicate spend of the marginal-only rules, and sampling from the exact vector reproduces Thompson sampling's regret to within replication noise, as the occupancy identity requires. Experiments under research/rs_crn/.
• A posterior predictive for pass@k (2026, note). An independent evaluation of the published empirical-Bayes pass@k estimator on released rollout records, under a design that genuinely extrapolates: two training attempts per prompt, six held out. The plug-in extrapolation understates aggregate pass@6 by seventeen points and its impossibility verdicts are contradicted a third of the time; shrinkage repairs most of the damage, posterior integration a measured further step, and the threshold decision a published diagnostic takes changes materially. Scripts and the outcome table under research/cavity_calculus/exp2_passk/.
• Computation of collision local times: a connection between new scalable probit methodology and Atlas models (2026, research note). The derivative of a win probability is the accrual rate of a collision local time: for Brownian races the photo-finish flux at horizon s is the rate at which the leading pair's local time accumulates, verified against sixty thousand simulated paths, and computable at scale — a Russell-3000-sized field prices its full rank-1/2 collision-rate curve in under two seconds. Sketches market-implied index leakage (reconstitution turnover as rank-boundary local time) via a top-k flux formula, derived and not yet implemented.
• Softmax masking is a choice model: the evidence for linear versus Gaussian renormalization of probability vectors (2026, provisional draft). When an outcome is withdrawn — scratched, delisted, or masked out of a softmax — proportional renormalization is the reflex and, in many models, the architecture. Psychology has tested it under restriction since 1957 as the constant-ratio rule; this study scores it out of sample against re-running the Gaussian contest among the survivors, over thirty-nine population comparisons. Gaussian renormalization has the lower held-out log loss in thirty, with the losses concentrated on one-dimensional, strongly confusable stimulus sets.
• Rating Formula 1: a case for non-Gaussian noise in rating systems (2026). Seventy-five seasons of grands prix, a retirement-slab noise density, a qualifying control, and a stratified test against historical bookmaker odds.
• Inferring relative ability from winning probability (SIAM, 2021). The lattice algorithm this package grew from: recovering ability densities from a vector of win probabilities, as in pari-mutuel markets.

Older presentations

• Who ya gonna call? and How to respond to an RFQ: the same order statistic in trading, where the best response to a request for quote is a horse race entry.

Demos

• Watch it converge — choose a covariance ensemble and a field size; the lattice answers in milliseconds while frequency simulation crawls its inverse-square-root slope, live in the browser.
• Polish your portfolio — concentration caps without breaking model-consistency: the nearest race satisfying your limits, solved live in the browser with the analytic race Jacobian, showing the redistribution running along the correlation structure.

• Fit a correlated race — the inverse problem, live: drag observed shares and a correlation, and the engine calibrates standalone abilities in the browser, against the naive independent fit.
• The photo-finish circuit — interactive: the race as a resistor network, computed live in your browser by the shipped algorithm. Click a runner to delete it and watch the share flow to factor-similar survivors, against the IIA prediction.
• race_field_demo.py — one shared survival field prices every competitor by division; the naive O(N²) route against the O(N) field.
• cavity_downdate_demo.py — the rank-one cavity twin: one inverse contains every leave-one-out inverse.
• js/factor — the calibration core in the browser: dependency-free JavaScript at machine-precision parity with the Python; it powers the circuit demo above.

Heritage

winning began as the reference implementation of the SIAM paper above. By Peter Cotton, with sibling packages at microprediction. The same computation — inferring latent ability from who wins — has been rediscovered field by field for a century, from Thurstone's 1927 law of comparative judgment to language-model leaderboards: the heritage, as a timeline →

Notes: disks do not fail independently — a common-cause factor in the public Backblaze data, and what it costs erasure-coding durability.

Get the source

github.com/microprediction/winning · pip install winning · Research scripts