Do disks fail independently?
Storage durability math assumes drives fail independently: mean time between failures, the annualized failure rate, and the guarantees of erasure coding all multiply per-drive probabilities as if the events were unrelated. That assumption is known to be wrong, and this package computes the correlated alternative — a stored dataset that survives until the k-th of its drives fails is a correlated order statistic, and reliability is the min-wins side of the same engine. So the public Backblaze drive-failure data looked like a clean place to show the effect. It turned into a lesson about reading it carefully instead.
The tempting observation
Under independent constant-hazard failure, a fleet's daily failure count is Poisson, variance equal to mean. Backblaze's first quarter of 2025 — about 340,000 drives, 1,067 failures — is over-dispersed by a factor of three, and the excess survives removing a smooth trend, so it is not simply cohorts aging. Six days carry more than three standard deviations of excess failures, and each spans seven to thirteen distinct drive models across all four manufacturers. Detrended, the manufacturers' daily failure residuals are positively correlated — Seagate and HGST, which fail by entirely different physics, at +0.40. A shared factor touching every vendor at once is the natural reading, and it is exactly the common-cause structure the engine is built to price.
Except it is mostly old, and mostly the calendar
Two problems. First, none of the statistics are new. Schroeder and Gibson showed in 2007 that disk failures are not Poisson and carry long-range autocorrelation; Ford and colleagues at Google, in 2010, defined failure “bursts,” showed them rack-correlated, and fed the correlation into durability, proving independence overstates availability by orders of magnitude. The dependent k-out-of-n durability math has a mature copula literature. The over-dispersion here is a re-confirmation on a newer dataset, not a discovery.
Second, and worse, the cross-vendor clustering is very likely an artifact of how the data is recorded. Backblaze's own documentation is explicit: a drive is counted as failed on the day it “vanishes from the reporting population,” reconciled against data-center maintenance ticketing at quarter-end — an administrative date, not the day the platter died. Backblaze runs a weekly maintenance window on Thursdays, and many removals are cosmetic: a firmware update, a loose cable. A technician pulling drives in a service window touches whatever sits in that vault — Seagate and HGST together — and produces exactly a same-day spike across every manufacturer, with no shared physics at all. The data agrees: failures run about 15% higher on weekdays, the first quarter is Friday-heavy, and half its spike days are Fridays — the day after Thursday maintenance. Removing the day-of-week pattern cuts the cross-manufacturer correlation from +0.19 to +0.12; the rest could be irregular batch pulls that no weekly control can catch. Failure dates cannot separate drives that failed together from drives that were recorded together.
What actually holds
The recorded daily failure distribution is over-dispersed, and predictably so: a one-parameter negative-binomial correction fit on the fourth quarter of 2024 predicts the first quarter of 2025's daily counts with 13% lower log-loss than Poisson, and 37% lower on the heaviest days. That is a genuine property of the data — but it describes the recording process at least as much as the drives, and it re-confirms Schroeder and Gibson rather than extending them. The honest conclusion is a negative: the public Backblaze data is the wrong instrument for physical failure correlation, because its failure timestamps are administrative. The real correlated-failure question — the one that does move durability — is Ford and colleagues', and answering it needs true time-of-death and placement, which the public release omits.
The engine still prices correlated durability exactly, given a physical correlated-failure model. What this exercise establishes is only where such a model has to come from, and it is not here. It also, usefully, shows the discipline the claim required: an intriguing correlation, a documented confound, and the willingness to let the confound win.
Reproduction scripts and full findings, including the confound tests → · Heritage · Back to winning