A Math Professor Caught the Pentium Dividing Wrong. Five Empty Table Cells Led to a $475 Million Charge.

In October 1994, Thomas Nicely, a mathematics professor at Lynchburg College in Virginia, confirmed that Intel’s Pentium processor sometimes divided wrong. Within three months, the flaw, called the FDIV bug after the chip’s floating-point divide instruction, had led Intel to take a $475 million charge.

Nicely’s guide to the flaw gave the worst-case example, first found by Tim Coe: divide 4,195,835 by 3,145,727. The correct answer begins 1.333820; a flawed Pentium returned 1.333739.

Intel had found the flaw itself by mid-1994. Most Pentium owners learned about it because a mathematician could not make his numbers agree.

Timeline: from a wrong prime count to a $475 million charge

13 June 1994. Nicely’s count of primes up to 20 trillion disagrees with the published value. Over the following months, hunting through his roughly 3,000 lines of code, he finds that his Borland C++ compiler produces wrong code with certain optimizations. “For some time I believed this to be the source of my woes,” he later wrote.

4 October. His revised code hits a new discrepancy: his Pentium-60 and a 486DX-33, an older Intel processor, disagree on sums of twin-prime reciprocals by far more than rounding error could explain. A binary search narrows the problem to one twin-prime pair, 824,633,702,441 and 824,633,702,443.

17–22 October. He reproduces the error on a second Pentium with a different motherboard, in a Power Basic program and in a Quattro Pro spreadsheet. The error disappears when the floating-point unit (FPU) is locked out. On 22 October, a third Pentium, a Packard-Bell on display at a Staples store, shows the error too. “I was now certain that the error was in the FPU of the Pentium chip,” he wrote.

24–30 October. He reports the problem to Intel technical support. By 30 October he has only an informal acknowledgment that Intel reproduced the error on a 66 MHz Pentium, so he emails people and organizations with access to other Pentium systems and asks them to check.

2 November. An Intel program manager calls to say that Nicely’s analysis is essentially correct, that Intel had noticed the problem in its own testing, and that fixed chips are out in sample quantities.

7 November. Electronic Engineering Times reports the flaw. CNN takes it to a mainstream audience about two weeks later.

30 November. Intel publishes a white paper estimating that a typical spreadsheet user would hit the error once in 27,000 years.

12 December. IBM halts shipments of its Pentium-based computers and estimates that a spreadsheet user could see an error as often as once every 24 days. Intel’s stock falls $4.25 to $58.50 before trading is halted at midday.

20 December. Intel offers to replace the processor for any owner who asks. Its chief executive, Andy Grove, apologizes for the old case-by-case policy: “To some people, this policy seemed arrogant and uncaring.”

17 January 1995. Intel announces a one-time pretax charge of $475 million.

Root cause: five table cells that read zero

Intel’s older 486 produced one bit of the quotient per step. The Pentium’s FPU used a faster method that produces two bits per step: SRT division, named after Sweeney, Robertson and Tocher, who devised it independently in 1957 and 1958. At each step, the divider looks up the next quotient digit, a value from −2 to +2, in a table.

According to BYTE magazine in 1995, the table had 2,048 cells, of which 1,066 were meant to hold values. Five cells along one boundary should have held +2. On the flawed chips they were empty, so the divider fetched a zero there, and the quotient came out slightly wrong.

Intel’s white paper blamed an error in a script used to load the table into a programmable logic array (PLA), the circuit that held it. BYTE added that “nobody checked the PLA to verify the table was copied correctly.” In 2024, chip researcher Ken Shirriff studied photographs of the Pentium’s silicon die and described a different cause: 16 entries were omitted “due to a mathematical mistake in the definition of the lookup table,” and five of those trigger the bug. On either account, the hardware did exactly what its table told it to do.

Why testing caught it late, and why Intel judged it minor

The flaw was hard to hit: it needed particular pairs of numbers. BYTE reported that the error usually appeared around the ninth or tenth decimal digit, with a chance of about 1 in 9 billion for random operands. Berkeley professor William Kahan noted in 1995 that the five bad entries went undetected “until too late despite billions of divisions tested.” His SRTEST program, built to find such flaws far sooner than random testing can, found the flaw on its 356th trial division.

Intel judged the risk too small to disclose, The Wall Street Journal later reported. In Intel’s 20 December announcement, Grove said the company’s policy until then had been “to talk with users to determine whether their needs required replacement of the processor.” He explained: “We were motivated by a belief that replacement is simply unnecessary for most people.” Nicely himself judged the flaw’s impact closer to Intel’s estimate than to IBM’s.

The two public estimates differed because their assumptions did. Intel’s 27,000-year figure assumed 1,000 divisions a day on random numbers; IBM assumed about 4.2 million a day, with some numbers used more than others. Kahan’s verdict on the random-number approach: “hardly anyone computes quotients of uncorrelated random numbers.”

What it cost

Intel said the $475 million pretax charge covered “replacement costs, replacement material and writedown of the company’s inventory of the earlier versions of Pentium processors,” and that it reduced 1994 earnings by $0.70 per share.

The episode is often described as a full recall, but Intel replaced chips only for owners who asked. In April 1995, The Wall Street Journal reported estimates that the overall return rate was under 10%, about half a million chips, out of an estimated 5.5 million defective Pentium computers.

What would have caught it

Verify the table itself, cell by cell. Under either account of the cause, checking every cell the divider can reach against the math would have found the gaps.
Test where the hardware can fail, not at random. Kahan’s SRTEST, whose design predates the Pentium’s by a decade, found the flaw almost at once.
Compute critical results two ways. Nicely caught the bug because he compared results across two different processors and distrusted a difference that rounding could not explain.
Estimate risk with real workloads, and tell users early. Owners heard about the flaw from a mathematician, months after Intel knew.

Known bugs should reach users from you first

The five cells were a defect in silicon. The rest of the story was about trust: who knew first, who told whom, and whose estimate to believe. For every known defect in your own systems, ask whether your users will hear about it from you first, or from someone like Thomas Nicely.

Explore

Stay connected