In 1988, a Class Project Fed Random Characters to Unix Utilities, Exposing Crashes and Hangs Across Seven Operating Systems

In the fall of 1988, computer science professor Barton Miller gave his graduate operating systems class at the University of Wisconsin–Madison an unusual assignment: build a program to generate random character streams and feed them into standard Unix command-line utilities. The experiment, which Miller named fuzz testing, revealed that crude, nonsensical inputs could easily crash or freeze basic software tools that programmers relied on every day.

The paper began from an expectation that frequently used operating-system utilities should be reliable. Miller and his students found that between 25% and 33% of the utility programs they tested crashed or hung when reading random characters, a weakness that showed up across seven Unix variants.

The Simplest Possible Test Oracle

The assignment in Miller’s Advanced Operating Systems course (CS736) asked students to write a generator that emitted random streams of ASCII (American Standard Code for Information Interchange) characters. The project specification gave students flexibility to explore different stream options, including printable ASCII characters, all ASCII characters, null bytes, and random-length lines.

Once built, the generator directed these unguided streams into familiar Unix utilities such as mail, vi, sed, awk, sort, and cc. The testing approach relied on an intentionally primitive criterion: if an application terminated abnormally (a crash) or stopped responding to input (a hang), it failed the test. If it processed the input and exited normally, it passed.

The utility under test could produce nonsense or unhelpful error messages and still pass, provided it neither crashed nor hung. This simple pass-or-fail rule—the test oracle—required no model of the utility’s expected output.

Three Teams and One Working Bug Hunter

The concept was simple, but making it work took careful experimental discipline. Three student groups in Miller’s fall 1988 class attempted the project. Two of the groups failed to achieve any crash results. Miller’s later account does not identify a specific technical reason for their failure.

The third group, consisting of graduate students Lars Fredriksen and Bryan So, succeeded. Their tools recorded random input streams to disk, and their scripts saved crash dumps and the offending input for later investigation. When they pointed their generator at common utilities, program after program collapsed.

Crashes Across Seven Unix Variants

Fredriksen and So expanded their tests across seven commercial and academic Unix variants. Miller later summarized the results as crashes or hangs in 25–33% of the utilities tested on these systems. The paper traced crashes in ftp and telnet to gets(), which could overrun input buffers, and identified unbounded string reads in tsort, the topological-sort utility.

The failures were not caused by subtle logical paradoxes. The diagnosed failures included unchecked array bounds, invalid pointers, and unsafe use of C input routines such as gets, which cannot check the destination buffer’s bounds. The paper also identified other causes, including unchecked return values, subprocess failures, faulty error handling, and race conditions.

Publishing the Data and Facing Skepticism

In December 1990, Miller, Fredriksen, and So published their findings in Communications of the ACM (Association for Computing Machinery) under the title “An Empirical Study of the Reliability of UNIX Utilities.” Beyond reporting vendor bugs, the authors made their fuzz generator source code, test scripts, raw crash logs, and proposed code patches openly available.

In his 2008 foreword, Miller recalled strong resistance from members of the testing and software engineering community who objected to the approach’s lack of a formal model and methodology. He described fuzz testing as a supplement to more systematic testing: another tool for finding bugs.

What Fuzz Testing Can and Cannot Prove

The lasting influence of Miller’s 1988 experiment lies in understanding what black-box random testing actually measures. Passing a fuzz test does not prove that a program is correct, secure, or free of logical errors. An application can silently miscalculate data or return wrong answers while still exiting with a success code.

In this experiment, fuzzing tested robustness by feeding applications random input and checking whether they crashed or hung. A passing result did not establish memory safety. Still, the approach endured: in 2022, the 1990 study received the Jean-Claude Laprie Award in Dependable Computing from the International Federation for Information Processing (IFIP) Working Group 10.4.

The simple idea that began in a Wisconsin classroom in 1988 demonstrated that before software can be trusted to reason correctly, it must first be built to withstand the unexpected.

Explore

Stay connected