Fernando Pérez Started IPython as a 259-Line ‘Afternoon Hack.’ The Notebook That Became Jupyter Took a Decade.

In late 2001, Fernando Pérez, a physics graduate student at the University of Colorado Boulder, started IPython with 259 lines of Python. The notebook he had in mind from the beginning took ten more years and several false starts, and then grew into Jupyter.

He later called that first version “just a little afternoon hack.”

Pérez was using Python to analyze the data for his PhD thesis, and he liked the language. In 2021 he also cited “a bit of an ethical consideration”: sharing his work with mentors back in Colombia “wasn’t possible with proprietary tools because they’re very expensive.” But he knew Maple, Mathematica and the numerical language IDL well, and next to them, or next to a Unix shell, Python’s interactive prompt “felt like a crippled toy,” he wrote in 2012.

His fix rested on one hook. When you type an expression at Python’s prompt, Python passes the result to a function called sys.displayhook, which prints it. Pérez realized that if he installed his own callable object there, it could hold state and keep earlier results for reuse.

His startup file, which Python ran before showing its first prompt, gave the prompt Mathematica-style numbers, In[1]:= and Out[1]=, and cached every printed result, so the fourth one stayed available as _p4. It also loaded Numeric and Gnuplot for number crunching and plotting. He called it a “mini-mathematica” in Python, then added: “femto- might be a better description, in fairness.”

He also had a degree to finish. In a 2021 interview, Pérez said he had been fired by his first PhD advisor and was struggling to complete his dissertation. Another professor in his department, Anna Hasenfratz, supervised him until graduation. She “had the patience to let me ‘productively procrastinate’ by building IPython,” he said — something he “could somewhat justify as a tool for finishing that dissertation.”

A looming deadline and two borrowed shells

Pérez soon read about two other interactive systems for Python, IPP by Janko Hauser and LazyPython by Nathan Gray, both written, like his, by scientists. He contacted the authors. They let him use their code but had no time to join. “As any self-respecting graduate student with a dissertation deadline looming would do,” he wrote, he threw himself full-time into merging all three into the first “real” IPython. “Eventually I did graduate, by the way.”

From the very beginning, Pérez wrote, Mathematica and its notebooks, and Maple’s worksheets before them, were “in my mind as the ideal environment for daily scientific work.” The shell was a start. A notebook was the goal.

Two notebooks that stalled

The first attempt came in 2005, when two Google Summer of Code students started building a notebook with Wx, a toolkit for desktop interfaces. Robert Kern put in more work, but the prototype, Pérez wrote, “never really became fully usable.”

Someone else got there first. William Stein, who used IPython for the terminal interface of his Sage mathematics software, invited Pérez to the first Sage Days workshop in early 2006, where Kern demonstrated the prototype. It was clearly not ready, and Stein began a browser-based notebook of his own. By October 2006, the Sage notebook was already usable. Pérez called Stein’s decision to go separately “completely reasonable and justified.”

IPython tried again in 2006 and 2007, after Brian Granger and Min Ragan-Kelley had joined the project. Ragan-Kelley got a browser-based prototype working. Unlike Sage’s notebook, it tried to keep the “IPython experience,” including direct access to the user’s own files. But it, too, “never reached production quality.” Pérez wrote that he did not recall all the reasons.

Splitting IPython in two

By 2009, the code that had grown out of the 2001 merge was, in Pérez’s words, “an incomprehensible and terribly interconnected mess with barely any test suite.” That summer, with resources from NiPy, a neuroimaging project, Granger was funded to work on IPython full-time. He took the pieces apart and reassembled them into cleaner modules.

In early 2010, on a teaching trip to Colombia, Pérez read about ZeroMQ, a messaging library, and mentioned it to Granger, who wrote Python bindings for it, pyzmq, within two days. On a weekend visit, one of them wrote a kernel — the process that runs the code — while the other wrote a client. In two days they had a Python shell working over ZeroMQ.

With the kernel in its own process, speaking a defined protocol, any interface could drive it: after a single session with the IPython team, Shahrokh Mortazavi and Dino Viehland of Microsoft built IPython support into Python Tools for Visual Studio “without ever asking us a single question,” Pérez wrote.

In October 2010, James Gao, a Berkeley neuroscience graduate student, wrote a quick prototype of a web notebook on the new design. In the summer of 2011, Granger turned it into a fully working system, using WebSockets, the Tornado web server, CodeMirror for editing code and MathJax for rendering math. On December 21, 2011, after four and a half months of intense development, IPython 0.12 shipped with its browser-based notebook.

Even an early retelling shortened that path. A 2012 post by Britain’s Software Sustainability Institute said the first prototype of the IPython Notebook “was coded in Summer of 2011 by Brian Granger.” In Pérez’s account, Granger built the first fully working system that summer from Gao’s 2010 prototype, after the attempts of 2005 and 2006–07.

From IPython to Jupyter

The Free Software Foundation gave Pérez its 2012 Award for the Advancement of Free Software, naming him “the creator of IPython.” On July 8, 2014, he launched Project Jupyter. One of his launch slides asked: “Why is it called IPython, if it can do Julia, R, Haskell, Ruby, … ?” The new name was inspired by Julia, Python and R.

In 2018, the Association for Computing Machinery’s Software System Award for 2017 went to the fifteen members of Jupyter’s steering council as of the project’s November 2016 nomination. In September 2018, by one analysis cited by Nature, GitHub held more than 2.5 million public Jupyter notebooks, up from 200,000 or so in 2015.

The value of being slow

Pérez titled one section of his 2012 retrospective “The value of being the slowest in the race.” The false starts, he wrote, “helped us much to better understand the hard parts of the problem and where the dead ends would lie.” Three lessons from that decade travel well:

Scratch your own itch, then look around. The 259 lines fixed one researcher’s daily annoyance, and two other scientists had already built shells for the same need.
Waiting can let the tools catch up. Pérez credited ZeroMQ, Tornado, WebSockets, MathJax and fast browser JavaScript engines: “Without these tools we couldn’t possibly have implemented what we have now.”
Some features wait on architecture. The working notebook came after the kernel moved into its own process behind a defined protocol; a browser then became one more client.

If a feature keeps stalling, look at what sits underneath it. Less than three weeks after the notebook shipped, Pérez posted the original 259 lines online, labeled “of historical interest only.” The idea they introduced is still there: in today’s IPython, the output of line 12 waits as _12 or Out[12].

Explore

Stay connected