No Iterations

If you think about programming languages you really need to think about what are they for?.¹

A programming language is what is between you and the computer.

It is the interface between the ideas on one side and the stuff inside the computer on the other side.

Maybe programming languages are for many things but here is a list of some really important things:

1. Expressing ideas
2. Expressing ideas to the computer
3. Make the computer think what I think
4. Make the computer tell me what it thinks
5. Change some entity in the computer (file, process, … )

The first idea is that making a big computer program is building a larger process out of smaller ones. I suppose you could truthfully say that sculptors make large shapes from stuck-together grains of clay. But that shows what's wrong with the bits-and-bytes approach. No sculptor ever thinks that way, nor scientist, or programmer. An architect first thinks of shapes and forms, then walls and floors, and only last about how those will be made.

Minsky (1984)²

Data structures, language constructs, and at the very lowly bottom, syntax are the small things that are not what programming is about. The way that architecture is not about clay.

A language is there to allow us to modify, make experiments, and observe. Nothing more, nothing less.

Using this mental framework there is no fundamental difference between operating systems, programming languages, libraries, frameworks, user interfaces, and even text editors.³

These are all societies of processes, some with collections of members with different emphases.

The hypothetical ideal language is completely out of the way. It would have infinitely small friction, infinitely small steps, effort and wait time to make my ideas have effects on the computer. It would get rid of the clay entirely and we would rise higher and beyond as architects riding the ideas.

We would be able to instantly and losslessly transfer thoughts to the computer and make the program change to our desire. We would use the competence of the computer seamlessly intermingled with the rest of our thoughts. Experiments would be thinking, with perfect reach, perfect control, and perfect access to the vast competencies, memories and remote connections of the computer realm. Racing at an unknown pace, building new kinds of towers of abstraction, thought, reason and creative spirit. Eventually hooked up to some constructors,⁴ our thoughts would shape the stuff of the universe.

From this view, building ever more powerful languages is a path to a technological singularity.

The most powerful programming language is Lisp. If you don't know Lisp (or its variant, Scheme), you don't know what it means for a programming language to be powerful and elegant. Once you learn Lisp, you will see what is lacking in most other languages.

Stallman

Magic is what makes a great book great. It is something that happens in the mind of the reader, not elsewhere.

It has to do with masterful craftsmanship, with love, with the spark between people. It has to do with the right time and the right place, how could it be otherwise? A piece of magic comes together with the most masterful output of a masterful craftsperson. It needs to be the right time and place, else it would not be peak output.

We live in a world where we cannot measure magic yet. Otherwise, we could be churning out great movies, books and games.

You might not know it yet, but great software and great tools, too, are subject to taste and aesthetics.⁵

Would it not make sense that the people talking about the most powerful language in a sort of mysterious way would allude to taste, too? If it is the most powerful language, it would be powerful because of reasons in the realm of mastery and craftsmanship. It would be because of magic, which is hard to explain and only accessible via aesthetics and taste.

What makes Emacs a superbly sick piece of software? What makes The Man From Earth a superbly sick movie?

The answer is you will know when you know. But you will not know before you know. And you will know for reasons that you have to discover yourself. But it won't be reasons that make no sense. It will be reasons that make perfect sense. Just that they are somewhere where you don't yet know how to look.

The only way to know magic for the moment is to hone our tastes and broaden our horizons.

What does it mean to develop a taste in programming? Stallman is giving a hint that trying Lisp will develop your taste.⁶

Let me throw in another clue: in order to know what different kinds of languages do, you should go and try the ones that are most different from what you already know.

Lisp is mentioned in this way again and again.⁷ It will broaden your horizon, it will stretch what you think about languages.

Can it be true? Does it not sound too good to be true? Why not? What in reality doesn't look like it is true? Is it not a fact that Emacs, a Lisp program, is a freaking psychedelic spaceship? Is it not a fact that society and mainstream are wrong about things all the time?

If Lisp is profoundly more powerful than other languages, then it will be powerful in a way that

• resonates with your programmer's heart,
• you cannot know before you feel it,
• would be hard to explain, because it would be magic.

What is your most profound desire as a programmer? I think it is having effects on the computer.

What I am about to tell you is that there is an activity during programming that you sort of know but don't know. Current programming is stuck with a limitation that simply does not make sense.

You want to make a quick experiment in the code. You make a little side project, and you set up some example data.

You tell your young programmers that they should value quick iterations. You jump through hoops to not have to re-compile right now. And not again. And you juggle in the right amount of changes in your batch so that afterward you see point 2/5 not working, instead of point 22/100.

You suffer when you have long deployment cycles in your feedback loop.

Sometimes you wish you had put a log or breakpoint somewhere because it just happened.

What if you have a complicated thing happening somewhere on the server? What if the program is right now in a state that, if only you could look at it, would tell you what to fix?

Sometimes it would be great to see the counterfactual quickly. What if this if goes into the other branch, does it still work? No, fuck it, commit, push, and tomorrow there is a bug report that x broke.

If you are faster at seeing what the program is doing, that is better.

This you know. And maybe you know from notebooks or some advanced hot reload tech what it would mean to be able to change the program while it is running.

Restart the program and the state is dropped on the floor. But if you see what it does, while it does it, that is a difference between night and day.⁸

Because it matters what is between you and the program.

Around 1959 to 1963, there were the MIT AI people, John McCarthy, Marvin Minsky, and the young bright-eyed hackers who later built the internet and influenced what computing is nowadays.⁹ Supported by J.C.R. Licklider, whose 1960 paper "Man-Computer Symbiosis" imagined people thinking together with an interactive computer, and who funded much of what followed.¹⁰

These people had a vision of what computing can be and they knew it would be interactive.

And maybe one of the best ideas ever: the stored program, usually called the von Neumann architecture.¹⁵

Put the program into the computer memory.

A stroke of genius. Now we have a magic stone that can be loaded up with ideas and then it can be anything.

Before, we had 1 machine with 1 program. What a program is and what a machine is was glommed together. Good design is about splitting things apart, and this was a beautiful and elegant design.

This in a way resonates with Dennett's Tower of Generate-and-Test.¹⁶

First, you have a machine built to behave (Darwinian): a machine with a fixed program.

Then you have a machine that is built to learn (Skinnerian).

And finally (a von Neumann machine), a machine that is built to deal with ideas (Popperian).

Because ideas, the memes, can be anything, including ideas about ideas, the Popperian machine has an open-ended future.¹⁷

Putting more ideas into memory was a very good idea.

The story of dynamic memory data continues with IPL, the language of one of the most important early AI works, the General Problem Solver (Newell, Shaw and Simon; begun 1957, published 1959).

In IPL, the Information Processing Language, you had dynamic data in the program, in lists (made from cells). You can create lists and make them point to something else while the program runs, instead of having to declare variables for everything.

You know of course what this means because you create lists and arrays all the time in your program.

JavaScript has a bit of Scheme in its genes: Brendan Eich was recruited to Netscape with the promise of "doing Scheme" in the browser.¹⁸ And it has a printer and a reader for its data: JSON.stringify and JSON.parse.

Douglas Crockford says he discovered JSON, rather than invented it: it was already there in how JavaScript writes its object literals.¹⁹

If you know the power of JSON, you know the power of having dynamic data in your program. Your program is not about customers and payments, it is allowed to be a level higher. Your program is allowed to be about data structures. Data abstraction is something like SQL or .NET LINQ, or a JSON parser. It is a language for talking about data and data transformations. map, filter, reduce don't know about customer names, they know about lists, sequences, streams.

This is the heritage of von Neumann's memory idea, propagated forward through time and one level up. Your program is loaded dynamically into the computer, but your program is also dynamically loading data into its memory. Your program is, in some way, simulating a von Neumann machine itself.

McCarthy took list processing from IPL, but not its assembly-like form, tied to one machine (the JOHNNIAC).²⁰ He had the idea of taking those lists and describing the computer process entirely in terms of them.

Let's take those lists, make the first thing the verb and implement a lambda calculus.

To understand Lisp, imagine you read JSON and treat the data as a command language for what your program does next.

function read(expression) // returns jsonData

function eval(jsonData) // returns an evaluated object

Eval is the big meaty function of your program. It needs to take that JSON, go through it and interpret the data as commands.

function print(object) // print to the user

function repl() {
    while (true) {
        var whatTheUserSaid = readFromTerminal();
        var jsonData = read(whatTheUserSaid);
        var object = eval(jsonData);
        print(object);
    }
}

This repl function can be the entry point and the game loop of our program. Now the user can sit at the console and interact with our language.

We define a few special operators. These are the things eval has to implement in the static part of the program:

• set: set a variable.
• pair: make a pair; a pair of pairs then is a list.
• first: the first value in a pair.
• rest: the second value of a pair (this can be a simple value or another pair).
• lambda: make a function.
• if: conditional evaluation.
• quote: return the jsonData instead of evaluating it.

This is a program that is simulating interactive computing. Where an operating system gives you access to files and processes, a Lisp gives you access to a society of symbols, functions, values, and data structures.

var globalEnv = {}; // a dict, hash map, lookup table

function define(symbol, value) // augment the global env

Make our Lisp interpreter know about the command define.

function eval(expression) {
    if (expression[0] == "define") {
        var symbol = expression[1];
        var value = expression[2];
        define(symbol, value);
    }
    // ...
}

Lisp:

["define", "foo", 10]

(This is valid Bobscheme.)

This is the cleaning up that McCarthy did to computer programming. Lists where the verb is at the front, no extra stuff. Put it into memory so you can modify it while it grows. Make a symbol type so you can give names to things, from inside the language. Make a function type so that functions are the same stuff as the rest of the program.

The fact that JSON is so useful is passed down directly from list processing. All the objects nest, there are only a few kinds of objects, and it bottoms out at atoms. A printer and a reader come shipped by default. I know without knowing that JavaScript gurus know the power of print, compared to the little machines you get from the C++ line of heritage.

McCarthy wanted to sit at a terminal and make the program do different things while he was programming it. He did not want to say up front all the variables in the system, because he wanted something more dynamic for programming AIs.

The interactivity is at the bottom. Hence it is called a building material.

You make a program that asks you what should be next in the program?. It is as simple as this. From this, all the goodness that makes Lisp elegant and beautiful comes. The simplicity of this enables easy-to-reason-about programs that write programs.²¹

With this, we can have a conversation with the program. We can directly interact, while the program is running.

Here is the conversation, concretely. A web server, started once:

(defn handler [req]
  {:status 200 :body "hello"})

;; pass the var #'handler, not the function value
(def server (jetty/run-jetty #'handler {:port 8080 :join? false}))

Now I change my mind about what the server says:

(defn handler [req]
  {:status 200 :body (str "hello, " (get-in req [:params :name] "stranger"))})

I evaluate this one form. The next request gets the new answer. No restart, no rebuild, and every other piece of state in the server is still there. That is not a faster iteration. There was no iteration. I told the running program something, and it listened.

And when a request does something strange, I keep it:

(defn handler [req]
  (def last-req req) ;; the real request, now a value in my REPL
  ...)

Then I call the pieces of the handler on last-req, one form at a time, until I see where it goes wrong.

Since I wrote this post, a new kind of programmer showed up at the REPL: the coding agent. On the Clojure codebase I work on, my agent evaluates forms in the same running REPL I use. It does not guess what a function returns from reading the source. It asks the program.

That is the same lesson the IBM research director missed, one more time. "Why don't they just think more carefully?" is what you would ask of a language model that only reads code. The better answer is the same as in 1960: give it a conversation with the machine.

Open questions:

• If agents converse with running programs, does "the source code" stop being the main artefact, and the running image become it again, as in Smalltalk?
• What does an agent need that a human at the REPL does not: a record of the conversation, a way to undo, a sandbox?

The power of Lisp is not quick iterations, but no iterations. Instead of iterating, in Lisp and Smalltalk we have a conversation. This concept is reified in the read eval print loop.

It is enabled by the idea of putting the program into the dynamic memory of your program. This resonates with the von Neumann idea of putting the program into the memory of the computer.

Jack Rusher's bottom line is don't settle for anything with less interactivity than Lisp and Smalltalk!

In a Lisp program, you can write programs that write Lisp programs. So it has an open future. … to me, programming has not changed much in 50 years, because they got locked into this strange set of limitations.

Marvin Minsky, The beauty of the Lisp language

Once you have in your fingers how to leverage the repl, it will be as obvious as the sky, vast and inescapable. This is better, this is simpler, and this gets stuff out of the way. This lets me use my brain the way a monkey reaches for a banana. Directly, the way I can delete and create files on my PC. I can define and undefine symbols in the program. I can make quick experiments while the program is running, without the mental distress of setting up the state.

It is the same difference that interactive computing makes compared to batch processing. It is not a small step, it is not a big step, it is not a step at all. It is a different world.

Whatever that is. At this date and time, I will call it Magic.

• Clojure and Interactive Programming
• Lisp
• Bobscheme, I make a toy Lisp implementation using JSON

Imagine a painter, fervently engaged in a dialogue with her canvas. Each stroke isn't a hurried message jotted down and sent off, anxiously awaiting feedback. No, the painter watches each trail of color blend and mix under her brush, immediately seeing the effects of her choices. She tries different pressure, strokes, colors, and brushes, in a cycle of continuous creation and observation, a dance between her mind's eye and the canvas. This is her conversation with her art - immediate, beautiful, and ever-changing.

Now, consider a programmer, in front of his screen. The code portrayed is not a fixed monolith of logic. Instead, it morphs, breathes, and reacts under his fingertips, mirroring that painter's canvas. The programmer writes a line of code, and the effects echo through the system immediately, no abstraction barrier held in between. The immediate feedback enables him to stir into different directions, tease out underrepresented nuances, debug on the fly and explore new vistas of his creation.

The code is fluid under the hands of the programmer in a perpetual dance of creation, modification, and observation. It is an intimate conversation with the ideational fabric that weaves itself into existence as the program - the programmers' thought reflections observed immediately, altered rapidly, and understood fully.

This, my friend, is the spirit of interactive programming. A live, pulsating creation constantly in flux rather than fixed scrolls etched into digital stone, navigating creativity at the speed of thought inspired by immediate feedback, riding the great prairies of abstraction and thought, reaching a level of intimacy and understanding with your program that feels like a piece of music resonating with your heartstrings.