Learning by building with AI

Learn by building is a philosophy I have tried to embrace with AI for the past few months. So I was following my curiosities and creating experiments and building things.

I genuinely felt like I was starting to understand the partnership more and more, and it has really grown over this whole time. But I did notice some pain points and frustrations and confusions, so it was good to explore those, and possible things to combat some.

I pictured a future where learning to use AI would become part of my personal life and professional life, which it has.

I thought recording the process and the findings was for myself. It is also a way to show my thought process. Now and in the future, showing the thinking is more relevant than ever.

Three thought experiments. Claude and I each answered, then we compared how we got there.

You hear sometimes that a model trained on so much tries to land on the most relevant, heaviest average. But creativity is something more than that.

During brainstorming sessions, it would be quick to converge while I was still trying to expand and explore. I noticed that a bunch of times, and that was interesting to me.

Object gravity

We started from a classic creativity test, listing as many uses for a brick as you can, which reached me through an episode of The Thinking Machine. Then we switched to a wine cork and ran fresh Claude sessions with no history.

Asked to free-associate from the cork, a session landed on memory: cork, wine, occasions, people, nostalgia. We suspected our own worked example was leaking its ending, so we stripped the example out and ran it again. It still landed on memory.

Then we reversed it. Three far destinations were picked first, verdict, tide and whistle, and the session had to find a route from the cork to each. All three worked, each one through a different true property of the cork.

Every object seems to have a pull its associations fall toward. Picking a far destination first gets you out. Looking back at the brick session, my prompts like be silly, be fictional, be explosive were already doing that. I was supplying far places, and Claude was building the routes back.

What I built in response

The brick session left us with four named moves. Scatter makes variations along one axis. Traverse jumps to a different axis. Connect ties far pieces into one path that lands. Reverse picks the destination first.

Were the notes worth anything on their own? Two fresh sessions got the same task on a new object. One got only the task. The other also got our three notes, with a prompt that practically asked for a list.

Claude’s prediction, written down before the run: a token gesture at the method, then back to a list.

The plain session wrote a 75-item list. The session with the notes ran the whole method, and found an ending that was nowhere in the notes. The prediction was wrong, and the write-up says so.

I was also thinking about how other new Claude users must be facing the same cold start problem, of where to start and how to move forward, and how to steer things.

Part of the cold start is the blank box, which can take anything and gives you very little to hold. Part of it is mechanical, and that part can be measured.

A small script reads my Claude Code session logs and adds up what each session is already carrying on its first turn, before I have typed anything: skills, tool descriptions, memory, agents.

The floor rose by nearly half in two months, as my setup kept growing. It is one heavy setup, so it says more about where tooling is heading than about an average user.

What I built in response

We talked about how a human can maybe hold six small things in working memory, while a frontier model has a context window of a few Harry Potter books. That disparity can cause a misalignment in the partnership.

In a long session, threads get opened faster than a person can hold them, and the options that did not get picked quietly disappear.

One thing we noticed along the way. For about two weeks I explored the same questions in two places: in Claude Code, where every idea became a dated file, and in a plain chat with ChatGPT. The file side produced fifteen named pieces we could pick back up and test. The chat side had good ideas and almost nothing to pick back up, apart from ten notes I copied out by hand.

What I built in response

A lot of this pointed at the interface. A session’s state is invisible. You cannot see what it has loaded, what kind of work you are in, or what the model thinks the goal is.

So I sketched Desks: one desk per part of your life, each holding its own memory, skills and agents, with a row of hats for the kind of work happening right now.

One rule held the sketch together. Every claim the interface makes about its own state has to show how sure it is: enforced is solid, declared is dashed, unknown is greyed out. A checkmark has to mean the thing is really loaded.

A small version ran in my terminal: a status line showing the active room and what it had loaded, and a hook that reminded the model of the declared hat on every turn.