LLMs: Experts, or Infinite Recall Machines?
Thousands, or perhaps millions of blog posts have been written about the effect that LLMs will have on human productivity. And for a variety of reasons, at least for now the impact is more significant for software development than it is for nearly any other task. Much of what has been written has been about what the impact is and will be for programmers, but I want to write here about why I feel like LLMs have certain strengths in software development.
Developers often won't admit to it, but a lot of the job is drudgery. Most of the history of computer science can be described as an effort to get rid of the drudgery and make programming easier. Developers get sick of doing the same menial things over and over again, so they invent better ways to do their jobs.
When it comes down to it, the bulk of all development work is simple and routine: a user wants to change their email address. Another order came in that we need to save to the database. A book needs to be added to our catalogue. We need to send data to this service and get an analysis back. Most of the lines of code in an application aren't complicated or cutting-edge in any way. They are dealing with one piece of data at a time, one operation at a time, and there's just not a lot of creativity or innovation needed for that type of work.
Developers used to write programs by punching holes in paper cards. Which, as you would expect, took a very long time. So ways of inputting instructions with a keyboard were invented, and then people could type assembly language with commands like mov rdx and xor rdi. And life was so much better! Then we started to see what we think of as modern computer languages, and we would just write fputs("hello world\n", stdout); to get something on the screen. But, you have to have a lot of lines of code to worry about where the text actually shows up on different systems, and how you buffer and flush the output, etc. So people started developing languages where you simply type out an instruction of print "Hello World", and move on. No thinking about where the memory for the text was stored, or what to do with it, or instructing the compiler how to get it to the screen, etc.
Going from punch cards to print "Hello World" is of course a huge improvement, but what is it about the progress that specifically makes a developer more productive? Even if we don't look clear back to punch cards, why is there a difference between fputs("hello world\n", stdout); (with all the surrounding code to make that line do what it is supposed to do) and print "Hello World"? Are developers just lazy and we like to type less and reach for fewer different keys on the keyboard? Is the benefit nothing more than gaining the few seconds it takes to type those extra lines?
In my opinion one of the major benefits of higher-level languages (like JavaScript, Python, C#, Ruby, etc.) is that they allow developers to better manage their own (human) working memory for what they are working on. Much of the industry progress over the last few decades has been focused on making software easier to reason about, or making it so that we can keep smaller chunks of information in our heads and compartmentalize away everything else. When I write print "Hello World", I spend very little of my limited human working memory on printing something to the screen — leaving more room to remember other things.
Microservices architecture made it so that entire parts of a massive, complicated system could be broken down into compartmentalized, defined inputs and outputs that are easily documented, referenced, and reasoned about. ORMs were invented so that people could think in a single language without needing to translate between, say, Python and SQL. If your (human) working memory is focused on some data being represented in your program, thinking about it strictly in terms of Python is easier than remembering all the SQL needed to access the data from a database, or to save it, etc. Even developers who avoid ORMs inevitably take the applicable concepts of an ORM and implement what they need into their application, which is another way of avoiding the overhead of all the database grunt work.
Now taking the arc of improvements in software development to our current point, we have LLMs. My experience with LLMs leads me to believe that in a practical sense, their killer trait that makes them a useful tool is that they have infinite working memory. I don't mean that in the strictest sense since even the very best LLMs can't hold everything they need to know directly in memory, but they are very good at quickly "remembering" what they need and retrieving information in an efficient manner. It's not that I directly use that ability to remember and extend my own working memory, but their capability for fast, infinite recall allows them to handle all the tedious stuff outlined at the beginning of the post with ease.
When a mere mortal like myself gets hung up on the exact syntax for how to insert data into a database in a way I haven't done recently, I have to think about how to phrase a search in Google. And then I have to click through 4 pages, realize my query wasn't specific enough, and keep trying until I find exactly what I needed. And by that point I have forgotten what I was trying to even insert into the database.
An LLM doesn't have that constraint. A good coding harness either already has the information encoded in its model, or it can sequence through the searching and retrieval process in an automated way that doesn't disrupt whatever else it is working on. The LLM can pause its work, look up how to do something, and jump back to the original task without losing anything at all. It can even start working on something else while it waits for the research to be completed. Meanwhile, I somehow find myself on the Wikipedia page for RAS syndrome chuckling at how the page is itself an example of redundant acronym syndrome syndrome. I don't even remember how I got on the page — wasn't I supposed to be inserting some data somewhere?
A study was done on how developers work1, where they found that only 7% of development sessions started immediately with the developer typing code in their editor. The other 93% involved navigating somewhere else before a single line of code was written. Google, other code, internal documentation, etc. — the destination didn't really matter, but the pattern of "I need to do something, let me go look it up" is extremely prevalent with all developers throughout the entire development process.
The entire paper is fascinating and worth a read as it explores the impact of interruption during programming tasks, and strategies that developers use to manage their own context. Without spending a lot of time elaborating on what is written there, the human brain just isn't wired to hold that much information at once. The "smarter" somebody is, and the better they are at managing their working memory (either by having a larger inherent capacity, or by developing approaches to working around their own limitations), the better they usually perform as a programmer. These approaches to managing working memory include jotting down notes, keeping appropriate documentation close at hand, getting good at searching through and navigating around the codebase, etc.
So basically, compared to a human being, a modern LLM and coding harness has:
- A perfect attention span. It's not seduced into rabbit holes 5 pages deep on Wikipedia, or by interesting articles in the sidebar on Stack Overflow.
- Perfect context retention throughout starting, researching, and finishing tasks. There are no gaps between needing to insert data into a database, looking up how to do it, and coming back to write the code.
- Effectively perfect memory of all programming knowledge. Some of it is encoded in the LLM directly and some of it needs to be looked up, but all of it is at arm's reach and there is basically no retrieval cost.
When this infinite working memory is paired with infinite persistence to keep looping until something works, you get the power of the coding harness.
None of this is to say that AI development is perfect. Every LLM I have ever used tends towards severely over-engineering things. And they get stuck in "doom loops" where they need help to figuratively pick their heads up and see the forest for the trees. These are still tools rather than fully autonomous systems that can fully execute a vision without intervention, and while they will undoubtedly continue to improve I don't see them getting to that fully autonomous capability anytime soon. The number of people arguing whether Fable, Opus 4.8, or Opus 5 is superior seems to point to a capability asymptote — or at least to the fact that we seem to be seeing diminishing returns with new frontier improvements.
There is so much variation in usage that there isn't much consensus on what it is that coding harnesses are good or bad at, but they generally need guidance around what engineers consider the "design" or architecture of a system. This design might be around an excellent UX that perfectly guides a user through a complex process, or it might be the decoupling of different components of a larger system, or it might be about directing different data tasks to the right type of database.
As an example of something I haven't seen LLMs be good at, I have yet to see them design a good relational data schema. And that might not be so much a limitation of the LLM as it is the nature of the task — but it still highlights the strengths and weaknesses of LLMs. "Build a good schema" is inherently subjective and doesn't work well as a prompt. "Write a function that returns the cumulative product of an arbitrary number of integers" is much less subjective, so LLMs will generally do better on the second type of task while struggling with the first.
Taking a bunch of business rules and relationships between real-world concepts and turning that into a well-designed relational schema for software that can grow without painting itself into a corner isn't an easy task for humans, and it's even harder for LLMs. The LLM will be good at answering questions like "Have I reached the 4th normal form everywhere on this schema?", but it's a completely different task to take a complex business process and model it out accurately from scratch. All the hard work is in gathering the right information and presenting it in a way that the LLM could design the right schema, which is basically exactly the same problem humans have with the process.
Treating LLMs as tools, I've found that giving them things that they are good at while being aware of the things that they aren't good at makes all the difference. And having a mental model of a super fast lookup-research-write-test loop helps frame their strengths and weaknesses as I use them. My job is now to keep the really big picture in my own working memory — data structures, components in a microservices architecture, a desired architectural pattern for the front-end, how business rules map to software systems, etc. Those always needed to be done, but an LLM lets you live more in the bigger picture than was possible before as you delegate everything else down to the LLM. When developers talk about treating their coding harness like a bunch of junior developers, the approach is most effective when you can keep the big picture in your own working memory while letting either the junior developers or the coding harness use their own working memory for the details.
Coding harnesses are a tool, and like most complex tools it takes practice to learn to use them well. Knowing what to delegate and where to step in can take some practice, but it is a valuable skill to develop. Knowing how to set the harnesses up for success is a skill. Thinking in terms of "context management" is a skill. Letting these harnesses know what success looks like in a concrete and well-defined manner is another skill.
All of these different skills come back to using LLMs and coding harnesses for what they are good at. If you treat a coding harness as a universal, system-wide expert, you'll likely delegate the wrong things — the business process mapping, the architecture, the judgment about what the business actually needs — and you'll get back something that turns out to be confidently wrong. If you treat it as a machine with perfect recall and endless patience, you can hand it the tedious majority of the work and see how far it gets.
It's the same thing every improvement in programming has been about: freeing things up in your working memory so you can work more effectively. Punch cards to assembly to print "Hello World" to a harness that can go look up how to do something without losing its place. Every element of progress let developers stop remembering something so they could think about something bigger, and AI-based coding is just one more step in that direction.
Notes
- Chris Parnin and Spencer Rugaber, Resumption strategies for interrupted programming tasks, Software Quality Journal 19(1), 2011. Based on an exploratory analysis of 10,000 recorded sessions from 86 programmers, alongside a survey of 414 more.