I’ve been spending a bit of time on LinkedIn lately after finally stopping my use of Xitter (you can find me on Mastodon instead). One of the tropes that I see being uncritically spouted over and over again is “I’m sooooooo productive, by spending $gazillion in LLM credits a day I’ll generate my new product at 50,000 lines of code day”.

While I’m applauding your effort to build a new, much needed product to fill a gaping hole in the market place, I’m sorry my dude (because it’s almost always a dude), but all you’re doing is generating technical debt at high velocity for a brief amount of time. Jon Bentley pointed out in his essays that turned into the book “Programming Pearls” (look it up) back in the 1980s that confusing, I’m sorry, measuring productivity by looking at the lines of code written is All The Wrong Measurements.

Good thing LLMs generate 100% bug free code, otherwise we’d be in trouble.

There’s no accounting for future maintainability when you look at the number of lines of code created.

“We’ll just get the LLM agents to fix their mistakes” is a nice approach if you can understand the problem in the generated code well enough to describe to the LLM what it assumed wrong and what it needs to do to fix it. Note that I said “understand” - if you’re generating that much code and I’m being charitable, it’ll take you a few minutes to understand the code and then you can debug it to see what’s going wrong, or tell the LLM to debug it and fix it. But the keyword here is understand.

What most people seem to forget or ignore is that pretty much any LLM is an electric monk connected to all the VCRs, and it doesn’t have the capability of distinguishing good practice from abysmal practice. And it’s not deterministic either, which is kinda sorta useful when it comes to programming.

Before someone calls my a hypocrite or a luddite, I use LLMs for code generation when it makes sense. For example, they are super useful for generating boilerplate code, or taking code written in $popular-language-1 and translate them into $popular-language-2 if you break the original code into small enough snippets. Things tend to get meh rather quickly when you move away from really popular languages simply because there isn’t that much source material for the stochastic word generator to fall back on in niche-ier languages. As someone who was/is thrilled by the proliferation of niche programming languages over the past decade or so, I actually find this concerning but that’s the topic for another blog post.

One advantage I have is that I’ve been building software for almost four decades now, and as a result have gained some understanding of what’s good code and what isn’t, so I can assess how usable and good the output of an LLM is before I merge it into the main codebase. You know, like one would do in a pre-merge code review because that’s exactly what it is. Specifically in a code review for something written by a very junior developer who Googled the solutions to the problem you tasked them with solving.

Some people may remember when 4GLs (or COBOL) were going to make programmers obsolete. If you do, make sure to discuss the need for a colonoscopy with your doctor. That unpleasantness aside, I’m perceiving the same silver-bulletted-ness that I can’t quite see materializing yet that we saw with those technologies. That doesn’t make them useless, it makes them a tool. A tool that we need to learn how to use, and most specifically when we use it.

And let’s use the measure of how long the LLM generated code survives in your codebase as a measure of success, and not how much unmaintainable code it generated between your first and second cup of tea.

Comments are off at the moment as I’m in the middle of a very slow server migration. I expect that I’ll turn comments back on once it’s complete, but that’s likely going to take another month or so