LLMs are the pair programmer you’ve always wanted
You can annoy the s#*t out of them all day
Recently I’ve been doing a lot of coding for some projects I’m working on. In the course of that work, I’ve been making extensive use of large language models (LLMs) including ChatGPT, GitHub Copilot, and Bard.
My take: While we’re only at the beginning, right now LLMs are the pair programmers you’ve always wanted. You can pester the s#*t out of them all day and they won’t get annoyed or ghost you. You can repeat yourself over and over without the risk of RTFM. You can ask embarrassing questions like how to do that thing you forgot in VIM. And as an added bonus it doesn’t get into tangential religious debates about VIM, EMACS, and VS Code and why you’re asking about VIM in the first place.
I’ve mostly been using ChatGPT along with Copilot in VS Code. I’ve sprinkled in Bard to compare results, when ChatGPT is down 😀, and when I can’t get the answer I want. At the moment I find the ChatGPT interface to be better than Bard, which is a sad moment for me because this user experience is exactly the kind of thing Google used to do better than others. And it isn’t even like ChatGPT has a killer interface, just that Bard is lackluster.
LLMs are amazing productivity tools for answering questions, providing patterns, and generating initial code. This is truly one of those 10x improvement moments where as a PM you can get users to change their behavior. 80% of the time my experience is a step function improvement over what I’ve used search + forums for in the past.
When things are off, I can generally re-prompt in a number of ways to figure out the answer I’m looking for. The rest of the time I fall back to the old ways, which at a meta level is searching forums for others who have had similar questions in the past.
Where have all the software engineers gone? Gone to LLMs everyone one… maybe not
Let’s all take a breath on the “LLMs will eat all jobs” front. It is too hard to predict the future, but I’m not worried that software engineering is going away anytime soon. For simple applications, yeah LLMs can be combined together with deterministic tool chains to create simple apps. This is not new (i.e. website builders and WYSIWYG dev tools) but LLMs are definitely at another level. At the same time, tools that ‘just build it’ have existed for many years but quickly get clunky once you get beyond the simplest functionality.
But you will be able to do more with fewer people. Again, that trend is nothing new. If you’ve been around a while you may remember database and server administrators. Cloud services unlocked more with less, and the trend continues with LLMs.
I am confident LLM tools will be an integral part of development workflows in the future. They will boost productivity of individuals substantially, and propagate design patterns (both good and bad) way quicker than anything we’ve seen before. They will enable motivated engineers to learn new code bases and languages much faster. I think we will see a bunch of specialized tools integrated into tool chains and workflows.
To get the most out of LLM type tools, there are two challenges (and business opportunities) to be solved. First, they will need the equivalent of a copy of the code base in memory. You really want the LLM to understand what’s there already without re-prompting. This will be a huge win for understanding large code bases amongst large teams. As a new member of a team, you’ll be able to ask how things work, what’s there, etc much more efficiently. No more ‘no one even knows how X works anymore’.
Second, the tool needs a way to ensure privacy. Like search engines, privacy really ISN’T an issue for the majority of people. However, for at-risk individuals it is a real risk vector. The same is true for mission critical businesses. And the severity is greater than traditional search simply by the nature of query vs prompt. Queries are relatively short, prompts can contain much more data. You don’t want entire code bases being exposed in the logs of an LLM vendor.
Overall, comparing my before and after coding productivity just using the LLMs available today, I’m nothing less than blown away. We live in exciting times.
