Abstract

I challenge the common wisdom of “make it work, make it right, make it fast”. Sometimes, you have to make it fast, then make it work. When working on machine learning, you have to process huge amounts of data, and code errors are almost impossible to debug or even detect without analysing that data at scale. As with any software engineering task, bugs are fixed through repeated cycles of finding and fixing a bug, then moving on to the next one. If you write slow code, each cycle of bug-fixing (“make it right”) takes a long time. But if you first focus on the performance of the program (“make it fast”) and then start fixing bugs, the amount of time spent in the bug fixing phase could be lower. I present an anecdote with absolutely no statistical evidence whatsoever.

Introduction

I couldn’t possibly write a blog post today without mentioning LLMs in some way.

In today’s software engineering culture there seems to be a huge focus on churning out code as fast as possible. That means less review of the code, and perhaps more importantly, less thoughtful structuring. In all the “slop” codebases I’ve worked on, a common pattern is that the same task will be solved in three different ways in three different places. LLMs do love tricolon, after all. More different places solving the same problem means more chances that one of them has a bug, or worse, that you don’t even remember to fix the bug in all three at once! If you’re writing a data-intensive pipeline, with this less-structured code also comes less thought about the flow of data through the pipeline. While there is a good chance that the LLM approximates an existing efficient pipeline structure from training data, that’s unlikely to remain the case through many agent debugging cycles. Of course, the agent can detect when things are running slowly and try to optimise it, but that usually involves weird compiler-level hacks rather than thoughtful structure for efficiency. The end result is that by the time you, the human, need to debug the code, it’s a nonsensical mess that runs slowly. Much the same as if you had a team of undergraduates writing it, which is how I got into this situation.

For a coursework, we were asked to write a deep reinforcement learning agent. Not a particularly difficult task in general, but we chose an unusual problem space so we had a fair bit of reward shaping to do, and we weren’t allowed to use RL libraries. At the point where we were beginning to debug our broken code, I started trying to optimise our code, for example by using numpy functions rather than pure-python maths, and by batching operations together.

One team member said there was no point optimising until we finished fixing the bugs, and that’s what I’m refuting.

My hypothesis

For us to fix the bugs, we needed to run the full pipeline and check what was happening at each stage (often by looking at the standard deviation of various inputs and outputs to see if data was getting lost or rescaled). Then we needed to tweak each section, either by fixing a bug or tuning a hyperparameter.

I think that if we had not spent time optimising the pipeline early on, we would have taken much longer to debug it. Of course, there were overheads introduced by the optimisation, for instance when a numpy method gives an error it can often be much more obscure than the corresponding pure-Python messages. However, I suspect that it was worth it.

That’s all

I don’t think this post really deserves a conclusion since I didn’t do any research. I suppose you could call it inconclusive.