I think my software engineering job will be safe so long as big companies keep using average code as their training set. This is because the average developer creates unnecessary complexity which creates more work for me.
The way the average dev structures their code requires like 10x the number of lines as I do and at least 10x the amount of time to maintain... The interest on technical debt compounds like interest on normal debt.
Whenever I join a new project, within 6 months, I control/maintain all the core modules of the system and everything ends up hooked up to my config files, running according to the architecture I designed. Happened at multiple companies. The code looks for the shortest path to production and creates a moat around engineers who can make their team members' jobs easier.
IMO, it's not so different to how entrepreneurship works. But with code and processes instead of money and people as your moat. I think once AI can replace top software engineers, it will be able to replace top entrepreneurs. Scary combination. We'll probably have different things to worry about then.
> Whenever I join a new project, within 6 months, I control/maintain all the core modules of the system and everything ends up hooked up to my config files, running according to the architecture I designed. Happened at multiple companies
I am regularly tempted to do this (I have done this a few times), but unless I truly own the project (being the tech lead or something), I stop myself. One of the reasons is reluctance to trespass uninvited on someone's else territory of responsibility, even if they do a worse job than I could. The human cost of such a situation (to the project and ultimately to myself) is usually worse than the cost of living with status quo. I wonder what your thoughts are on this.
Humans don’t learn to write messy complex code. Messy, complex code is the default, writing clean code takes skill.
You’re assuming the LLM produces extra complexity because it’s mimicking human code. I think it’s more likely that LLMs output complex code because it requires less thought and planning, and LLMs are still bad at planning.
Totally agree with the first observation. The default human state seems to be confusion. I've seen this over and over in junior coders.
It's often very creative how junior devs approach problems. It's like they don't fully understand what they're doing and the code itself is part of the exploration and brainstorming process trying to find the solution as they write... Very different from how senior engineers approach coding when it's like you don't even write your first line until you have a clear high level picture of all the parts and how they will fit together.
About the second point, I've been under the impression that because LLMs are trained on average code, they infer that the bugs and architectural flaws are desirable... So if it sees your code is poorly architected, it will generate more of that poorly architected code on top. If it sees hacks in your codebase, it will assume hacks are OK and give you more hacks.
When I use an LLM on a poorly written codebase, it does very poorly and it's hard to solve any problem or implement any feature and it keeps trying to come up with nasty hacks... Very frustrating trial and error process; eats up so many tokens.
But when I use the same LLM on one of my carefully architected side projects, it usually works extremely well, never tries to hack around a problem. It's like having good code lets you tap into a different part of its training set. It's not just because your architecture is easier to build on top, but also it follows existing coding conventions better and always addresses root causes, no hacks. Its code style looks more like that of a senior dev. You need to keep the feature requests specific and short though.
> About the second point, I've been under the impression that because LLMs are trained on average code, they infer that the bugs and architectural flaws are desirable
This is really only true about base models that haven’t undergone post training. The big difference between ChatGPT and GPT3 was OpenAI’s instruct fine tuning. Out of the box, language models behave how you describe. Ask them a question and half the time they generate a list of questions instead of an answer. The primary goal of post training is to coerce the model into a state in which it’s more likely to output things as if it were a helpful assistant. The simplest version is text at the start of your context window like: “the following is code was written by a meticulous senior engineer”. After a prompt like that the most likely next tokens will never be the models imitation of a sloppy code. Instruct fine tuning does the same thing but as permanent modifications to the weights of the model.
You're just very opinionated. Other software engineers just give you space because they don't want to confront you and they don't want any conflict with you as it's just waste of time.
6 months is also average time it takes people like you to burn out on a project. Usually starting with relatively simple change/addition requested by customer that turns into 3 month long refactor - "because architecture is wrong". And we just let you do it, because we know fighting windmills is futile.
Depends on the size and complexity of the problem that the system is solving. For very complex problems, even the most succinct solution will be complex and not all parts of the code can be throwaway code. You have to start stacking the layers of abstractions and some code becomes critical. Like think of the Linux Kernel, you can't throw away the Linux Kernel. You can't throw away Chromium or the V8 engine... Millions of systems depend on those. If they had issues or vulnerabilities and nobody to maintain, it would be a major problem for the global economy.
companies have been abandoning products for decades, and shuffling ongoing support onto other entities. nothing has to be "thrown away" as you keep suggesting.
Even if a throw away and replace strategy is used, eventually a system's complexity will overrun any intelligence's ability to work effectively with it. Poor engineering will cause that development velocity drop off to happen earlier.
Although it's sad, I have to agree with what you're alluding to. I think there is huge overhead and waste (in terms of money, compute resources and time) hidden in the software industry, and at the end of the day it just comes down to people not knowing how to write software.
There is a strange dynamic currently at play in the software labour market where the demand is so huge that the market can bear completely inefficient coders. Even though the difference between a good and a bad software engineer is literally orders of magnitude.
Quite a few times I encountered programmers "in the wild" - in a sauna, on the bus etc, and overheard them talking about their "stack". You know the type, node.js in a docker container. I cannot fathom the amount of money wasted at places that employ these people.
I also project that actually, if we adopt LLMs correctly, these engineers (which I would say constitute a large percentage) will disappear. The age of useless coding and infinite demand is about to come to an end. What will remain is specialist engineer positions (base infra layer, systems, hpc, games, quirky hardware, cryptographers etc). I'm actually kind of curious what the effect on salary will be for these engineers, I can see it going both ways.
If they became big companies with that "unnecessary complexity", maybe code quality does not matter as much as you want to believe. Furthermore, even the fastest or well behaved horses were replaced.
The way the average dev structures their code requires like 10x the number of lines as I do and at least 10x the amount of time to maintain... The interest on technical debt compounds like interest on normal debt.
Whenever I join a new project, within 6 months, I control/maintain all the core modules of the system and everything ends up hooked up to my config files, running according to the architecture I designed. Happened at multiple companies. The code looks for the shortest path to production and creates a moat around engineers who can make their team members' jobs easier.
IMO, it's not so different to how entrepreneurship works. But with code and processes instead of money and people as your moat. I think once AI can replace top software engineers, it will be able to replace top entrepreneurs. Scary combination. We'll probably have different things to worry about then.