Big bang rewrites are needed in order to move forward faster.
A huge issue with sticking to an old codebase for such a long time is that it gets older and older. You get new talent that doesn't want to manage it and leave, so you're stuck with the same old people that implemented the codebase in the first place. Sure they're smart, knowledgable people in the year 2000, but think of how fast technology changes. Change, adapt, or die.
A big bang rewrite will nine out of ten times slow you down, it will not accelerate things, and the most likely outcome is that not only will it be slower, it might fail entirely.
It's a complete fallacy to think that you're going to do much better than the previous crew if you are not prepared to absorb the lessons they left behind in that old crusty code.
It's not a given that legacy code means "no people still around, no docs and no tests".
I'm on a rewrite project and I'm 10 years in, and the whole crew from the last project (also around 15 years) is till in this project too. That helps.
The causes of the big bang rewrite are usually not just "this code smells let's rewrite it" but rather that the old product reached some technical dead end. Perhaps it can't scale. Perhaps it's a desktop product written in an UI framework that doesn't support high DPI screens and suddenly all the customers have high DPI screens. Obviously in that situation you'd aim to just replace a layer of the application (a persistence layer, an UI layer) but as we all know that's not how it works. The cost of a rewrite shouldn't be underestimated - as you said there is no reason to believe that if it took 50 man years for the last team then the new team will take 50 too. But that is in itself not a reason to not do it.
Fair enough. So the real lesson then is 'it depends', as with everything else. But the kind of jobs where the cleanup crews get called in are on the verge of hopeless and it is not rare that we do these on a 'no-cure, no pay' basis.
Great to see you be part of such a long lived team, that's a rarity these days. That's got to be a fantastic company to work for. Usually even relatively modest turnover (say 15% per year) is enough to effectively replace all the original players within a couple of years, most software projects long outlive their creators presence at the companies they were founded in. Add in some acquisitions or spin-outs and it gets to the point where nobody even knows who wrote the software to begin with.
Software is only as good as the people that write it. In an ideal world, you'll have a team that specializes in this sort of things, can understand the business needs, and get it done.
There are always risks with every action taken. You can't be scared to take a big risk for a bigger payout versus sucking it up and doing things the way they've been done for 15 years.
First, to learn the problem. Second, to learn the solution. Thrice, to do it right.
Skip a step at your own peril.
Incrementalism and do-over both have their place.
If you're resurrecting legacy code, I can't imagine successfully rewriting it until after you understand both the problem and solution. Alternately, change the business (processes), so that the legacy can be retired / mooted.
Anything that fails isn't moving anything anywhere.
And in the type of place that has a dysfunctional, legacy software system running core business operations, don't count on all the other ducks being in a row (anything resembling agile, ability to release to prod on a reasonable cadence, ability to provision sufficient test data, working test systems, etc).
If it's an internal system that you've been working on and maintaining... for 10 years... maybe (just maybe). If you're a consultant stepping in, I wouldn't touch that option for love or money.
This might be true (big bang rewrite) for small web sites or non business critical utility software, but if your cash flow depends on the software, you do not want failing software to stop that cash flow.
I think this mostly applies to newer startups who have changed teams with no or little hand off.
Fighting technical debt is hard. Fighting it with a blindfold is harder. Fighting it with 0 frame of reference is daunting. Fighting it the rest of the company is demanding new features right now is a recipe for stagnation, bugs, and burn out.
It is amazing how much of our profession's knowledge ends up as a odd if statement buried deep in the code to some method or stored procedure dealing with an edge case that gets missed in the big bang rewrite. Its also amazing how much money the failure to preserve that knowledge can cost.
I wonder if its time for professional software archeologists?
> I wonder if its time for professional software archeologists?
No, but it is time to make a real effort to teach the lessons learned to newcomers. I really feel that as an industry we completely fail at that. Blog posts such as these are my feeble attempt at trying to make a contribution to solving this problem.
Writing software is still "creative" and less "engineering". There aren't many ways to build a bridge but many ways to express yourself in language. Natural language or computer language that is.
Add this to "business requirements" and you get the big pile of manure we walk in every day. Like how does the knowledge of IEEE754 help me if the requirement is to sum up some value of the last three days, unless the last three days are on a weekend or holiday. (ok, stupid example) The point is domain language does not translate to computer language very well and a programmer is not a domain expert. He is .. just a programmer, a creative programmer, and we are millions each doing their thing a little different.
Strangely, at all the big corp jobs I've been at, the good programmers have become domain experts as part of the job. How else could they have a real feel for what the business needed and if the code was correct?
Oh yes, I totally agree with you on this. But it is a long road to become a domain expert. Don't forget people eventually leave jobs and you get new hires. Leaves lot of room for errors.
That's why you need product owners/managers to cover the business logic and design a system that incorporates it. There needs to be tiers of coordinations to make sure a system is built to spec. TDD plays a big part in rebuilding legacy codebases.
A huge issue with sticking to an old codebase for such a long time is that it gets older and older. You get new talent that doesn't want to manage it and leave, so you're stuck with the same old people that implemented the codebase in the first place. Sure they're smart, knowledgable people in the year 2000, but think of how fast technology changes. Change, adapt, or die.