In the full interview, Joel addresses the concern over "bloatware", where software takes up progressively more disk and/or memory space. Among programmers, "bloatware" is a derogatory term.
Moore's law makes much of the whining about bloatware ridiculous. In 1993, Microsoft Excel 5.0 took up about $36 worth of hard drive space. In 2000, Microsoft Excel 2000 takes up about $1.03 in hard drive space. All adjusted for inflation. So stop whining about how bloated it is.
Actually, the whole article is pretty interesting. I'm putting a copy here for when the link goes bad. Read on…
An Interview with Joel Spolsky of JoelonSoftware
Part I of II
Overview
We recently sat down with someone we regard as one of the industry's most fascinating personalities, Joel Spolsky, president and one of the founders of Fog Creek Software (www.fogcreek.com), located in New York City. Joel worked at Microsoft from 1991 to 1994 and has over ten years of experience managing the software development process. As a Program Manager on the Microsoft Excel team, Joel designed Excel Basic and drove Microsoft's Visual Basic for Applications strategy. (Joel takes particular pride in the fact that on the day Bill Gates asked if date math functions were compatible across the company's different procedure and function libraries, he, Joel Spolsky, was able to reassure the great man himself that with the exception of January and February 1900, all Microsoft application libraries counted dates the same way.)
As a Technical Manager at Juno Online Services, Joel designed and implemented many of the Internet features of Juno. His web site Joel on Software (www.JoelonSoftware.com) is read by thousands of developers worldwide every day and we strongly recommend you bookmark it as soon as you're done reading this article (oh, OK, do it now). His first book based on that site, User Interface Design for Programmers, was reviewed on this web site and we regard it as a must have and read by anyone involved in developing and marketing software.
Fog Creek Software has just released its newest and greatest product, CityDesk, a content management system. CityDesk is priced at $79 for the home version and $349 for the professional version. The product is slated to be reviewed on SoftwareMarketSolution.com shortly. (On a happy note, Joel has promised to commit ritual suicide via web cam if we discover any category one bugs in the product. With this as an incentive, we promise you a truly in-depth look at CityDesk.)
SMS: Joel, what, in your opinion, is the single greatest development sin a software company can commit?
Joel: Deciding to completely rewrite your product from scratch, on the theory that all your code is messy and bug prone and is bloated and needs to be completely rethought and rebuild from ground zero.
SMS: Uh, what's wrong with that?
Joel: Because it's almost never true. It's not like code rusts if it's not used. The idea that new code is better than old is patently absurd. Old code has been used. It has been tested. Lots of bugs have been found, and they've been fixed. There's nothing wrong with it.
SMS: Well, why do programmers constantly go charging into management's offices claiming the existing code base is junk and has to be replaced?
Joel: My theory is that this happens because it's harder to read code than to write it. A programmer will whine about a function that he thinks is messy. It's supposed to be a simple function to display a window or something, but for some reason it takes up two pages and has all these ugly little hairs and stuff on it and nobody knows why. OK. I'll tell you why. Those are bug fixes. One of them fixes that bug that Jill had when she tried to install the thing on a computer that didn't have Internet Explorer. Another one fixes a bug that occurs in low memory conditions. Another one fixes some bug that occurred when the file is on a floppy disk and the user yanks out the diskette in the middle. That LoadLibrary call is sure ugly but it makes the code work on old versions of Windows 95. When you throw that function away and start from scratch, you are throwing away all that knowledge. All those collected bug fixes. Years of programming work.
SMS: Well, let's assume some of your top programmers walked in the door and said "we absolutely have to rewrite this thing from scratch, top to bottom." What's the right response?
Joel: What I learned from Charles Ferguson's great book (High St@kes, No Prisoners) is that you have to hire programmers who can understand the business goals. People who can answer questions like: "What does it really cost the company if we rewrite?" "How many months will it delay shipping the product?" "Will we sell enough marginal copies to justify the lost time and market share?" If your programmer insists on a rewrite, they probably don't understand the financials of the company, or the competitive situation. Explain this to them. Then get an honest estimate for the rewrite effort and insist on a financial spreadsheet showing a detailed cost/benefit analysis for the rewrite.
SMS: Yeah, great, but, believe it or not, programmers have been known to, uh, "shave the truth" when it comes to such matters.
Joel: What you're seeing is the famous programmer tactic: all features that I want take 1 hour, all features that I don't want take 99 years. If you suspect you are being lied to, just drill down. Get a schedule with granularity measured in hours, not months. Insist that each task have an estimate that is two days or less. If it's longer than that, you need to break it down into sub-tasks or the schedule can't be realistic.
SMS: Are there any circumstances where a complete code rewrite is justified?
Joel: Probably not. The most extreme circumstance I can think of would be if you are simultaneously moving to a new platform and changing the architecture of the code dramatically. Even in this case you are probably better looking at the old code as you develop the new code.
SMS: Hmmm. Let's take a look at your theory and compare it to some real world software meltdowns. For instance, what happened at Netscape?
Joel: Way back in April 2000, I wrote on my web site that Netscape made the "single worst strategic mistake that any software company can make" by deciding to rewrite their code from scratch. Lou Montulli, one of the 5 programming superstars who did the original version of Navigator, E-mailed me to say, "I agree completely, it's one of the major reasons I resigned from Netscape." This one decision cost Netscape four years and they're not really done yet. That's three years they spent with their prize aircraft carrier in 200,000 pieces in dry-dock. They couldn't add new features, couldn't respond to the competitive threads from IE, and had to sit on their hands while Microsoft completely ate their lunch.
SMS: OK, how about Borland? Another famous meltdown. Any ideas?
Joel: Borland also got into the habit of throwing away perfectly good code and starting from scratch. Even after the purchase of Ashton-Tate, Borland bought Arago and tried to make that into dBase for Windows, a doomed project that took so long that Microsoft Access ate their lunch. With Paradox, they jumped into a huge rewrite effort with C++ and took forever to release the Windows version of the product. And it was buggy and slow where Paradox for DOS was solid and fast. Then they did it all over again with Quattro Pro, rewriting it from scratch and astonishing the world with how little new functionality it had.
SMS: Yeah, and their pricing strategy didn't help.
Joel: While I was on the Excel team, Borland cut the MSRP on QPro from around $500 to around $100. Clueless newbie that I was, I thought this was the beginning of a bloody price war. Lewis Levin, Excel BUM ("Business Unit Manager") was ecstatic. "Don't you see, Joel, once they have to cut prices, they've lost." He had no plan to respond to the lower price. And he didn't need to.
SMS: Having worked at Ashton-Tate, we have to tell you the dBase IV code base was no thing of beauty. But, we take your point. Actually, we saw this syndrome at work in Ashton-Tate's word processing division. After they bought MultiMate, they spent about two years planning a complete rewrite of the product, and wasted months evaluating new "engines" for the next version. Nothing ever happened. When a new version of the product WAS released, it was based on the same "clunky" engine everyone had been moaning about. Of course, in those two years WordPerfect and Microsoft ate Ashton-Tate's word processing lunch.
Joel: Ashton-Tate had a word processor?
SMS: Yes, but nothing good as WordStar, mind!
Joel: Hmm. That reminds me that Microsoft learned the "no rewrite" lesson the hard way. They tried to rewrite Word for Windows from scratch in a doomed project called Pyramid which was shut down, thrown away, and swept under the rug. Fortunately for Microsoft, they did this with parallel teams, and had never stopped working on the old code base, so they had something to ship, making it merely a financial disaster, not a strategic one.
SMS: OK, Lotus?
Joel: Too many MBAs at all levels and not enough people with a technical understanding of what could and needed to be built.
SMS: And I suppose building a brand new product called "Jazz" instead of getting 123 over to the Mac as quickly as possible, thus staking Microsoft to a two year lead with Excel, is an example of the same thing?
Joel: Actually they made a worse mistake: they spent something like 18 months trying to squeeze 123/3.0 into 640KB. By the time the 18 months were up, they hadn't succeeded, and in the meantime, everybody bought 386s with 4 megs. Microsoft always figured that it's better to let the hardware catch up with the software rather than spending time writing code for old computers owned by people who aren't buying much software any more.
SMS: WordPerfect?
Joel: That's an interesting case, and leads to another development sin software companies often make: using the wrong level tools for the job. At WordPerfect, everything, including everything, had to be written in Assembler. Company Policy. If a programmer needed a little one-off utility, it had to be hand-coded and hand-optimized in Assembler. They were the only people on earth writing all-Assembler apps for Windows. Insane. It's like making your ballerinas wearing chains and balls and taping their arms to their side.
SMS: What should they have been coding in?
Joel: In those days? C. Or maybe Pascal. Programmers should only use lower level tools for those parts of the product where they are adding the most value. For example, if you're writing a game where the 3D effects are your major selling point, you can't use an off the shelf 3D engine, you have to roll your own. But if the major selling point of your game is the story, don't waste time getting great 3D graphics — just use a library. But WordPerfect was writing UI code that operates in "user time," and doesn't need to be particularly fast. Hand coded assembler is insane and adds no value.
SMS: Yes, but isn't such code tight and small? Don't products built this way avoid the dreaded "bloatware" label?
Joel: Don't get me started! If you're a software company, there are lots of great business reasons to love bloatware. For one, if programmers don't have to worry about how large their code is, they can ship it sooner. And that means you get more features, and features make users' lives better (if they use them) and don't usually hurt (if they don't). As a user, if your software vendor stops, before shipping, and spends two months squeezing the code down to make it 50% smaller, the net benefit to you is going to be imperceptible, but you went for two months without new features that you needed, and THAT hurt.
SMS: Could this possibly account for the fact that no one uses WordStar version 3.3 anymore despite the fact it can fit on one 1.4 meg floppy?
Joel: That and "Control-K." But seriously, Moore's law makes much of the whining about bloatware ridiculous. In 1993, Microsoft Excel 5.0 took up about $36 worth of hard drive space. In 2000, Microsoft Excel 2000 takes up about $1.03 in hard drive space. All adjusted for inflation. So stop whining about how bloated it is.
SMS: Well, we've had much personal experience with the press slamming a product we were managing. Those non-existent tables in WordStar keep coming back to mind. Somehow the fact that the product would fit on a 360K floppy just didn't seem to mean as much as the idea that the reviewer couldn't use our product to write his resume.
Joel: There's a famous fallacy that people learn in business school called the 80/20 rule. It's false, but it seduces a lot of dumb software startups. It seems to make sense. 80% of the people use 20% of the features. So you convince yourself that you only need to implement 20% of the features, and you can still sell 80% as many copies. The trouble here, of course, is that it's never the same 20%. Everybody uses a different set of features. When you start marketing your "lite" product, and you tell people, "hey, it's lite, only 1MB," they tend to be very happy, then they ask you if it has word counts, or spell checking, or little rubber feet, or whatever obscure thing they can't live without, and it doesn't, so they don't buy your product.
SMS: Let's talk about product marketing and development at Microsoft. How did these two groups work together at Microsoft?
Joel: Well, in theory, the marketing group (called Product Management) was supposed to give the development team feedback on what customers wanted. Features requests from the field. That kind of stuff. In reality, they never did.
SMS: Really?
Joel: Really. Yes, we listened to customers, but not through Product Management — they were never very good at channeling this information. So the Program Management (design) teams just went out and talked to customers ourselves. One thing I noticed pretty quickly is that you don't actually learn all that much from asking customers what features they want. Sure, they'll tell you, but it's all stuff you knew anyway.
SMS: You paint a picture of the programmer almost as a semi-deity. But in our experience, we've seen powerful technical personalities take down major companies. For instance, in The Product Marketing Handbook for Software, we describe how the development staff at MicroPro refused to add a tables capability to WordStar despite the fact that the lack of this capability was hurting sales. In our last article on SMS, in which we discuss Novell's decline and fall, we point out that one of the company's technical gods, Drew Major, refused to implement a GUI on top of Netware while you guys at Microsoft were pounding them to death with NT and a comprehensible and demonstrable interface. How do you manage situations like these?
Joel: This is a hard problem. I've seen plenty of companies with prima donna programmers who literally drive their companies into the ground. If the management of the company is technical (think Bill Gates), management isn't afraid to argue with them and win — or fire the programmer and get someone new in. If the management of the company is not technical enough (think John Sculley), they act like scared rabbits, strangely believing that this ONE person is the only person on the planet who can write code, and it's not a long way from there to the failure of the company. If you're a non-technical CEO with programmers who aren't getting with the program, you have to bite the bullet and fire them. This is your only hope. And it means you're going to have to find new technical talent, so your chances aren't great. That's why I don't think technology companies that don't have engineers at the very top have much of a chance.