> tl;dr Things you should never do: rewrite the code from scratch.
Considering vim itself is a from-scratch 'vi' clone, that's interesting advice. :)
But in any case, I see this as more of a fork than a rewrite. It's more akin to the XEmacs/GNU Emacs situation, where someone submits significant patches upstream which aren't accepted, and then forks their own version to continue that line of development.
They're pretty comparable. A "clone" can almost always be considered a subset of a "rewrite". Both involve a new software system that is directly inspired by an existing one.
The main difference is that a clone is often written by developers who had no involvement with the development of the original system. But that's not always true, especially for any old or sizable software system, where the original developers may no longer be involved.
> Things you should never do: rewrite the code from scratch.
I can clearly see the benefit of this old wisdom under context of limited resource and limited time, esp. in a commercial company.
However, as an experimental, personal(?), non-profit, free/open-source project, I do think neovim has value in its own right:
1. It won't affect any existing plan of vim, and won't change any existing code in vim. We can still happily use vim as usually.
2. If it succeeded, then we would have another powerful free/open source editor in our community.
3. If it failed, then we as a community have nothing to lose. The author might be unhappy then, but he would at least gain lots of experience.
There are so many rewritings happen in this community, and they are definitely good things:
- svn is a rewriting of cvs, and git is a totally new one (with more features?) written from scratch.
- nginx was written even if there's apache httpd
- There's zookeeper, but people wrote etcd
- etc.
I came to say this exactly.
In a commercial project where time and money are limiting factors, you are almost insane to rewrite from scratch.
In an OSS context, money is not always the biggest mover, and doing this might actually succeed, or at least not fail because the company fails.
It certainly may be that the people lose interest before it gets anywhere, but who cares!
The only reason that I could see is that someone wants to fold all their code changes back into vim's original codebase, and they both seem to have different end goals, so eventually they will drift away from each other.
Rewriting is a significant risk, and rewrites often fail because people don't appreciate that fact. But there are ways to rewrite successfully, and often doing so is the correct course of action.
A big problem with rewrites is that it puts a strain on development resources, and it's not possible to maintain both old and new code at the same time. Another big problem is prematurely committing to the rewrite, before it has proven itself. When the inevitable happens and schedules slip, defects arise, and features need to be cut this can prove disastrous for a rewrite that you've foolishly committed to in advance. Also, people often ignore the value of years of productive use which provides testing and validation of immense value.
If you have the resources often the best choice is to spin off a new project for a rewrite and have them compete in the open. With luck the rewrite could become mature enough to overtake the original (as has happened with many projects). Alternately, work in phases doing refactoring in steps, perhaps with new features behind feature flags.
Many people think that rewriting code is easier than green field development, in reality it's often more difficult. It takes a great deal of skill, and often luck, to manage a good rewrite.
A significant problem is that there are a lot of developers today who revere Spolsky as some sort of a god, and absolutely refuse to question or reconsider anything that he has written.
This trend is particularly bad with those who've gotten involved with software development since 2005 or so.
I chalk it up to a lack of experience with massive, multi-decade software projects. If all a developer has worked with are small, recently-written Ruby on Rails apps, for example, then I don't think they can appreciate situations where rewrites of some degree are the only viable option.
Hopefully this attitude regarding Spolsky's writing will change with time, as these people encounter the real-world situations where significant or total rewrites are basically the only possible option.
An even bigger problem, in my opinion, is the idea (usually only implicit) that any advice can apply to the entire panoply of software development. As though writing Flappy Bird and writing control software for the Falcon 9 weren't as different as banging together some shelving using plywood from home depot and machining custom components from exotic alloys using a lathe. An important part of maturing as a developer is learning to understand that different situations call for different techniques and being able to know what things are truly important with a given project and which aren't.
A related problem, or maybe the same problem in a different outfit, is the notion that my personal experience is enough for me to opine on all software everywhere.
For me there's a useful sort of arrogance, the "I could code that in two weeks even if I don't know the language" kind of arrogance. I say useful, because that had gotten me to jump into projects that I never would have dared touch.
But the dangerous side of that arrogance is, "I know enough to say what what these other people are doing is stupid". Every time I learn a new sort of software or a new domain, I am reminded that it's much easier to make bold pronouncements from ignorance than from knowledge.
With that thinking we never get Linux (NeoMinix), tmux (NeoScreen), Clang (NeoGCC), WebKit (NeoGecko), Subversion (NeoCVS), or Vim itself (NeoVi/Elvis/Stevie/etc).
Neovim is being sold as a VIM refactor, but apparently with no new functionality
Very different from Linux (had an aim of being a playground for new features), tmux is different from screen as Clang is different from GCC, Subversion is a totally different beast from CVS
"Enable the implementation of new/modern user interfaces without any modifications to the core source.
"Improve the extensibility power with a new plugin architecture based on coprocesses. Plugins will be written in any programming language without any explicit support from the editor. "
It is exactly designed to be a playground for new features, but with the goal of them not having to be in the core.
To a degree, I think these distinctions both support and are a logical consequence of haberman's argument (please correct me if I'm wrong). As computing systems evolves, we learn more about how to address their constraints and how these constraints impact how users choose/desire to interface with their programs. Vim is publicly ~22 years old. During it's maturation, many computing milestones have been met which overcome many of the original assumptions or challenges which prevented us from using some of the interfaces we prefer today. Async (re: MarcWebber) being one such example.
I believe tmux / screen is a beautiful example, given Screen has a lot of black magic and has a very challenging codebase to work with. A lot of the philosophy built into tmux are cumulative learnings deduced from what was done correctly or incorrectly within screen. This materializes as features missing (or implemented with different intent) from tmux that exist in screen -- the philosophy behind sessions is one such big underlying principle change.
I like to think of this as similar to using a genetic algorithm to evolve an overly complex/convoluted solution over a period of time (based on some assumptions which may no longer even be applicable), and then explicitly defining/reconstructing a comparable solution using more deterministic and intentional approaches. The focus of these two development cycles are fundamentally different and perhaps neither wrong (the former attempts to discover novel ways of addressing a problem whose solution is unknown, the latter prunes this known search space).
Neovim isn't a simple refactor though. It adds the ability to do many things which aren't currently possible or are extremely difficult to do with the current vim. Sometimes there is no easy way out and you have to put in the work to fix previous limitations.
It's basically trying to improve over vim, still using command/insertion modes, inverting the commands (by using movement/action instead of action/movement)… It's pretty configurable and the codebase is new.
None of those are from-scratch rewrites of existing software. They're mostly (not all) clones of existing software, with some (e.g. Linux) being more "inspired by", and some (e.g. Clang) being "We can do this better".
When we started work on Commandos 2, we threw away everything from Commandos 1, and started from scratch. 15 years down the road, all the coders still agree that was the best decision ever.
Man, I loved Commandos 1 back in the day. I recently got a bit nostalgic and bought the whole series on Steam, and they are still great games! But MAN are the controls opaque. Thinking back, I think the game came with a little cardboard sheet with all the controls on it, which would have been easier to deal with.
tl;dr Things you should never do: rewrite the code from scratch.
Sentiment like this is not valuable. The experience that it comes from is often informative, but as a conclusion, this explicitly ignores circumstances that sometimes do make rewriting a good, compelling course of action.
Truly bad, unmaintainable code exists. It exists on projects that have long intended futures. It exists in all sorts of environments on all sorts of projects for all sorts of reasons. To discount the idea of a rewrite out of hand is a failure to objectively review the specific circumstances around whatever it is you're working on.
Often there are better ways, and yes, often a rewrite is a bad idea. But absolutes are so rarely valid, and this one is most certainly not universally true.
Joel has lots of right things to say, but I'm afraid that one - "Never rewrite from scratch" - is far too sternly interpreted to the point where it is actually more harmful than beneficial.
Of course you can rewrite from scratch. You just should not expect to reach the same feature coverage instantly, and should approach the work methodically. Most of all, you should have a good reason. "This code is spaghetti" is usually not one. "I want to implement feature X and the current codebase does not really support is" could be a good reason.
Also, a rewrite is pointless from the point of view of code quality unless you are familiar with similar codebases operating in a similar domain.
Main advantages or rewrite are, that when you are a domain expert and implementing and designing core routines, you get constant flashes of intuition "Aha, I should add this interface here because that will enable me to implement Foo and Bar using this instead of writing a FooSubsystem and BarSubsystem" and so on.
If you have no experience from the domain, you get those insights only after you've written the code and are actually new proud owner of new heap of semi-maintainable code.
This isn't a "rewrite the code from scratch", but instead is an incremental but aggressive refactoring. There's at least one wildly successful example of this successfully being done for GCC, which is a project even bigger and more complex than Vim. Around GCC 2.8, it was forked into EGCS where major changes were made. This fork was so successful that it then became the official branch of GCC, and modern versions of GCC are direct descendants of EGCS.
Who knows, Neovim may very well drop support for vimscript and the entire ecosystem would have to start over. In the long run that may be a great thing.
I tend to agree with Joel's thesis here, but the example is bad: Netscape was doomed either way, the rewrite was an act of desperation, and I'd say it succeeded because it led to Firefox which slowly clawed back those first few points of marketshare from Microsoft—enough to finally get them to resume browser development. Even assuming they could have pulled a rabbit out of their hat with the Netscape 4 codebase (which was unlikely considering the history of that codebase, and it's legacy compared to the talent and experience that was being pumped into the greenfield Internet Explorer (yes, ironic I know)), IE was still going to eat their lunch. Netscape was between a rock and a hard place. But hey, at least their code lives on today right?
Teams I've been a part of are 3/4 (3 successes, 1 failure) on total rewrites over my career. The failure was entirely my fault as a developer not knowing my own limits (coding, design, security, devops, business reqs, managing up can't happen (at least for me) in the same 60 hours a week). It still almost succeeded on strength of the design and ability of my amazing coworkers to aggressively simplify and consolidate all the things.
Rewrites fall into two categories: essential to business and doomed to fail. If it's essential that the business execute on a rewrite, it will happen or the business will fail. If it's not essential, it very well might be the reason the business fails (siphoning away talent and squandering time when a refactor or benign maintenance would suffice). Precisely because massive rewrites are so risky, they are undertaken by the foolish (myself firmly in this camp). The heuristic under which they might succeed is bare necessity.
What makes you think this is a rewrite? If this is a rewrite from scratch, they are insanely fast coders, considering that they have a full-blown editor in less than a month (first commit is from January, 31)
That first commit also shows it is not a rewrite, though, since it has a comment starting with "Import vim from changeset v5628:c9cad40b4181"
> but I'm sure that's the way Netscape thought of it as well
Don't really think so. At least as far Netscape 6 went, they came up with an entirely different framework and moved over to Java (as far as I remember). I have rarely seen that sort of thing characterized as a refactor (even on their mailing lists).
The Seamonkey Project [1] is the current state of the Mozilla Application Suite[2], which Netscape 6 and 7 were branched from[3][4]. Written in C++ mostly.
Some of us have been using Gecko as their renderer for almost sixteen years now. Seems KHTML(w), the base of both Webkit and Blink[5], is the same age. Seems we've gotten a good run out of the existing codebases. In fact, this means that Gecko (and khtml) have been in use twice as long as NCSA Mosaic, which the first Netscape was based on[6]. Imagine how Spolsky must feel about the Servo rewrite.
I think that this is an unfair comparison. Netscape and Borland were business with financial targets, deadlines and (more importantly,) competitors vying for the same market share.
These guys don't have these pressures, nor did (as per haberman's comment) Linux, tmux, Clang, WebKit, etc as far as I'm aware.
I'm somewhat confused by your comment, since the GitHub page explicitly says that NeoVim "is not a project to rewrite Vim from scratch". Where's the discrepancy here? Perhaps I'm just too tired right now.
Fun food for thought: NCSA Mosaic started in 1992, and ended in 1997. Netscape 4.0, the last user of this engine, stopped development in 1998. Gecko, the core of Mozilla, started in 1998, and is still in development 16 years later.
So, the engine that Spolsky was saying was a mistake has now been in use for twice as long as the engine it was replacing.
A general statement such as "never rewrite" is as good as something like "never grow a beard" - while it may be a good advice for someone working in a traditional corporate environment, it certainly doesn't make sense, let's say, for a freelancer.
There is probably no Spolskyism that I disagree with more. And yet, I don't really disagree with what he said in context. The context is within a business where the opportunity cost could make it not worth it. If the only concern is making the software better, my experience is that rewriting does wonders for old code.
What you're missing is that Netscape rewrote their own product from scratch. Neovim is a fork bringing very invasive changes (not rewriting from scratch) to a project by a different developer for the purpose of extending the original project in ways the current maintainer doesn't want to.
So it's more like taking the old netscape code and making gecko.
http://www.joelonsoftware.com/articles/fog0000000069.html
I realize Neovim is being characterized as an "aggressive refactor," but I'm sure that's the way Netscape thought of it as well.
tl;dr Things you should never do: rewrite the code from scratch.