WordPress Release Testing

WordPress is a great blogging software. But its frequent releases that break backward compatibility is upsetting some users. Is the WordPress team to blame for poor release testing? Someone has come out to defend the WordPerss team, but I kind of disagree with the arguments. Some of the blame has to fall on the WordPress team.

Just to take care of some of the question raised in the defense of the WordPress team:

• In the light of addressing security vulnerabilities, is it better to have frequent releases or less frequent (such as twice-yearly) releases? Of course, frequent releases to address security vulnerabilities is good. I don’t see any harm in frequent releases to introduce new features too. Just don’t break backward compatibility.
• Is it the WordPress team’s responsibility to test every plugin known to man? No, but if you know you are breaking backward compatibility, at least do some reasonable testing with well-established favourites.

Software development is complicated. It gets worse when it is a large community project, when it is rapidly evolving, and where there is a lot of interdependencies.

Retaining backward compatibility is difficult. It is one of the reasons why some APIs get so convoluted. Occasionally, backward compatibility is sacrificed when it is the better choice in moving forward. But sometimes, developers even go through great lengths to retain backward compatibility, so that they don’t break even the unsupported or undocumented uses of their product.

I think the issue of “supported” or “unsupported” uses is important to the matter at hand. If a certain prescribed use of a product is “supported”, it makes sense for the developers to put in some effort to make sure it works through normal use of the product. Here, I’m assuming normal use of the product also includes upgrading through versions, etc.

A lot of complaints about the WordPress’ lack of backward compatibility stems from plugins that become broken after upgrading to a new version of WordPress. Are plugins a “supported” feature of WordPress? Yes, I believe so. Plugins, of course, may be written by 3rd party developers unrelated to the WordPress team. It will be difficult for the WordPress team to ensure all plugins work properly, will work properly after upgrades to WordPress, etc.

But things are not so simple. The WordPress team does not work alone. They have designed their software to work with others.

Let me use an analogy. The Linux kernel was designed so that it can run an operating environment on top of it, and with the operating environment, many other applications, services, etc. The kernel developers don’t write the operating environment on top of it, nor are they responsible for the 3rd party applications. But if they break the published calling interface to userland, surely they cannot absolve themselves from any responsibility. Oh well, of course, contractually perhaps they are not responsible, that’s fine. But surely, they cannot morally announce “none of my problem, it’s your fault for not catching up with me”.

To the credit of WordPress team, they do tell everyone to backup. Not just the WordPress site directory itself, but also the database. It’s in the upgrade instructions. If anything breaks, you can just roll back to whatever you had before you commenced the upgrade. So what’s the big deal? (Okay, it is still annoying that the upgrade broke, but at least you didn’t loose anything other than a bit of time.)