Software development
A test framework in Drupal 7
Every major Drupal release we should ask ourselves what steps we can take to double the capacity of our community.
I spent the weekend in Paris where we had a two day code sprint. Our main accomplishment was getting Drupal's test framework into Drupal core -- the culmination of three years of hard work carried out by many people and companies in the Drupal community.
Here is why a strong investment in testing will help double the capacity of our community:
- For developers, upgrading, maintaining and releasing your modules becomes easier. The combination of test results and code coverage reports makes it easier to determine the release readiness of your code. This translates to fewer betas, shorter code freeze periods and more frequent releases. Furthermore, design for testability leads to easier to use and more complete APIs. It is guaranteed to make Drupal a better development platform.
- For end users, it is important that we provide quantifiable reporting on the health of Drupal core and the many contributed modules.
- For patch reviewers, tests are great because it allows them to focus on the architectural and the algorithmic changes that the patch introduces. With good test coverage, we can rely on the tests to discover any unwanted side-effects. Patches can be committed faster.
- For people new to Drupal, tests lower the barrier to entry and encourage collaboration and innovation, two of Drupal's core values. With good test coverage, you don't necessarily have to understand the entire code base before you can comfortably propose changes or help maintain a module.
- For me, it means I'll sleep better at night. ;-) With hundreds of thousands of people using Drupal, the availability of a test framework takes some pressure of my shoulders.
As of today, it is expected that you submit test cases with your patches for Drupal 7. Writing good tests takes time: it is not unlikely that you'll spent twice as long working on a patch. This might take some time getting used to but you'll find that it pays off and that it is actually very rewarding.
Thanks to Ori Pekelman of AF83 for being such a great host when in Paris. And a special thanks to everyone who helped get the test framework in Drupal 7.
Drupal and testing
At DrupalCon Barcelona about 20 people gathered together for the Quality Assurance Birds of a Feather session. This led to the development of a continuous testing infrastructure for Drupal. People that want to write tests for Drupal can, but don't have to.
Second, in my State of Drupal presentation at DrupalCon Barcelona, I shared a list of the top-10 most requested Drupal improvements. On spot 9 we had "better internal APIs" and on spot 10 we had "better external APIs (import/export, web services)". APIs that are designed for testability, are often better APIs. This comes from the fact that testability leads to more cohesive code with loose coupling. Important for readable, agile code.
Third, Drupal 6 has been in a code freeze for 6 month. Often times, the behavior introduced by one patch inadvertently breaks the behavior of existing functionality. In absence of tests, it takes a while to shake out all the bugs and before you're actually confident to roll a new major Drupal release.
As a result, people in the community have suggested to put more emphasis on testing in Drupal. Writing tests is expensive so it is hard to model the cost and benefits. However, the only way to past the costs and to experience the true benefits, is to take testing to the extreme. There is no middle ground. If we want to be serious about testing, we need to enforce, not just encourage, testing in the Drupal 7 development cycle and beyond. (And maybe we should consider introducing tests for Drupal 5 and Drupal 6 maintenance releases.)
In such a world, patches that fix bugs should come with a test case that demonstrates the behavior being fixed. That patch would then be added to the regression test suite and run continuously to avoid that the bug gets re-introduced. In addition, patches that add new features or enhancements should come with test cases for this new functionality. Everything should be tested.
Of course, this would require a significant commitment from the whole Drupal developer community. There is bound to be some controversy about the need to learn how to use Drupal's test framework and the overhead and complications of having to write and submit a test. But over time, fewer people will be frustrated by bugs, the burden on patch reviewers will go down, code freezes will become shorter, APIs will continuously improve, and we would have more confidence in the stability of our releases. Last but not least, it actually improves overal productivity and encourages change and experimentation -- something the Drupal community really takes pride in.
As we learned before: sweeping changes are required to make major advances in technology, and often times there is a lot of pain before the pay-off. Are we willing to take some pain? Are we willing to enforce tests to be written as part of the Drupal 7 development process? I'm willing to fully embrace testing, but only if you are willing to write tests ...
Linus Torvalds on Git
While distributed source code management tools like Git, Mercurial, Monotone and Bazaar (which some Drupal developers use) offer advantages over CVS or SVN, they are not as accessible or well-supported. Drupal developers and Drupal designers are not Linux kernel hackers, and for many of them CVS is the biggest barrier to entry. Without desktop integration for both Windows and the Mac (like TurtoiseCVS or MacCVSClient offer), I don't see how we can migrate away from CVS without leaving half of our community behind ...
Not to mention the fact that our entire release management system is built on top of CVS. It would be a lot of work to switch that to another system.
Creating passionate users
Like many, I'm a long-time reader of Creating Passionate Users, a blog co-authored by Kathy Sierra. Last month at Euro OSCON I had the opportunity to attend a 3 hour tutorial by Kathy Sierra, and now I can't wait for the "Creating Passionate Users" book to come out.
I'm a fan.
I'm a fan because over the past year, Kathy has permanently changed my perspective on user experience (and because she managed to put in words what I've known intuitively for a long time). To give you an idea, I've included the blog posts (and graphs) that had the most impact on me.
Excessive dogfooding and zealotism
In software development one is eating his own dog food when he is using its own product (i.e. the software he develops). I'm using Drupal for my personal website, so I'm eating my own dogfood. Drupal.org is using Drupal, so the Drupal community is eating its own dogfood. Many developers in the Drupal community, or the Open Source community in general, eat their own dog food.
There are a number of well-known advantages to eating our own dogfood: it provides evidence that Drupal works and that we are confident using it ourselves. Furthermore, as users of our own software, we help discover bugs and identify shortcomings. When developers are users there is plenty of incentive to fix bugs and the feedback loop doesn't become much shorter than that. For those reasons, eating your own dogfood is a common strategy in software development.