You are here

Drupal

Why the big architectural changes in Drupal 8

There has been a lot of chatter about Drupal 8. Will Drupal 8 be performant? Will Drupal 8 be easy to develop modules for? Will I have to learn Symfony? I want to address these concerns and explain why Drupal 8 introduces such big changes.

Lessons from the past: the pain before the gain

The reason Drupal has been successful is because we always made big, forward-looking changes. It’s a cliché, but change has always been the only constant in Drupal. The result is that Drupal has stayed relevant, unlike nearly every other Open Source CMS over the years. The biggest risk for our project is that we don't embrace change.

The downside is that with every major release of Drupal, we've gone through a lot of pain adjusting to this change. I first wrote about it in 2006 when trying to get Drupal 4.7 released:

"So let's capture that thought for future reference. Sweeping changes are required to make major advances in technology, and often times there is a lot of pain before the pay-off."

We decided that big changes were required

Drupal 7 is a fantastic CMS, but at the same time there are some fairly big limitations, including an incomplete Entity API, a lack of separation between content and configuration, leading to deployment challenges, a lack of separation between logic and presentation in the theme layer, and more. We created solutions for those challenges using contributed modules, but those solutions were in many cases incomplete. In Drupal 8, we decided to tackle a lot of these problems head-on, through the Configuration Management Initiative, the Twig templating layer, and a complete Entity API.

The web landscape has also dramatically changed around Drupal since January 2011, when Drupal 7 was released. Mobile browsing is ubiquitous, and so are third-party services that people may want to integrate their Drupal sites with. Web site users expect a much higher bar when it comes to ease of use. We anticipated these trends and as a result, we spent the past 2.5 years working on Drupal 8's mobile features and user experience improvements.

But are all these great improvements enough?

We need to modernize Drupal 8

One of our biggest challenges with Drupal, is that it is hard for organizations of all sizes to find Drupal talent (developers, themers, site builders, etc). Drupal 7 didn't address this problem (e.g. we held on to procedural programming instead of object-oriented programming), and in fact made it a bit worse with the introduction of even more "Drupalisms" (e.g. excessive use of structured arrays). For most people new to Drupal, Drupal 7 is really complex.

The most effective way to address the Drupal talent issue, as well as the complexity issue, is to bring Drupal in line with modern frameworks and platforms, so there is less Drupal-specific knowledge to learn in order to become proficient. We decided to adopt modern PHP concepts and standards, object-oriented programming, and the Symfony framework. While a lot of the Drupal concepts (Fields, Views, Entities, Nodes) continue to exist in Drupal 8, they are now implemented using object-oriented programming design patterns.

The advantages and disadvantages of object-oriented programming are well-understood. The disadvantages are size, verbosity, the amount of work it takes to write (including the design planning that goes into it) and slower performance. For people new to object-oriented programming there may be a steep learning curve; some of the key programming techniques, such as inheritance and polymorphism, can be challenging initially. The advantages are encapsulation (both to hide implementation details and to avoid tampering with internal values), faster development thanks to re-use, extensibility, and better maintainability. Compared to procedural programs, object-oriented programs are easier to maintain, extend and refactor. So although a lot of work is spent to write the program, less work is needed to maintain it over time.

For Drupal 8 this means that the code will be more abstract, more verbose, and slower, yet also be more maintainable, more modular, and more accessible to non-Drupal developers. The end result is that Drupal 8 should help us attract new people to Drupal in a way Drupal 7 didn't.

This is exactly what happened with other projects like Symfony; Symfony 2 was a complete rearchitecture of Symfony 1. A lot of people were alienated by that, yet at the same time Symfony 2 took off like a rocketship. The same thing has happened with the major releases of Drupal as well, despite how much change each one brings.

Change is scary for the existing Drupal developers

However, as we get closer to the release of Drupal 8, existing Drupal developers have become increasingly aware of the massive changes that Drupal 8 will bring. This has resulted in some fear; some people feel like they have to re-learn everything they know, and that developing for Drupal 8 has changed to the point where it's no longer fun. This fear is completely understandable. Change is hard, it can be scary, and often takes a long time to be absorbed.

But even if we completely streamlined the Drupal 8 developer experience, Drupal 8 will still look and work radically different under the hood. As mentioned, there are advantages and disadvantages to object-oriented programming and modern design patterns. But if we care about the long-term success of Drupal, we can't preserve the past. The risk of sticking with the old Drupal 7 architecture is that we won't be able to attract many more Drupal developers, and that over time, Drupal will become the odd one out.

There is a lot of work left to be done

Part of the fear out there is well-founded because in the current state of development, Drupal 8 isn't good enough. While there are many things to like about Drupal 8, I'm also the first to admit that we have work to do on the Drupal 8 developer experience (as well as performance) before Drupal 8 can ship. Creating a well-designed object-oriented application takes more time and design work than creating a procedural one. Some major improvements have already landed, but we're still working hard on improving the developer experience. We need more help, especially from Drupal 7 developers, on how we can make Drupal 8 more approachable for them. I'm not afraid to make major API changes if the developer experience improvement is suitably large. So if you have concerns about Drupal 8, now is the time to step up and help. In return, I promise we'll work on Drupal 8 until it is ready. Thank you!

Open Source code: by the people, for the people

Many organizations adopt Open Source for reasons like flexibility and agility. Everyone needs to do more with less. But in government, Open Source drives both civic engagement and government participation like never before. Because of digital, the world feels much smaller and more connected. And Open Source gives people the opportunity to rally around a cause, no matter where they live.

Think about how we petitioned our government before we had the We The People website. I bet you have to think pretty hard about how it was done (I do!). Now, a website has brought to life the First Amendment rights of U.S. citizens. Millions of people's voices are heard. People pull together based on common concerns. The White House built We The People using Drupal and shared the code on GitHub, opening up the opportunity for other governments to easily create their own online petitioning systems.

Now, all kinds of open government data made available through the Data.gov project makes it possible for any developer, anywhere, to create a civic app. These apps have made us see our cities and towns in a different light.

Open City is one example of a group of local volunteers who create Open Source apps using government data. While the group is based in Chicago, the idea is that any city can grab code from an Open City app and make it their own.

Here are a few interesting examples: Clear Streets tracks a city's plows in real time. Living outside of Boston, I know we could use an app like that! Crime in Chicago lets citizens compare crime statistics in certain areas of town, which could be useful for people making decisions about where to move their families.

What is perhaps the most gratifying is that as open-source developers, we can collaborate on projects and help people around the world. It's part of what gets us out of bed in the morning. Earlier this year, participants in DrupalCon Portland launched a website in 24 hours to help people in Moore, Oklahoma, find transportation and housing after the tornado. Two weeks later, the site was discovered on Twitter in Germany and was repurposed to help people affected by the flooding in northern Europe. This type of project inspires us all to see how technology can make an immediate difference.

Other events, such as the National Day for Civic Hacking, encourage developers to use open government data to "collaboratively create, build, and invent". The idea that hackathons can help build and create a healthy, citizen-powered technology ecosystem within government is relatively new, and full of promise. Tim O'Reilly believes that government can escape the "vending machine" mentality (citizens put in tax dollars and get out services) by thinking of "Government as a Platform" for participation. I couldn't agree more.

Open Source ideals are already spreading in governments throughout the world, with good reason. A global network of developers is motivated to help. It's one of the best examples of civic engagement. As digital citizens, we all now have the power to contribute. One person's time and talent can make a huge difference. That is a movement I'm proud to be a part of.

(I originally wrote this blog post as a guest article for VentureBeat. I'm cross-posting it to my blog.)

Drupal 8 Q&A session at DrupalCon Prague

Do you have questions about the upcoming Drupal 8 release? On Thursday, September 26 at DrupalCon Prague, I'll be moderating a question-and-answer core conversation with a panel of Drupal 8 leaders, including core committers and initiative leads. Questions are submitted in advance online, and anyone can submit a question. I will curate the submissions to ask the panel the most interesting and relevant questions. We'll also publish the full list of submissions after DrupalCon (with personal information removed) so that the community has a chance to get answers for questions we don't have time for during the session.

This is a rare opportunity for the community to communicate directly with the decision-makers who are shaping Drupal 8 into the best release of Drupal yet. Help us make the most of of it -- submit your questions now!

Drupal 8 APIs are freezing but not frozen

Since Drupal 8's API freeze was announced on July 1, there has been some confusion about what API freeze means, from core developers as well as module and theme developers starting Drupal 8 upgrades of their projects. This post clarifies the API completion process, and documents what different audiences can expect from Drupal 8's development cycle and when.

API freezing is a process, not a point in time

As noted in the original API freeze announcement, as well as the Drupal core release cycle page, API freeze marks the point at which API changes are only allowed when needed to fix a major or critical issue. The process of completing the necessary remaining changes, however, will take time, and while that is happening, many important APIs are still changing.

The chart below illustrates the API freezing process, as well as what various groups of people affected by Drupal 8 development can expect during the rest of the release cycle:

Drupal api freezing

For core developers, July 1 marked the point at which core maintainers began to postpone API changes not necessary for solving critical and major issues to Drupal 9, so that module and theme developers who've already started porting their projects only need to keep up with changes that are sufficiently important to a healthy Drupal 8 release. We've labeled this part of the release cycle the "API completion" to clarify this process. During this phase, any proposed API changes must go through a process involving core committer approval. Core committers will increasingly restrict what sorts of changes are allowed as time goes on.

I'd like to start porting my module or theme now. What can I expect?

Module and theme developers are encouraged to start porting now so they can uncover critical and major API problems while they can still be fixed. But if you are hoping to go through the process of porting your module or theme only once, the best time for that is after the first release candidate (see "What can I expect for beta?" below). At that point, the number of critical issues should be at or just above zero, meaning API changes should be extremely rare and only occur if there's no other way to resolve a severe problem.

You can read a summary of the most important outstanding API changes, or view the ongoing list of approved API change issues to see if your module is affected by changes that are still in progress. The list currently includes many issues related to the routing system, the entity field API, and the theme system/markup. If your project touches these systems, you may want to hold off porting a while longer, or at least be aware that these elements (as well as others) are still in flux and might change significantly before Drupal 8 is released.

What can I expect from beta releases?

Beta 1 will still be the point at which we encourage any interested site builders to try out the beta and provide feedback on it, so it will include an upgrade path from Drupal 7 for testing. It's a great time for site builders to set up test sites in order to take Drupal 8 for a spin and provide feedback while things are still somewhat malleable. Just know that because Drupal 8 is still in development, critical upgrade path issues may arise that cause data loss, so make sure you're backed up!

Previously, we also identified beta as the point at which we'd have a stable API against which module and theme developers could port code. However, after evaluating the list of major and critical API changes still outstanding, we realize that those changes will take more time to complete than we would like to wait to release our first beta. Therefore, we now recommend that developers who are not interested in providing early API feedback on Drupal 8 wait until the first release candidate.

Closing thoughts

Release management is difficult, especially in a large distributed Open Source community like Drupal. Getting Drupal 8 released involves many people, many processes, and even technology challenges. The quality of the Drupal 8 release is really important so as a result, there are no hard-and-fast rules and we'll continue to adjust in the best interest of the Drupal project. I hope this update proves that point, and that it clarifies how and when we would like you to get involved. Let's make Drupal 8 our best release to date - in a timely manner!

Managing comments on multiple WordPress sites

There is a pursuit we all share; not having to deal with spammers. The promise of a clean spam-free web.

With that in mind, I'm pleased to announce that we released a new version of the Mollom plugin for WordPress. Mollom is an anti-spam solution for websites that offers some unique features not available in other solutions like Akismet. For example, the new Mollom plugin for WordPress ships with complete support for the Mollom Content Moderation Platform — enabling you to moderate all of your WordPress sites from a single unified interface.

This is a great feature for organizations that have many sites. If you have 20 WordPress sites and 10 Drupal sites, you can now moderate these sites from a single user interface that offers powerful moderation workflows.

The new WordPress plugin leverages Mollom's PHP library that implements Mollom REST API. The library is the basis for the Mollom module for Drupal but was designed to be reused by other systems. If you have a content management system other than Drupal or WordPress, you can also connect it with the Mollom Content Moderation Platform.

Give it a try! You may like it.

Drupal 8 API freeze

July 1st has arrived. As announced earlier, this marks the start of the Drupal 8 API freeze (formerly known as the "code freeze"). I'm very excited about how Drupal 8 is shaping up; it will be a much more powerful and easier to use Drupal. While there is a lot of work ahead of us, I feel good about moving forward with the next phase of the Drupal 8 development cycle.

The two main goals of the "API freeze" are (a) to resolve release-blocking issues known as "critical bugs" and "critical tasks" and (b) to provide developers with a stable API to port their modules from Drupal 7 to Drupal 8. This means that during the API freeze we will no longer make backwards compatibility-breaking changes to public APIs except when deemed that it is necessary to resolve important bugs or where the API has already been deprecated. Changes that do not break backwards compatibility are still allowed, including API additions, at the maintainers' discretion.

During this API freeze, we're also going to do a few things differently than we did with previous release cycles. I'll explain each of those changes below.

Deprecating Drupal 7 APIs

Currently, Drupal 8 includes backwards compatibility layers that support Drupal 7 APIs while we complete conversions of core modules to the new Drupal 8 APIs. An example is the routing support in hook_menu(), which will replaced by the Drupal 8 routing system. The Drupal 7 APIs are being marked deprecated in phpDoc, and contributed module developers should not use them because they will be removed prior to the Drupal 8 release.

Deprecating Drupal 8 APIs

When appropriate, maintainers can still add new APIs to Drupal 8 that deprecate existing APIs. In this case, the deprecated APIs will continue to be supported and not be removed until Drupal 9. This is to avoid breaking contributed modules that have been upgraded to Drupal 8 already.

Adding stand-alone features

A certain class of features may get committed despite being over our commit thresholds. The main criteria are that these features have to be self contained (no follow-up tasks) and easy to roll back (limited inter-module dependencies). If a single critical or major is filed as a result of these commits, we will favour rollback over going forward. As a result, these kind of features should have almost no impact on the rest of core development, nor introduce technical debt.

The road to the first beta

During the API freeze, we will also switch from publishing alpha releases to beta releases. This will happen when there are no known critical bugs in the upgrade path from the last Drupal 7 release.

Between API freeze and beta 1, removing temporary backward compatibility layers and deprecated Drupal 7 functions is allowed and encouraged. After beta 1, we get more strict about backward compatibility breaks (temporary backward compatibility layers and deprecated functions not removed by then might not be eligible for removal any more).

Summary

After more than 2 years of non-stop work, it is pretty exciting to enter API freeze. It is a big milestone for all of us. Frankly, Drupal 8 will be our best release ever, by far, and I can't wait to get it in your hands. But we can't get Drupal 8 released without you. Please take a moment of your time to download and try out the alpha releases. If you are a module developer, make sure the things that are important to you are working, and working well so you can start upgrading your modules the day we start releasing betas. If you find a problem, please report it. Every bug you uncover is a chance to improve the experience for millions of users. Thank you, and we hope to see you in the issue queues!

Drupal 8 alpha releases

A month ago we started the Drupal 8 alpha cycle to encourage module developers to test out Drupal 8 and to try upgrading their modules. Today, we published the second alpha release of Drupal 8: Drupal 8.x alpha 2!

I think it is exciting that after years of hard work by many, we have now begun to post alpha releases.

The purpose of alpha releases is to allow module developers to identify API deficiencies while they can still be corrected. We want Drupal's API to be easy to learn, easy to use (even without documentation), hard to misuse, easy to read and maintain, and easy to extend. Good API design really matters and we only have one chance to get it right so please download the alpha release, try upgrading some contributed modules, and provide us feedback along with suggestions for improvement.

Time is of the essence as API changes to Drupal 8 are only allowed for a little longer. We're about to enter the polish phase of the Drupal 8 development cycle, where we will soon switch to beta releases and no longer allow API changes unless needed to fix release blocking issues. From then on, most API improvements will have to wait until Drupal 9.

Pages

© 1999-2014 Dries Buytaert Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License.
Drupal is a Registered Trademark of Dries Buytaert.