You are here

Drupal

How Acquia plans to give back to Drupal 8 in Q4

People wonder what we do at Acquia's Office of the CTO (OCTO). In order to provide some more transparency, I wanted to share how we plan to give back between today and the end of the year.

Drupal 8 beta 1

Now that we're forgoing an upgrade path for a migration path, we need to redefine the release criteria for 'beta 1'. We also need to track the issues that block beta in order to help escalate the most critical of the critical issues. We will work with the Drupal 8 core maintainers to define and communicate these criteria, and help with timely patch reviews and issue management for beta-blocking issues.

Migrate module in core

As a co-author of the Migrate module, Moshe will be assisting the core development team on the goal to get Migrate module functionality into core and support a Drupal 6 to Drupal 8, as well as the Drupal 7 to Drupal 8 migration path.

Improve the Drupal 8 developer experience

We want developers to be productive and enjoy writing Drupal 8 modules, so we will be fleshing out the D8DX battleplan based on discussions at DrupalCon Prague, and working with others in the Drupal community to ensure that we fix the most important developer experience problems in preparation of Drupal 8's release.

Develop learning resources

We will be working on a central resource on Drupal.org for developers to find the information they need in order to port their modules to Drupal 8, including documentation, tools, and other resources.

Evaluate semantic versioning

There was a lot of talk at DrupalCon Prague about lessons learned in Drupal 7 and 8 and how to do better in the future. One such area of improvement is a release strategy that allows for iterative innovation in Drupal core every few months. We plan to work with other community leaders and the Drupal security team in order to come up with a strategy around this. A component of this strategy may be to adopt semantic versioning.

Improve Drupal 8 performance

We will also dedicate the OCTO team's time to help the Drupal core community identify and fix some major performance issues in Drupal core.

Authoring experience improvements

Major development efforts on the UX features the Spark team helped get into Drupal 8 core—WYSIWYG, in-place editing, mobile-friendly toolbar —are winding down. We still have work to do on fixing up some loose ends, and are committed to see this through. We will also backport the major Spark modules to Drupal 7.

Communicate Drupal 8 progress

We aim to continue the weekly D8 progress reports that you can find at This Week In Core, and are actively seeking other core developers to help get these important posts out.

A drop in an ocean

We're a handful of contributors in the OCTO and can only do so much. We will continue doing these things within a community of hundreds of other contributors and supporting their work in other ways. We're looking forward to much Drupal 8 progress in the next 3 months!

To start an Open Source project or not?

People ask me what it is like to be the head of a big Open Source project, and whether they should Open Source their project or not. I wanted to talk about that a bit more in this blog post so more people can pick up my answer.

Having been the project lead of the Drupal project for the past 13 years, I’ve watched my dorm-room activity transform into a community filled with passionate people all working toward the same goal: changing the world and making it a better place through open source.

Today Drupal powers more than 1.5 million sites. Drupal is a source of innovation for business and government. Most importantly, Drupal has helped individuals build a dream, giving smaller groups and organizations a bigger voice, as tools are democratized. But it has also allowed large businesses to develop new ideas, bring and build transformative experiences to the digital world.

The ambitious individuals who would lead the next generation of open source projects will experience moments of joy and excitement. It's exhilarating when your passion drives you to help create solutions to challenging problems. Your joy will be tempered with plenty of moments of frustration and doubt, as roadblocks may stand in your way during crucial points of development. But the successful leaders will be the ones who aren’t dissuaded from their work.

Creating a successful open source project requires much more work than writing good code. If your project is growing, then one day you'll start to see that you are a leader. You’re creating a vision, a culture, and inspiring people to come on board. This evangelism requires a lot of travel, conferences, fundraising, people management, project management and more. Make sure this passion is also within you.

I’ve had the opportunity to travel the world, evangelizing Drupal and have a leading role in a passionate, active community that is making a real difference. I’ve also founded a non-profit organization and a commercial company on that same promise.

As you start to build a community of participants who are willing to commit their time and passion to your project, you’ll soon realize that in life, the luckiest people in the world are those driven by the desire to be a part of something great. When you work in open source, you’ll be surrounded by people like these. Knowing you help make a difference and that hundreds of thousands of people depend on your project, helps you make sense of your commitment. So even on a bad day, it's still exciting.

The world would certainly benefit from having more Open Source, but its not a small undertaking as others come to depend on it. Only you can decide whether you have what it takes. When I started Drupal, I didn't really understand what I was getting myself into. It has been a lot of work, but knowing what I know today, I'd do it again. In a heartbeat.

State of Drupal presentation (September 2013)

Last week in Prague, I gave my traditional State of Drupal presentation. A total of 1,830 Drupalists were present at DrupalCon, a new record for our European DrupalCon!

In good tradition, you can download a copy of my slides (PDF, 31 MB) or you can watch a video recording of my keynote. The keynote starts at 11:42, but don't miss out on the singing carrots introduction. A video recording of the keynote is embedded in this post.

Why the big architectural changes in Drupal 8

There has been a lot of chatter about Drupal 8. Will Drupal 8 be performant? Will Drupal 8 be easy to develop modules for? Will I have to learn Symfony? I want to address these concerns and explain why Drupal 8 introduces such big changes.

Lessons from the past: the pain before the gain

The reason Drupal has been successful is because we always made big, forward-looking changes. It’s a cliché, but change has always been the only constant in Drupal. The result is that Drupal has stayed relevant, unlike nearly every other Open Source CMS over the years. The biggest risk for our project is that we don't embrace change.

The downside is that with every major release of Drupal, we've gone through a lot of pain adjusting to this change. I first wrote about it in 2006 when trying to get Drupal 4.7 released:

"So let's capture that thought for future reference. Sweeping changes are required to make major advances in technology, and often times there is a lot of pain before the pay-off."

We decided that big changes were required

Drupal 7 is a fantastic CMS, but at the same time there are some fairly big limitations, including an incomplete Entity API, a lack of separation between content and configuration, leading to deployment challenges, a lack of separation between logic and presentation in the theme layer, and more. We created solutions for those challenges using contributed modules, but those solutions were in many cases incomplete. In Drupal 8, we decided to tackle a lot of these problems head-on, through the Configuration Management Initiative, the Twig templating layer, and a complete Entity API.

The web landscape has also dramatically changed around Drupal since January 2011, when Drupal 7 was released. Mobile browsing is ubiquitous, and so are third-party services that people may want to integrate their Drupal sites with. Web site users expect a much higher bar when it comes to ease of use. We anticipated these trends and as a result, we spent the past 2.5 years working on Drupal 8's mobile features and user experience improvements.

But are all these great improvements enough?

We need to modernize Drupal 8

One of our biggest challenges with Drupal, is that it is hard for organizations of all sizes to find Drupal talent (developers, themers, site builders, etc). Drupal 7 didn't address this problem (e.g. we held on to procedural programming instead of object-oriented programming), and in fact made it a bit worse with the introduction of even more "Drupalisms" (e.g. excessive use of structured arrays). For most people new to Drupal, Drupal 7 is really complex.

The most effective way to address the Drupal talent issue, as well as the complexity issue, is to bring Drupal in line with modern frameworks and platforms, so there is less Drupal-specific knowledge to learn in order to become proficient. We decided to adopt modern PHP concepts and standards, object-oriented programming, and the Symfony framework. While a lot of the Drupal concepts (Fields, Views, Entities, Nodes) continue to exist in Drupal 8, they are now implemented using object-oriented programming design patterns.

The advantages and disadvantages of object-oriented programming are well-understood. The disadvantages are size, verbosity, the amount of work it takes to write (including the design planning that goes into it) and slower performance. For people new to object-oriented programming there may be a steep learning curve; some of the key programming techniques, such as inheritance and polymorphism, can be challenging initially. The advantages are encapsulation (both to hide implementation details and to avoid tampering with internal values), faster development thanks to re-use, extensibility, and better maintainability. Compared to procedural programs, object-oriented programs are easier to maintain, extend and refactor. So although a lot of work is spent to write the program, less work is needed to maintain it over time.

For Drupal 8 this means that the code will be more abstract, more verbose, and slower, yet also be more maintainable, more modular, and more accessible to non-Drupal developers. The end result is that Drupal 8 should help us attract new people to Drupal in a way Drupal 7 didn't.

This is exactly what happened with other projects like Symfony; Symfony 2 was a complete rearchitecture of Symfony 1. A lot of people were alienated by that, yet at the same time Symfony 2 took off like a rocketship. The same thing has happened with the major releases of Drupal as well, despite how much change each one brings.

Change is scary for the existing Drupal developers

However, as we get closer to the release of Drupal 8, existing Drupal developers have become increasingly aware of the massive changes that Drupal 8 will bring. This has resulted in some fear; some people feel like they have to re-learn everything they know, and that developing for Drupal 8 has changed to the point where it's no longer fun. This fear is completely understandable. Change is hard, it can be scary, and often takes a long time to be absorbed.

But even if we completely streamlined the Drupal 8 developer experience, Drupal 8 will still look and work radically different under the hood. As mentioned, there are advantages and disadvantages to object-oriented programming and modern design patterns. But if we care about the long-term success of Drupal, we can't preserve the past. The risk of sticking with the old Drupal 7 architecture is that we won't be able to attract many more Drupal developers, and that over time, Drupal will become the odd one out.

There is a lot of work left to be done

Part of the fear out there is well-founded because in the current state of development, Drupal 8 isn't good enough. While there are many things to like about Drupal 8, I'm also the first to admit that we have work to do on the Drupal 8 developer experience (as well as performance) before Drupal 8 can ship. Creating a well-designed object-oriented application takes more time and design work than creating a procedural one. Some major improvements have already landed, but we're still working hard on improving the developer experience. We need more help, especially from Drupal 7 developers, on how we can make Drupal 8 more approachable for them. I'm not afraid to make major API changes if the developer experience improvement is suitably large. So if you have concerns about Drupal 8, now is the time to step up and help. In return, I promise we'll work on Drupal 8 until it is ready. Thank you!

Open Source code: by the people, for the people

Many organizations adopt Open Source for reasons like flexibility and agility. Everyone needs to do more with less. But in government, Open Source drives both civic engagement and government participation like never before. Because of digital, the world feels much smaller and more connected. And Open Source gives people the opportunity to rally around a cause, no matter where they live.

Think about how we petitioned our government before we had the We The People website. I bet you have to think pretty hard about how it was done (I do!). Now, a website has brought to life the First Amendment rights of U.S. citizens. Millions of people's voices are heard. People pull together based on common concerns. The White House built We The People using Drupal and shared the code on GitHub, opening up the opportunity for other governments to easily create their own online petitioning systems.

Now, all kinds of open government data made available through the Data.gov project makes it possible for any developer, anywhere, to create a civic app. These apps have made us see our cities and towns in a different light.

Open City is one example of a group of local volunteers who create Open Source apps using government data. While the group is based in Chicago, the idea is that any city can grab code from an Open City app and make it their own.

Here are a few interesting examples: Clear Streets tracks a city's plows in real time. Living outside of Boston, I know we could use an app like that! Crime in Chicago lets citizens compare crime statistics in certain areas of town, which could be useful for people making decisions about where to move their families.

What is perhaps the most gratifying is that as open-source developers, we can collaborate on projects and help people around the world. It's part of what gets us out of bed in the morning. Earlier this year, participants in DrupalCon Portland launched a website in 24 hours to help people in Moore, Oklahoma, find transportation and housing after the tornado. Two weeks later, the site was discovered on Twitter in Germany and was repurposed to help people affected by the flooding in northern Europe. This type of project inspires us all to see how technology can make an immediate difference.

Other events, such as the National Day for Civic Hacking, encourage developers to use open government data to "collaboratively create, build, and invent". The idea that hackathons can help build and create a healthy, citizen-powered technology ecosystem within government is relatively new, and full of promise. Tim O'Reilly believes that government can escape the "vending machine" mentality (citizens put in tax dollars and get out services) by thinking of "Government as a Platform" for participation. I couldn't agree more.

Open Source ideals are already spreading in governments throughout the world, with good reason. A global network of developers is motivated to help. It's one of the best examples of civic engagement. As digital citizens, we all now have the power to contribute. One person's time and talent can make a huge difference. That is a movement I'm proud to be a part of.

(I originally wrote this blog post as a guest article for VentureBeat. I'm cross-posting it to my blog.)

Drupal 8 Q&A session at DrupalCon Prague

Do you have questions about the upcoming Drupal 8 release? On Thursday, September 26 at DrupalCon Prague, I'll be moderating a question-and-answer core conversation with a panel of Drupal 8 leaders, including core committers and initiative leads. Questions are submitted in advance online, and anyone can submit a question. I will curate the submissions to ask the panel the most interesting and relevant questions. We'll also publish the full list of submissions after DrupalCon (with personal information removed) so that the community has a chance to get answers for questions we don't have time for during the session.

This is a rare opportunity for the community to communicate directly with the decision-makers who are shaping Drupal 8 into the best release of Drupal yet. Help us make the most of of it -- submit your questions now!

Drupal 8 APIs are freezing but not frozen

Since Drupal 8's API freeze was announced on July 1, there has been some confusion about what API freeze means, from core developers as well as module and theme developers starting Drupal 8 upgrades of their projects. This post clarifies the API completion process, and documents what different audiences can expect from Drupal 8's development cycle and when.

API freezing is a process, not a point in time

As noted in the original API freeze announcement, as well as the Drupal core release cycle page, API freeze marks the point at which API changes are only allowed when needed to fix a major or critical issue. The process of completing the necessary remaining changes, however, will take time, and while that is happening, many important APIs are still changing.

The chart below illustrates the API freezing process, as well as what various groups of people affected by Drupal 8 development can expect during the rest of the release cycle:

Drupal api freezing

For core developers, July 1 marked the point at which core maintainers began to postpone API changes not necessary for solving critical and major issues to Drupal 9, so that module and theme developers who've already started porting their projects only need to keep up with changes that are sufficiently important to a healthy Drupal 8 release. We've labeled this part of the release cycle the "API completion" to clarify this process. During this phase, any proposed API changes must go through a process involving core committer approval. Core committers will increasingly restrict what sorts of changes are allowed as time goes on.

I'd like to start porting my module or theme now. What can I expect?

Module and theme developers are encouraged to start porting now so they can uncover critical and major API problems while they can still be fixed. But if you are hoping to go through the process of porting your module or theme only once, the best time for that is after the first release candidate (see "What can I expect for beta?" below). At that point, the number of critical issues should be at or just above zero, meaning API changes should be extremely rare and only occur if there's no other way to resolve a severe problem.

You can read a summary of the most important outstanding API changes, or view the ongoing list of approved API change issues to see if your module is affected by changes that are still in progress. The list currently includes many issues related to the routing system, the entity field API, and the theme system/markup. If your project touches these systems, you may want to hold off porting a while longer, or at least be aware that these elements (as well as others) are still in flux and might change significantly before Drupal 8 is released.

What can I expect from beta releases?

Beta 1 will still be the point at which we encourage any interested site builders to try out the beta and provide feedback on it, so it will include an upgrade path from Drupal 7 for testing. It's a great time for site builders to set up test sites in order to take Drupal 8 for a spin and provide feedback while things are still somewhat malleable. Just know that because Drupal 8 is still in development, critical upgrade path issues may arise that cause data loss, so make sure you're backed up!

Previously, we also identified beta as the point at which we'd have a stable API against which module and theme developers could port code. However, after evaluating the list of major and critical API changes still outstanding, we realize that those changes will take more time to complete than we would like to wait to release our first beta. Therefore, we now recommend that developers who are not interested in providing early API feedback on Drupal 8 wait until the first release candidate.

Closing thoughts

Release management is difficult, especially in a large distributed Open Source community like Drupal. Getting Drupal 8 released involves many people, many processes, and even technology challenges. The quality of the Drupal 8 release is really important so as a result, there are no hard-and-fast rules and we'll continue to adjust in the best interest of the Drupal project. I hope this update proves that point, and that it clarifies how and when we would like you to get involved. Let's make Drupal 8 our best release to date - in a timely manner!

Pages

© 1999-2014 Dries Buytaert Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License.
Drupal is a Registered Trademark of Dries Buytaert.