In the past, after every major release of Drupal, most innovation would shift to two areas: (1) contributed modules for the current release, and (2) core development work on the next major release of Drupal. This innovation model was the direct result of several long-standing policies, including our culture of breaking backward compatibility between major releases.
In many ways, this approach served us really well. It put strong emphasis on big architectural changes, for a cleaner, more modern, and more flexible codebase. The downsides were lengthy release cycles, a costly upgrade path, and low incentive for core contributors (as it could take years for their contribution to be available in production). Drupal 8's development was a great example of this; the architectural changes in Drupal 8 really propelled Drupal's codebase to be more modern and flexible, but also came at the cost of four and a half years of development and a complex upgrade path.
As Drupal grows — in lines of code, number of contributed modules, and market adoption — it becomes harder and harder to rely purely on backward compatibility breaks for innovation. As a result, we decided to evolve our philosophy starting after the release of Drupal 8.
The only way to stay competitive is to have the best product and to help people adopt it more seamlessly. This means that we have to continue to be able to reinvent ourselves, but that we need to make the resulting changes less scary and easier to absorb. We decided that we wanted more frequent releases of Drupal, with new features, API additions, and an easy upgrade path.
To achieve these goals, we adopted three new practices:
- Semantic versioning: a major.minor.patch versioning scheme that allows us to add significant, backwards-compatible improvements in minor releases like Drupal 8.1.0 and 8.2.0.
- Scheduled releases: new minor releases are timed twice a year for predictability. To ensure quality, each of these minor releases gets its own beta releases and release candidates with strict guidelines on allowed changes.
- Experimental modules in core: optional alpha-stability modules shipped with the core package, which allow us to distribute new functionality, gather feedback, and iterate faster on the modules' planned path to stability.
Now that Drupal 8 has been released for about 10 months and Drupal 8.2 is scheduled to be released next week, we can look back at how this new process worked. Drupal 8.1 introduced two new experimental modules (the BigPipe module and a user interface for data migration), various API additions, and usability improvements like spell checking in CKEditor. Drupal 8.2 further stabilizes the migration system and introduces numerous experimental alpha features, including significant usability improvements (i.e. block placement and block configuration), date range support, and advanced content moderation — among a long list of other stable and experimental improvements.
It's clear that these regular feature updates help us innovate faster — we can now add new capabilities to Drupal that previously would have required a new major version. With experimental modules, we can get features in users' hands early, get feedback quickly, and validate that we are implementing the right things. And with the scheduled release cycle, we can deliver these improvements more frequently and more predictably. In aggregate, this enables us to innovate continuously; we can bring more value to our users in less time in a sustainable manner, and we can engage more developers to contribute to core.
It is exciting to see how Drupal 8 transformed our capabilities to continually innovate with core, and I'm looking forward to seeing what we accomplish next! It also raises questions about what this means for Drupal 9 — I'll cover that in a future blog post.
Four weeks ago we went on a vacation in Tuscany. I finally had some time to process the photos and write down our memories from the trip.
We booked a last-minute house in a vineyard called Fattoria di Fubbiano. The vineyard has been producing wine and olive oil since the 14th century. On the eastern edge of the estate, is Al Magrini, a Tuscan farmhouse surrounded by vines and olive trees.
When we arrived, we were struck by the remoteness. We had to drive through dirt roads for 10 minutes to get to our house. But once we got there, we were awestruck. The property overlooks a valley of olive groves and vines. We could have lunch and dinner outside among the rose bushes, and enjoy our own swimming pool with its own sun beds, deck chairs and garden umbrellas.
While it was full of natural beauty, it was also very simple. We quickly realized there was no TV or internet, no living room, and only a basic kitchen; we couldn't run two appliances at the same time. But nothing some wine and cheese can't fix. After some local cheese, olives and wine, we went for a swim in the pool. Vacation had started!
We had dinner in a great little restaurant in the middle of nowhere. We ate some local, traditional food called "tordelli lucchesi". Nearly every restaurant in Lucca serves a version of this traditional Lucchesan dish. Tordelli look like ravioli, but that is where the resemblance ends. The filling is savory rather than cheesy, and the cinnamon- and sage-infused ragù with which the tordelli are served is distinctly Tuscan. The food was exceptional.
We were woken up by loud screaming from Stan: "Axl got hurt! He fell out of the window!". Our hearts skipped several beats because the bedrooms were on the second floor and we told them they couldn't go downstairs in the morning.
Turns out Axl and Stan wanted to surprise us by setting the breakfast table outside. They snuck downstairs and originally set the table inside, wrote a sweet surprise note in their best English, and made "sugar milk" for everyone -- yes, just like it sounds they added tablespoons full of sugar to the milk. Axl then decided he wanted to set the table outside instead. They overheard us saying how much we enjoyed eating breakfast outside last time we were in Italy. They couldn't open the door to the backyard so Axl decided to climb out of the window, thinking he could unlock the door from the outside. In the process, he fell out of the window from about one meter. Fortunately since it was a first floor window (ground level window), Axl got nothing but a few scratches. Sweet but scary.
Later on, we went to the grocery store and spent most of the day at the pool. The boys can't get enough of playing in the water with the inflatable crocodile "Crocky" raft Stan had received for his birthday two years ago. Vanessa can't get enough of the sun and she also confiscated my Kindle.
With no Kindle to read on, I discovered poop next to the pool. I thought it was from a wild horse and was determined to go to look for it in the coming days.
In the late afternoon, we had snacks and prosecco, something which became our daily tradition on vacation. The Italian cheese was great and the "meloni" was so sweet. The food was simple, but tasted so much better than at home. Maybe it's the taste of vacation.
Vanessa did our first load of laundry which needed to dry in the sun. The clothes were a little crunchy, but there was something fulfilling about the simplicity of it.
In good tradition, I made coffee in the morning. As I head downstairs the morning light peeks through all the cracks of the house, and highlights the old brick and stone walls. The coffee machine is charmingly old school. We had to wait 20 minutes or so for the whole pot to brew.
Vanessa made french toast for breakfast. She liked to shout in Dutch "Het is vakantie!" during the breakfast preparation. Stan moaned repeatedly during breakfast - he loved the french toast! It made us laugh really hard.
Today was a national holiday in Italy so everything is closed. We decided to spend the time at the pool; no one was complaining about that. Most weeks feels like a marathon at work, so it was nice to do absolutely nothing for a few days, not keep track of time, and just enjoy our time together.
To take a break from the pool, we decided to walk through the olive groves looking for those wild horses. Axl and Stan weren't especially fond of the walk as it started off uphill. Stan told us "I'm sweating" as if we would turn back. Instead of wild horses we found a small mountain village. The streets were empty and the shutters were closed to keep the peak heat of the day out. It seemed like we had stepped back in time 30-40 years.
Sitting next to the pool gave me a lot of time to think and reflect. It's nice to have some headspace. Our afternoon treat by the pool was iced coffee! We kept the leftover coffee from the morning to pour over ice for a refreshing drink. One of Vanessa's brilliant ideas.
Our evening BBQs are pretty perfect. We made Spanish style bruschetta; first grilling the bread, then rubbing it with garlic and tomato, drizzle some local olive oil over it, and add salt and pepper. After the first bite it was requested we make this more often.
We really felt we're all connecting. We even had an outdoor dance party as the sun was setting. Axl wrote in our diary: "Vanessa laughed so hard she almost peed her pants. LOL.". Stan wanted to know if his moves made her laugh the hardest.
Every evening we would shower to wash off the bug spray, because mosquitos were everywhere. When it was finally my time to shower, we ran out of water -- just when I was all soaped up. Fortunately, we had a bottle of Evian that I could use to rinse off (just like the Kardashians).
We set the alarm for 7:30am so we could head to Lucca, a small city 30 minutes from our house -- 15 minutes of that is spent getting out of the vineyard and mountain trails. We were so glad we rented "Renny", our 4x4 Jeep Renegade, as there are no real paved roads in the vineyard.
We visited "La Boutique Dei Golosi", a tiny shop that sold local wines, olive oils and other Italian goods. The shop owner, Alain, opened bottles of wine and let us taste different olive oils on bread. He offered the boys samples of everything the adults tried and was impressed that they liked it. Interestingly enough, all four of us preferred the same olive oil. We shipped 5 bottles home, along with several bottles of wine, limoncello and 3 jars of white truffle paste. It was fun knowing a big box of Italian goods would arrive once we were home.
When we got back from Lucca, we fired up the grill and drank our daily bottle of prosecco. Every hour we hear bells ring -- it's from the little town up on the hill. The bells are how we kept track of time. The go-at-your-own-pace lifestyle is something all North Americans should experience. The rhythm of Tuscany's countryside is refreshing -- the people there know how to live.
Axl and Stan enjoyed the yard. When they weren't playing soccer or hunting for salamanders, they played ninjas using broomsticks. Axl was "Poison Ivy" and Stan was "Bamboo Sham". Apparently, they each have special moves that they can use once every battle.
Today we went wine tasting at our vineyard, Fattoria di Fubbiano, and got a tour of the cellar. It was great that the tour was in "inglese". We learned that they manage 45 hectares and produce 100,000 bottle of wine annually. We bought 21 of them and shipped them home so there is only 99,979 left. The best part? We could walk home afterwards. :)
Our charcoal reserves are running low; a sign of a great vacation.
We visited Montecatini Alto, about a 40 minute drive from our house. To get to Montecatini Alto, we took a funicular built in 1898. They claim it is the oldest working cable car in the world. I believe them.
Montecatini Alto is a small medieval village that dates back to 1016. It's up on a hill. The views from the village are amazing, overlooking a huge plain. I closed my eyes and let my mind wonder, trying to image how life was back then over a thousand year ago.
The city square was filled with typical restaurants, cafes and shops. We poked around in some of the shops and Stan found a wooden sword he wanted, but couldn't decide if that's what he wanted to spend his money on. To teach Axl and Stan about money, we let them spend €20 of their savings on vacation. Having to use their own money made them think long and hard on their purchases. Since the shops close from 1pm to 2:30pm, we went for lunch in one of the local restaurants on the central square while Stan contemplated his purchase. It's great to see Axl explore the menu and try new things. He ordered the carbonara and loved it. Stan finally decided he wanted the sword bad enough, so we went back and he bought it for €10.
When we got back to our vineyard, we spotted wild horses! Finally proof that they exist. Vanessa quickly named them Hocus, Pocus and Dominocus.
In the evening we had dinner in a nearby family restaurant called "Da Mi Pa". The boys had tordelli lucchesi and then tiramisu for dessert. Chances are slim but I hope that they will remember those family dinners. They talked about the things that are most important in life, as well as their passions (computer programming for Axl and soccer for Stan). The conversations were so fulfilling and a highlight of the vacation.
Spontaneous last minute decision on what to do today. We came up with a list of things to do and Axl came up with a voting system. We decided to visit the Leaning Tower of Pisa. We were all surprised how much the tower actually leans and of course we did the goofy photos to prove we were there. These won't be published.
Last day of the vacation. We're all a bit sad to go home. The longer we stay, the happier we get. Happier not because of where we were, but about how we connected.
Today, we're making the trek to Florence. One of the things Florence is known for is leather. Vanessa wanted to look for a leather jacket, and I wanted to look for a new duffel bag. We found a shop that was recommended to us; one of the shop owners is originally from the Greater Boston area. Enio, her husband, was very friendly and kind. He talked about swimming in Walden Pond, visiting the Thoreau's House, etc. The boys couldn't believe he had been to Concord, MA. Enio really opened up and gave us a private tour to his leather workshop. His workshop consisted of small rooms filled with piles and piles of leather and all sorts of machinery and tools.
I had a belt made with my initials on it (on the back). Stan got a bracelet made out of the leftover leather from the belt. Axl also got a bracelet made, and both had their initials stamped on them. Vanessa bought a gorgeous brown leather jacket, a purse and funky belt. And last but not least, l found a beautiful handmade ram-skin duffel bag in a cool green color. Enio explained that it takes him two full days to make the bag. It was expensive but will hopefully last for many, many years. I wanted to buy a leather jacket but as usual they didn't have anything my size.
We strolled across the Ponte Vecchio and made some selfies (like every other tourist). We had a nice lunch. Pasta for Vanessa, Axl and myself. Stan still has an aversion to ragù even though he ate it 3 times that week and loved it every time. Then we had our "grand finale gelato" before we headed to the airport.
I've made no secret of my interest in the open web, so it won't come as a surprise that I'd love to see more web applications and fewer native applications. Nonetheless, many argue that "the future of the internet isn't the web" and that it's only a matter of time before walled gardens like Facebook and Google — and the native applications which serve as their gatekeepers — overwhelm the web as we know it today: a public, inclusive, and decentralized common good.
I'm not convinced. Native applications seem to be winning because they offer a better user experience. So the question is: can open web applications, like those powered by Drupal, ever match up to the user experience exemplified by native applications? In this blog post, I want to describe inversion of control, a technique now common in web applications and that could benefit Drupal's own user experience.
Native applications versus web applications
Using a native application — for the first time — is usually a high-friction, low-performance experience because you need to download, install, and open the application (Android's streamed apps notwithstanding). Once installed, native applications offer unique access to smartphone capabilities such as hardware APIs (e.g. microphone, GPS, fingerprint sensors, camera), events such as push notifications, and gestures such as swipes and pinch-and-zoom. Unfortunately, most of these don't have corresponding APIs for web applications.
A web application, on the other hand, is a low-friction experience upon opening it for the first time. While native applications can require a large amount of time to download initially, web applications usually don't have to be installed and launched. Nevertheless, web applications do incur the constraint of low performance when there is significant code weight or dozens of assets that have to be downloaded from the server. As such, one of the unique challenges facing web applications today is how to emulate a native user experience without the drawbacks that come with a closed, opaque, and proprietary ecosystem.
Inversion of control
In the spirit of open source, the Drupal Association invited experts from the wider front-end community to speak at DrupalCon New Orleans, including from Ember and Angular. Ed Faulkner, a member of the Ember core team and contributor to the API-first initiative, delivered a fascinating presentation about how Drupal and Ember working in tandem can enrich the user experience.
Inversion of control in end user experiences
Ed's changes are not in production on Drupal.com, but in his demo, clicking a teaser image causes it to "explode" to become the hero image of the destination page. Pairing Ember with Drupal in this way allows a user to visually and mentally transition from a piece of teaser content to its corresponding page via an animated transition between pages — all without a page refresh. The animation is very impressive and the animated GIF above doesn't do it full justice. While this transition across pages is similar to behavior found in native mobile applications, it's not currently possible out of the box in Drupal without extensive client-side control.
These impressive interactions are possible using the Ember plugin Liquid Fire. Fewer than 20 lines of code were needed to build the animations in Ed's demo, much like how SDKs for native mobile applications provide easy-to-implement animations out of the box. Of course, Ember isn't the only tool capable of this kind of functionality. The RefreshLess module for Drupal by Wim Leers (Acquia) also uses client-side control to enable navigating across pages with minimal server requests. Unfortunately, RefreshLess can't tap into Liquid Fire or other Ember plugins.
Inversion of control in editorial experiences
We can apply this principle of inversion of control not only to the end user experience but also to editorial experiences. The last demos in Ed's presentation depict CardStack Editor, a fully decoupled Ember application that uses inversion of control to overlay an administrative interface to edit Drupal content, much like in-place editing.
Inversion of control as the rule or exception?
Those of you following the outside-in work might have noticed some striking similarities between outside-in and the work Ed has been doing: both use inversion of control. The primary purpose of our outside-in interfaces is to provide for an in-context editing experience in which state changes take effect live before your eyes; hence the need for inversion of control.
Thinking about the future, we have to answer the following question: does Drupal want inversion of control to be the rule or the exception? We don't have to answer that question today or tomorrow, but at some point we should.
In the longer term, client-side frameworks like Ember will allow us to build web applications which compete with and even exceed native applications with regard to perceived performance, built-in interactions, and a better developer experience. But these frameworks will also enrich interactions between web applications and device hardware, potentially allowing them to react to pinch-and-zoom, issue native push notifications, and even interact with lower-level devices.
In the meantime, I maintain my recommendation of (1) progressive decoupling as a means to begin exploring inversion of control and (2) a continued focus on the API-first initiative to enable application-like experiences to be developed on Drupal.
I'm hopeful Drupal can exemplify how the open web will ultimately succeed over native applications and walled gardens. Through the API-first initiative, Drupal will provide the underpinnings for web and native applications. But is it enough?
Inversion of control is an important principle that we can apply to Drupal to improve how we power our user interactions and build robust experiences for end users and editors that rival native applications. Doing so will enable us to enhance our user experience long into the future in ways that we may not even be able to think of now. I encourage the community to experiment with these ideas around inversion of control and consider how we can apply them to Drupal.
There exist millions of Open Source projects today, but many of them aren't sustainable. Scaling Open Source projects in a sustainable manner is difficult. A prime example is OpenSSL, which plays a critical role in securing the internet. Despite its importance, the entire OpenSSL development team is relatively small, consisting of 11 people, 10 of whom are volunteers. In 2014, security researchers discovered an important security bug that exposed millions of websites. Like OpenSSL, most Open Source projects fail to scale their resources. Notable exceptions are the Linux kernel, Debian, Apache, Drupal, and WordPress, which have foundations, multiple corporate sponsors and many contributors that help these projects scale.
We (Dries Buytaert is the founder and project lead of Drupal and co-founder and Chief Technology Officer of Acquia and Matthew Tift is a Senior Developer at Lullabot and Drupal 8 configuration system co-maintainer) believe that the Drupal community has a shared responsibility to build Drupal and that those who get more from Drupal should consider giving more. We examined commit data to help understand who develops Drupal, how much of that work is sponsored, and where that sponsorship comes from. We will illustrate that the Drupal community is far ahead in understanding how to sustain and scale the project. We will show that the Drupal project is a healthy project with a diverse community of contributors. Nevertheless, in Drupal's spirit of always striving to do better, we will also highlight areas where our community can and should do better.
Who is working on Drupal?
In the spring of 2015, after proposing ideas about giving credit and discussing various approaches at length, Drupal.org added the ability for people to attribute their work to an organization or customer in the Drupal.org issue queues. Maintainers of Drupal themes and modules can award issues credits to people who help resolve issues with code, comments, design, and more.
Drupal.org's credit system captures all the issue activity on Drupal.org. This is primarily code contributions, but also includes some (but not all) of the work on design, translations, documentation, etc. It is important to note that contributing in the issues on Drupal.org is not the only way to contribute. There are other activities—for instance, sponsoring events, promoting Drupal, providing help and mentoring—important to the long-term health of the Drupal project. These activities are not currently captured by the credit system. Additionally, we acknowledge that parts of Drupal are developed on GitHub and that credits might get lost when those contributions are moved to Drupal.org. For the purposes of this post, however, we looked only at the issue contributions captured by the credit system on Drupal.org.
What we learned is that in the 12-month period from July 1, 2015 to June 30, 2016 there were 32,711 issue credits—both to Drupal core as well as all the contributed themes and modules—attributed to 5,196 different individual contributors and 659 different organizations.
Despite the large number of individual contributors, a relatively small number do the majority of the work. Approximately 51% of the contributors involved got just one credit. The top 30 contributors (or top 0.5% contributors) account for over 21% of the total credits, indicating that these individuals put an incredible amount of time and effort in developing Drupal and its contributed modules:
How much of the work is sponsored?
As mentioned above, from July 1, 2015 to June 30, 2016, 659 organizations contributed code to Drupal.org. Drupal is used by more than one million websites. The vast majority of the organizations behind these Drupal websites never participate in the development of Drupal; they use the software as it is and do not feel the need to help drive its development.
Technically, Drupal started out as a 100% volunteer-driven project. But nowadays, the data suggests that the majority of the code on Drupal.org is sponsored by organizations in Drupal's ecosystem. For example, of the 32,711 commit credits we studied, 69% of the credited work is "sponsored".
We then looked at the distribution of how many of the credits are given to volunteers versus given to individuals doing "sponsored work" (i.e. contributing as part of their paid job):
Looking at the top 100 contributors, for example, 23% of their credits are the result of contributing as volunteers and 56% of their credits are attributed to a corporate sponsor. The remainder, roughly 21% of the credits, are not attributed. Attribution is optional so this means it could either be volunteer-driven, sponsored, or both.
As can be seen on the graph, the ratio of volunteer versus sponsored don't meaningfully change as we look beyond the top 100—the only thing that changes is that more credits that are not attributed. This might be explained by the fact that occasional contributors might not be aware of or understand the credit system, or could not be bothered with setting up organizational profiles for their employer or customers.
As shown in jamadar's screenshot above, a credit can be marked as volunteer and sponsored at the same time. This could be the case when someone does the minimum required work to satisfy the customer's need, but uses his or her spare time to add extra functionality. We can also look at the amount of code credits that are exclusively volunteer credits. Of the 7,874 credits that marked volunteer, 43% of them (3,376 credits) only had the volunteer box checked and 57% of them (4,498) were also partially sponsored. These 3,376 credits are one of our best metrics to measure volunteer-only contributions. This suggests that only 10% of the 32,711 commit credits we examined were contributed exclusively by volunteers. This number is a stark contrast to the 12,888 credits that were "purely sponsored", and that account for 39% of the total credits. In other words, there were roughly four times as many "purely sponsored" credits as there were "purely volunteer" credits.
When we looked at the 5,196 users, rather than credits, we found somewhat different results. A similar percentage of all users had exclusively volunteer credits: 14% (741 users). But the percentage of users with exclusively sponsored credits is only 50% higher: 21% (1077 users). Thus, when we look at the data this way, we find that users who only do sponsored work tend to contribute quite a bit more than users who only do volunteer work.
None of these methodologies are perfect, but they all point to a conclusion that most of the work on Drupal is sponsored. At the same time, the data shows that volunteer contribution remains very important to Drupal. We believe there is a healthy ratio between sponsored and volunteer contributions.
Who is sponsoring the work?
Because we established that most of the work on Drupal is sponsored, we know it is important to track and study what organizations contribute to Drupal. Despite 659 different organizations contributing to Drupal, approximately 50% of them got 4 credits or less. The top 30 organizations (roughly top 5%) account for about 29% of the total credits, which suggests that the top 30 companies play a crucial role in the health of the Drupal project. The graph below shows the top 30 organizations and the number of credits they received between July 1, 2015 and June 30, 2016:
While not immediately obvious from the graph above, different types of companies are active in Drupal's ecosystem and we propose the following categorization below to discuss our ecosystem.
|Traditional Drupal businesses||Small-to-medium-sized professional services companies that make money primarily using Drupal. They typically employ less than 100 employees, and because they specialize in Drupal, many of these professional services companies contribute frequently and are a huge part of our community. Examples are Lullabot (shown on graph) or Chapter Three (shown on graph).|
|Digital marketing agencies||Larger full-service agencies that have marketing led practices using a variety of tools, typically including Drupal, Adobe Experience Manager, Sitecore, WordPress, etc. They are typically larger, with the larger agencies employing thousands of people. Examples are Sapient (shown on graph) or AKQA.|
|System integrators||Larger companies that specialize in bringing together different technologies into one solution. Example system agencies are Accenture, TATA Consultancy Services, Capgemini or CI&T.|
|Technology and infrastructure companies||Examples are Acquia (shown on graph), Lingotek (shown on graph), BlackMesh, RackSpace, Pantheon or Platform.sh.|
|End-users||Examples are Pfizer (shown on graph), Examiner.com (shown on graph) or NBC Universal.|
Most of the top 30 sponsors are traditional Drupal companies. Sapient (120 credits) is the only digital marketing agency showing up in the top 30. No system integrator shows up in the top 30. The first system integrator is CI&T, which ranked 31st with 102 credits. As far as system integrators are concerned CI&T is a smaller player with between 1,000 and 5,000 employees. Other system integrators with credits are Capgemini (43 credits), Globant (26 credits), and TATA Consultancy Services (7 credits). We didn't see any code contributions from Accenture, Wipro or IBM Global Services. We expect these will come as most of them are building out Drupal practices. For example, we know that IBM Global Services already has over 100 people doing Drupal work.
When we look beyond the top 30 sponsors, we see that roughly 82% of the code contribution on Drupal.org comes from the traditional Drupal businesses. About 13% of the contributions comes from infrastructure and software companies, though that category is mostly dominated by one company, Acquia. This means that the technology and infrastructure companies, digital marketing agencies, system integrators and end-users are not meaningfully contributing code to Drupal.org today. In an ideal world, the pie chart above would be sliced in equal sized parts.
How can we explain that unbalance? We believe the two biggest reasons are: (1) Drupal's strategic importance and (2) the level of maturity with Drupal and Open Source. Various of the traditional Drupal agencies have been involved with Drupal for 10 years and almost entirely depend on on Drupal. Given both their expertise and dependence on Drupal, they are most likely to look after Drupal's development and well-being. These organizations are typically recognized as Drupal experts and sought out by organizations that want to build a Drupal website. Contrast this with most of the digital marketing agencies and system integrators who have the size to work with a diversified portfolio of content management platforms, and are just getting started with Drupal and Open Source. They deliver digital marketing solutions and aren't necessarily sought out for their Drupal expertise. As their Drupal practices grow in size and importance, this could change, and when it does, we expect them to contribute more. Right now many of the digital marketing agencies and system integrators have little or no experience with Open Source so it is important that we motivate them to contribute and then teach them how to contribute.
There are two main business reasons for organizations to contribute: (1) it improves their ability to sell and win deals and (2) it improves their ability to hire. Companies that contribute to Drupal tend to promote their contributions in RFPs and sales pitches to win more deals. Contributing to Drupal also results in being recognized as a great place to work for Drupal experts.
We also should note that many organizations in the Drupal community contribute for reasons that would not seem to be explicitly economically motivated. More than 100 credits were sponsored by colleges or universities, such as the University of Waterloo (45 credits). More than 50 credits came from community groups, such as the Drupal Bangalore Community and the Drupal Ukraine Community. Other nonprofits and government organization that appeared in our data include the Drupal Association (166), National Virtual Library of India (25 credits), Center for Research Libraries (20), and Welsh Government (9 credits).
Infrastructure and software companies
Infrastructure and software companies play a different role in our community. These companies are less reliant on professional services (building Drupal websites) and primarily make money from selling subscription based products.
Acquia, Pantheon and Platform.sh are venture-backed Platform-as-a-Service companies born out of the Drupal community. Rackspace and AWS are public companies hosting thousands of Drupal sites each. Lingotek offers cloud-based translation management software for Drupal.
The graph above suggests that Pantheon and Platform.sh have barely contributed code on Drupal.org during the past year. (Platform.sh only became an independent company 6 months ago after they split off from CommerceGuys.) The chart also does not reflect sponsored code contributions on GitHub (such as drush), Drupal event sponsorship, and the wide variety of value that these companies add to Drupal and other Open Source communities.
Consequently, these data show that the Drupal community needs to do a better job of enticing infrastructure and software companies to contribute code to Drupal.org. The Drupal community has a long tradition of encouraging organizations to share code on Drupal.org rather than keep it behind firewalls. While the spirit of the Drupal project cannot be reduced to any single ideology-- not every organization can or will share their code -- we would like to see organizations continue to prioritize collaboration over individual ownership. Our aim is not to criticize those who do not contribute, but rather to help foster an environment worthy of contribution.
We saw two end-users in the top 30 corporate sponsors: Pfizer (158 credits) and Examiner.com (132 credits). Other notable end-users that are actively giving back are Workday (52 credits), NBC Universal (40 credits), the University of Waterloo (45 credits) and CARD.com (33 credits). The end users that tend to contribute to Drupal use Drupal for a key part of their business and often have an internal team of Drupal developers.
Given that there are hundreds of thousands of Drupal end-users, we would like to see more end-users in the top 30 sponsors. We recognize that a lot of digital agencies don't want, or are not legally allowed, to attribute their customers. We hope that will change as Open Source continues to get more and more adopted.
Given the vast amount of Drupal users, we believe encouraging end-users to contribute could be a big opportunity. Being credited on Drupal.org gives them visibility in the Drupal community and recognizes them as a great place for Open Source developers to work.
The uneasy alliance with corporate contributions
As mentioned above, when community-driven Open Source projects grow, there becomes a bigger need for organizations to help drive its development. It almost always creates an uneasy alliance between volunteers and corporations.
This theory played out in the Linux community well before it played out in the Drupal community. The Linux project is 25 years old now has seen a steady increase in the number of corporate contributors for roughly 20 years. While Linux companies like Red Hat and SUSE rank highly on the contribution list, so do non-Linux-centric companies such as Samsung, Intel, Oracle and Google. The major theme in this story is that all of these corporate contributors were using Linux as an integral part of their business.
The 659 organizations that contribute to Drupal (which includes corporations), is roughly three times the number of organizations that sponsor development of the Linux kernel, "one of the largest cooperative software projects ever attempted". In fairness, Linux has a different ecosystem than Drupal. The Linux business ecosystem has various large organizations (Red Hat, Google, Intel, IBM and SUSE) for whom Linux is very strategic. As a result, many of them employ dozens of full-time Linux contributors and invest millions of dollars in Linux each year.
In the Drupal community, Acquia has had people dedicated full-time to Drupal starting nine years ago when it hired Gábor Hojtsy to contribute to Drupal core full-time. Today, Acquia has about 10 developers contributing to Drupal full-time. They work on core, contributed modules, security, user experience, performance, best practices, and more. Their work has benefited untold numbers of people around the world, most of whom are not Acquia customers.
In response to Acquia’s high level of participation in the Drupal project, as well as to the number of Acquia employees that hold leadership positions, some members of the Drupal community have suggested that Acquia wields its influence and power to control the future of Drupal for its own commercial benefit. But neither of us believe that Acquia should contribute less. Instead, we would like to see more companies provide more leadership to Drupal and meaningfully contribute on Drupal.org.
Who is sponsoring the top 30 contributors?
|1||dawehner||560||84.1%||77.7%||9.5%||Drupal Association (182), Chapter Three (179), Tag1 Consulting (160), Cando (6), Acquia (4), Comm-press (1)|
|3||alexpott||409||0.2%||97.8%||2.2%||Chapter Three (400)|
|4||Berdir||383||0.0%||95.3%||4.7%||MD Systems (365), Acquia (9)|
|5||Wim Leers||382||31.7%||98.2%||1.8%||Acquia (375)|
|6||jhodgdon||381||5.2%||3.4%||91.3%||Drupal Association (13), Poplar ProductivityWare (13)|
|7||joelpittet||294||23.8%||1.4%||76.2%||Drupal Association (4)|
|8||heykarthikwithu||293||99.3%||100.0%||0.0%||Valuebound (293), Drupal Bangalore Community (3)|
|9||mglaman||292||9.6%||96.9%||0.7%||Commerce Guys (257), Bluehorn Digital (14), Gaggle.net, Inc. (12), LivePerson, Inc (11), Bluespark (5), DPCI (3), Thinkbean, LLC (3), Digital Bridge Solutions (2), Matsmart (1)|
|10||drunken monkey||248||75.4%||55.6%||2.0%||Acquia (72), StudentFirst (44), epiqo (12), Vizala (9), Sunlime IT Services GmbH (1)|
|11||Sam152||237||75.9%||89.5%||10.1%||PreviousNext (210), Code Drop (2)|
|12||borisson_||207||62.8%||36.2%||15.9%||Acquia (67), Intracto digital agency (8)|
|13||benjy||206||0.0%||98.1%||1.9%||PreviousNext (168), Code Drop (34)|
|14||edurenye||184||0.0%||100.0%||0.0%||MD Systems (184)|
|15||catch||180||3.3%||44.4%||54.4%||Third and Grove (44), Tag1 Consulting (36), Drupal Association (4)|
|16||slashrsm||179||12.8%||96.6%||2.8%||Examiner.com (89), MD Systems (84), Acquia (18), Studio Matris (1)|
|18||mbovan||174||7.5%||100.0%||0.0%||MD Systems (118), ACTO Team (43), Google Summer of Code (13)|
|20||rakesh.gectcr||163||100.0%||100.0%||0.0%||Valuebound (138), National Virtual Library of India (NVLI) (25)|
|23||mikeryan||150||0.0%||89.3%||10.7%||Acquia (112), Virtuoso Performance (22), Drupalize.Me (4), North Studio (4)|
|24||jhedstrom||149||0.0%||83.2%||16.8%||Phase2 (124), Workday, Inc. (36), Memorial Sloan Kettering Cancer Center (4)|
|27||stefan.r||146||0.7%||0.7%||98.6%||Drupal Association (1)|
|28||bojanz||145||2.1%||83.4%||15.2%||Commerce Guys (121), Bluespark (2)|
|29||penyaskito||141||6.4%||95.0%||3.5%||Lingotek (129), Cocomore AG (5)|
|30||larowlan||135||34.1%||63.0%||16.3%||PreviousNext (85), Department of Justice & Regulation, Victoria (14), amaysim Australia Ltd. (1), University of Adelaide (1)|
We observe that the top 30 contributors are sponsored by 45 organizations. This kind of diversity is aligned with our desire not to see Drupal controlled by a single organization. The top 30 contributors and the 45 organizations are from many different parts in the world and work with customers large or small. We could still benefit from more diversity, though. The top 30 lacks digital marketing agencies, large system integrators and end-users -- all of whom could contribute meaningfully to making Drupal for them and others.
Evolving the credit system
The credit system gives us quantifiable data about where our community's contributions come from, but that data is not perfect. Here are a few suggested improvements:
- We need to find ways to recognize non-code contributions as well as code contributions outside of Drupal.org (i.e. on GitHub). Lots of people and organizations spend hundreds of hours putting together local events, writing documentation, translating Drupal, mentoring new contributors, and more—and none of that gets captured by the credit system.
- We'd benefit by finding a way to account for the complexity and quality of contributions; one person might have worked several weeks for just one credit, while another person might have gotten a credit for 30 minutes of work. We could, for example, consider the issue credit data in conjunction with Git commit data regarding insertions, deletions, and files changed.
- We could try to leverage the credit system to encourage more companies, especially those that do not contribute today, to participate in large-scale initiatives. Dries presented some ideas two years ago in his DrupalCon Amsterdam keynote and Matthew has suggested other ideas, but we are open to more suggestions on how we might bring more contributors into the fold using the credit system.
- We could segment out organization profiles between end users and different kinds of service providers. Doing so would make it easier to see who the top contributors are in each segment and perhaps foster more healthy competition among peers. In turn, the community could learn about the peculiar motivations within each segment.
Like Drupal the software, the credit system on Drupal.org is a tool that can evolve, but that ultimately will only be useful when the community uses it, understands its shortcomings, and suggests constructive improvements. In highlighting the organizations that sponsor work on Drupal.org, we hope to provoke responses that help evolve the credit system into something that incentivizes business to sponsor more work and that allows more people the opportunity to participate in our community, learn from others, teach newcomers, and make positive contributions. We view Drupal as a productive force for change and we wish to use the credit system to highlight (at least some of) the work of our diverse community of volunteers, companies, nonprofits, governments, schools, universities, individuals, and other groups.
Our data shows that Drupal is a vibrant and diverse community, with thousands of contributors, that is constantly evolving and improving the software. While here we have examined issue credits mostly through the lens of sponsorship, in future analyses we plan to consider the same issue credits in conjunction with other publicly-disclosed Drupal user data, such as gender identification, geography, seasonal participation, mentorship, and event attendance.
Our analysis of the Drupal.org credit data concludes that most of the contributions to Drupal are sponsored. At the same time, the data shows that volunteer contribution remains very important to Drupal.
As a community, we need to understand that a healthy Open Source ecosystem is a diverse ecosystem that includes more than traditional Drupal agencies. The traditional Drupal agencies and Acquia contribute the most but we don't see a lot of contribution from the larger digital marketing agencies, system integrators, technology companies, or end-users of Drupal—we believe that might come as these organizations build out their Drupal practices and Drupal becomes more strategic for them.
To grow and sustain Drupal, we should support those that contribute to Drupal, and find ways to get those that are not contributing involved in our community. We invite you to help us figure out how we can continue to strengthen our ecosystem.
We hope to repeat this work in 1 or 2 years' time so we can track our evolution. Special thanks to Tim Lehnen (Drupal Association) for providing us the credit system data and supporting us during our research.
Over the weekend, Drupal 8.2 beta was released. One of the reasons why I'm so excited about this release is that it ships with "more outside-in". In an "outside-in experience", you can click anything on the page, edit its configuration in place without having to navigate to the administration back end, and watch it take effect immediately. This kind of on-the-fly editorial experience could be a game changer for Drupal's usability.
When I last discussed turning Drupal outside-in, we were still in the conceptual stages, with mockups illustrating the concepts. Since then, those designs have gone through multiple rounds of feedback from Drupal's usability team and a round of user testing led by Cheppers. This study identified some issues and provided some insights which were incorporated into subsequent designs.
Two policy changes we introduced in Drupal 8 — semantic versioning and experimental modules — have fundamentally changed Drupal's innovation model starting with Drupal 8. I should write a longer blog post about this, but the net result of those two changes is ongoing improvements with an easy upgrade path. In this case, it enabled us to add outside-in experiences to Drupal 8.2 instead of having to wait for Drupal 9. The authoring experience improvements we made in Drupal 8 are well-received, but that doesn't mean we are done. It's exciting that we can move much faster on making Drupal easier to use.
In-place block configuration
As you can see from the image below, Drupal 8.2 adds the ability to trigger "Edit" mode, which currently highlights all blocks on the page. Clicking on one — in this case, the block with the site's name — pops out a new tray or sidebar. A content creator can change the site name directly from the tray, without having to navigate through Drupal's administrative interface to theme settings as they would have to in Drupal 7 and Drupal 8.1.
Making adjustments to menus
In the second image, the pattern is applied to a menu block. You can make adjustments to the menu right from the new tray instead of having to navigate to the back end. Here the content creator changes the order of the menu links (moving "About us" after "Contact") and toggles the "Team" menu item from hidden to visible.
In-context block placement
In Drupal 8.1 and prior, placing a new block on the page required navigating away from your front end into the administrative back end and noting the available regions. Once you discover where to go to add a block, which can in itself be a challenge, you'll have to learn about the different regions, and some trial and error might be required to place a block exactly where you want it to go.
Starting in Drupal 8.2, content creators can now just click "Place block" without navigating to a different page and knowing about available regions ahead of time. Clicking "Place block" will highlight the different possible locations for a block to be placed in.
These improvements are currently tagged "experimental". This means that anyone who downloads Drupal 8.2 can test these changes and provide feedback. It also means that we aren't quite satisfied with these changes yet and that you should expect to see this functionality improve between now and 8.2.0's release, and even after the Drupal 8.2.0 release.
As you probably noticed, things still look pretty raw in places; as an example, the forms in the tray are exposing too many visual details. There is more work to do to bring this functionality to the level of the designs. We're focused on improving that, as well as the underlying architecture and accessibility. Once we feel good about how it all works and looks, we'll remove the experimental label.
We deliberately postponed most of the design work to focus on introducing the fundamental concepts and patterns. That was an important first step. We wanted to enable Drupal developers to start experimenting with the outside-in pattern in Drupal 8.2. As part of that, we'll have to determine how this new pattern will apply broadly to Drupal core and the many contributed modules that would leverage it. Our hope is that once the outside-in work is stable and no longer experimental, it will trickle down to every Drupal module. At that point we can all work together, in parallel, on making Drupal much easier to use.
Users have proven time and again in usability studies to be extremely "preview-driven", so the ability to make quick configuration changes right from their front end, without becoming an expert in Drupal's information architecture, could be revolutionary for Drupal.
If you'd like to help get these features to stable release faster, please join us in the outside-in roadmap issue.
I'd also like to thank everyone who contributed to these features and reviewed them, including Bojhan, yoroy, pwolanin, andrewmacpherson, gtamas, petycomp, zsofimajor, SKAUGHT, nod_, effulgentsia, Wim Leers, catch, alexpott, and xjm.