Who sponsors Drupal development?

There exist millions of Open Source projects today, but many of them aren't sustainable. Scaling Open Source projects in a sustainable manner is difficult. A prime example is OpenSSL, which plays a critical role in securing the internet. Despite its importance, the entire OpenSSL development team is relatively small, consisting of 11 people, 10 of whom are volunteers. In 2014, security researchers discovered an important security bug that exposed millions of websites. Like OpenSSL, most Open Source projects fail to scale their resources. Notable exceptions are the Linux kernel, Debian, Apache, Drupal, and WordPress, which have foundations, multiple corporate sponsors and many contributors that help these projects scale.

We (Dries Buytaert is the founder and project lead of Drupal and co-founder and Chief Technology Officer of Acquia and Matthew Tift is a Senior Developer at Lullabot and Drupal 8 configuration system co-maintainer) believe that the Drupal community has a shared responsibility to build Drupal and that those who get more from Drupal should consider giving more. We examined commit data to help understand who develops Drupal, how much of that work is sponsored, and where that sponsorship comes from. We will illustrate that the Drupal community is far ahead in understanding how to sustain and scale the project. We will show that the Drupal project is a healthy project with a diverse community of contributors. Nevertheless, in Drupal's spirit of always striving to do better, we will also highlight areas where our community can and should do better.

Who is working on Drupal?

In the spring of 2015, after proposing ideas about giving credit and discussing various approaches at length, Drupal.org added the ability for people to attribute their work to an organization or customer in the Drupal.org issue queues. Maintainers of Drupal themes and modules can award issues credits to people who help resolve issues with code, comments, design, and more.

Example issue credit on drupal org
A screenshot of an issue comment on Drupal.org. You can see that jamadar worked on this patch as a volunteer, but also as part of his day job working for TATA Consultancy Services on behalf of their customer, Pfizer.

Drupal.org's credit system captures all the issue activity on Drupal.org. This is primarily code contributions, but also includes some (but not all) of the work on design, translations, documentation, etc. It is important to note that contributing in the issues on Drupal.org is not the only way to contribute. There are other activities—for instance, sponsoring events, promoting Drupal, providing help and mentoring—important to the long-term health of the Drupal project. These activities are not currently captured by the credit system. Additionally, we acknowledge that parts of Drupal are developed on GitHub and that credits might get lost when those contributions are moved to Drupal.org. For the purposes of this post, however, we looked only at the issue contributions captured by the credit system on Drupal.org.

What we learned is that in the 12-month period from July 1, 2015 to June 30, 2016 there were 32,711 issue credits—both to Drupal core as well as all the contributed themes and modules—attributed to 5,196 different individual contributors and 659 different organizations.

Despite the large number of individual contributors, a relatively small number do the majority of the work. Approximately 51% of the contributors involved got just one credit. The top 30 contributors (or top 0.5% contributors) account for over 21% of the total credits, indicating that these individuals put an incredible amount of time and effort in developing Drupal and its contributed modules:

RankUsernameIssues
1dawehner560
2DamienMcKenna448
3alexpott409
4Berdir383
5Wim Leers382
6jhodgdon381
7joelpittet294
8heykarthikwithu293
9mglaman292
10drunken monkey248
11Sam152237
12borisson_207
13benjy206
14edurenye184
15catch180
16slashrsm179
17phenaproxima177
18mbovan174
19tim.plunkett168
20rakesh.gectcr163
21martin107163
22dsnopek152
23mikeryan150
24jhedstrom149
25xjm147
26hussainweb147
27stefan.r146
28bojanz145
29penyaskito141
30larowlan135

How much of the work is sponsored?

As mentioned above, from July 1, 2015 to June 30, 2016, 659 organizations contributed code to Drupal.org. Drupal is used by more than one million websites. The vast majority of the organizations behind these Drupal websites never participate in the development of Drupal; they use the software as it is and do not feel the need to help drive its development.

Technically, Drupal started out as a 100% volunteer-driven project. But nowadays, the data suggests that the majority of the code on Drupal.org is sponsored by organizations in Drupal's ecosystem. For example, of the 32,711 commit credits we studied, 69% of the credited work is "sponsored".

We then looked at the distribution of how many of the credits are given to volunteers versus given to individuals doing "sponsored work" (i.e. contributing as part of their paid job):

Contributions top range

Looking at the top 100 contributors, for example, 23% of their credits are the result of contributing as volunteers and 56% of their credits are attributed to a corporate sponsor. The remainder, roughly 21% of the credits, are not attributed. Attribution is optional so this means it could either be volunteer-driven, sponsored, or both.

As can be seen on the graph, the ratio of volunteer versus sponsored don't meaningfully change as we look beyond the top 100—the only thing that changes is that more credits that are not attributed. This might be explained by the fact that occasional contributors might not be aware of or understand the credit system, or could not be bothered with setting up organizational profiles for their employer or customers.

As shown in jamadar's screenshot above, a credit can be marked as volunteer and sponsored at the same time. This could be the case when someone does the minimum required work to satisfy the customer's need, but uses his or her spare time to add extra functionality. We can also look at the amount of code credits that are exclusively volunteer credits. Of the 7,874 credits that marked volunteer, 43% of them (3,376 credits) only had the volunteer box checked and 57% of them (4,498) were also partially sponsored. These 3,376 credits are one of our best metrics to measure volunteer-only contributions. This suggests that only 10% of the 32,711 commit credits we examined were contributed exclusively by volunteers. This number is a stark contrast to the 12,888 credits that were "purely sponsored", and that account for 39% of the total credits. In other words, there were roughly four times as many "purely sponsored" credits as there were "purely volunteer" credits.

When we looked at the 5,196 users, rather than credits, we found somewhat different results. A similar percentage of all users had exclusively volunteer credits: 14% (741 users). But the percentage of users with exclusively sponsored credits is only 50% higher: 21% (1077 users). Thus, when we look at the data this way, we find that users who only do sponsored work tend to contribute quite a bit more than users who only do volunteer work.

None of these methodologies are perfect, but they all point to a conclusion that most of the work on Drupal is sponsored. At the same time, the data shows that volunteer contribution remains very important to Drupal. We believe there is a healthy ratio between sponsored and volunteer contributions.

Who is sponsoring the work?

Because we established that most of the work on Drupal is sponsored, we know it is important to track and study what organizations contribute to Drupal. Despite 659 different organizations contributing to Drupal, approximately 50% of them got 4 credits or less. The top 30 organizations (roughly top 5%) account for about 29% of the total credits, which suggests that the top 30 companies play a crucial role in the health of the Drupal project. The graph below shows the top 30 organizations and the number of credits they received between July 1, 2015 and June 30, 2016:

Contributions top organizations

While not immediately obvious from the graph above, different types of companies are active in Drupal's ecosystem and we propose the following categorization below to discuss our ecosystem.

Category Description
Traditional Drupal businesses Small-to-medium-sized professional services companies that make money primarily using Drupal. They typically employ less than 100 employees, and because they specialize in Drupal, many of these professional services companies contribute frequently and are a huge part of our community. Examples are Lullabot (shown on graph) or Chapter Three (shown on graph).
Digital marketing agencies Larger full-service agencies that have marketing led practices using a variety of tools, typically including Drupal, Adobe Experience Manager, Sitecore, WordPress, etc. They are typically larger, with the larger agencies employing thousands of people. Examples are Sapient (shown on graph) or AKQA.
System integrators Larger companies that specialize in bringing together different technologies into one solution. Example system agencies are Accenture, TATA Consultancy Services, Capgemini or CI&T.
Technology and infrastructure companies Examples are Acquia (shown on graph), Lingotek (shown on graph), BlackMesh, RackSpace, Pantheon or Platform.sh.
End-users Examples are Pfizer (shown on graph), Examiner.com (shown on graph) or NBC Universal.

Most of the top 30 sponsors are traditional Drupal companies. Sapient (120 credits) is the only digital marketing agency showing up in the top 30. No system integrator shows up in the top 30. The first system integrator is CI&T, which ranked 31st with 102 credits. As far as system integrators are concerned CI&T is a smaller player with between 1,000 and 5,000 employees. Other system integrators with credits are Capgemini (43 credits), Globant (26 credits), and TATA Consultancy Services (7 credits). We didn't see any code contributions from Accenture, Wipro or IBM Global Services. We expect these will come as most of them are building out Drupal practices. For example, we know that IBM Global Services already has over 100 people doing Drupal work.

Contributions by organization type

When we look beyond the top 30 sponsors, we see that roughly 82% of the code contribution on Drupal.org comes from the traditional Drupal businesses. About 13% of the contributions comes from infrastructure and software companies, though that category is mostly dominated by one company, Acquia. This means that the technology and infrastructure companies, digital marketing agencies, system integrators and end-users are not meaningfully contributing code to Drupal.org today. In an ideal world, the pie chart above would be sliced in equal sized parts.

How can we explain that unbalance? We believe the two biggest reasons are: (1) Drupal's strategic importance and (2) the level of maturity with Drupal and Open Source. Various of the traditional Drupal agencies have been involved with Drupal for 10 years and almost entirely depend on on Drupal. Given both their expertise and dependence on Drupal, they are most likely to look after Drupal's development and well-being. These organizations are typically recognized as Drupal experts and sought out by organizations that want to build a Drupal website. Contrast this with most of the digital marketing agencies and system integrators who have the size to work with a diversified portfolio of content management platforms, and are just getting started with Drupal and Open Source. They deliver digital marketing solutions and aren't necessarily sought out for their Drupal expertise. As their Drupal practices grow in size and importance, this could change, and when it does, we expect them to contribute more. Right now many of the digital marketing agencies and system integrators have little or no experience with Open Source so it is important that we motivate them to contribute and then teach them how to contribute.

There are two main business reasons for organizations to contribute: (1) it improves their ability to sell and win deals and (2) it improves their ability to hire. Companies that contribute to Drupal tend to promote their contributions in RFPs and sales pitches to win more deals. Contributing to Drupal also results in being recognized as a great place to work for Drupal experts.

We also should note that many organizations in the Drupal community contribute for reasons that would not seem to be explicitly economically motivated. More than 100 credits were sponsored by colleges or universities, such as the University of Waterloo (45 credits). More than 50 credits came from community groups, such as the Drupal Bangalore Community and the Drupal Ukraine Community. Other nonprofits and government organization that appeared in our data include the Drupal Association (166), National Virtual Library of India (25 credits), Center for Research Libraries (20), and Welsh Government (9 credits).

Infrastructure and software companies

Infrastructure and software companies play a different role in our community. These companies are less reliant on professional services (building Drupal websites) and primarily make money from selling subscription based products.

Acquia, Pantheon and Platform.sh are venture-backed Platform-as-a-Service companies born out of the Drupal community. Rackspace and AWS are public companies hosting thousands of Drupal sites each. Lingotek offers cloud-based translation management software for Drupal.

Contributions by technology companies

The graph above suggests that Pantheon and Platform.sh have barely contributed code on Drupal.org during the past year. (Platform.sh only became an independent company 6 months ago after they split off from CommerceGuys.) The chart also does not reflect sponsored code contributions on GitHub (such as drush), Drupal event sponsorship, and the wide variety of value that these companies add to Drupal and other Open Source communities.

Consequently, these data show that the Drupal community needs to do a better job of enticing infrastructure and software companies to contribute code to Drupal.org. The Drupal community has a long tradition of encouraging organizations to share code on Drupal.org rather than keep it behind firewalls. While the spirit of the Drupal project cannot be reduced to any single ideology-- not every organization can or will share their code -- we would like to see organizations continue to prioritize collaboration over individual ownership. Our aim is not to criticize those who do not contribute, but rather to help foster an environment worthy of contribution.

End users

We saw two end-users in the top 30 corporate sponsors: Pfizer (158 credits) and Examiner.com (132 credits). Other notable end-users that are actively giving back are Workday (52 credits), NBC Universal (40 credits), the University of Waterloo (45 credits) and CARD.com (33 credits). The end users that tend to contribute to Drupal use Drupal for a key part of their business and often have an internal team of Drupal developers.

Given that there are hundreds of thousands of Drupal end-users, we would like to see more end-users in the top 30 sponsors. We recognize that a lot of digital agencies don't want, or are not legally allowed, to attribute their customers. We hope that will change as Open Source continues to get more and more adopted.

Given the vast amount of Drupal users, we believe encouraging end-users to contribute could be a big opportunity. Being credited on Drupal.org gives them visibility in the Drupal community and recognizes them as a great place for Open Source developers to work.

The uneasy alliance with corporate contributions

As mentioned above, when community-driven Open Source projects grow, there becomes a bigger need for organizations to help drive its development. It almost always creates an uneasy alliance between volunteers and corporations.

This theory played out in the Linux community well before it played out in the Drupal community. The Linux project is 25 years old now has seen a steady increase in the number of corporate contributors for roughly 20 years. While Linux companies like Red Hat and SUSE rank highly on the contribution list, so do non-Linux-centric companies such as Samsung, Intel, Oracle and Google. The major theme in this story is that all of these corporate contributors were using Linux as an integral part of their business.

The 659 organizations that contribute to Drupal (which includes corporations), is roughly three times the number of organizations that sponsor development of the Linux kernel, "one of the largest cooperative software projects ever attempted". In fairness, Linux has a different ecosystem than Drupal. The Linux business ecosystem has various large organizations (Red Hat, Google, Intel, IBM and SUSE) for whom Linux is very strategic. As a result, many of them employ dozens of full-time Linux contributors and invest millions of dollars in Linux each year.

In the Drupal community, Acquia has had people dedicated full-time to Drupal starting nine years ago when it hired Gábor Hojtsy to contribute to Drupal core full-time. Today, Acquia has about 10 developers contributing to Drupal full-time. They work on core, contributed modules, security, user experience, performance, best practices, and more. Their work has benefited untold numbers of people around the world, most of whom are not Acquia customers.

In response to Acquia’s high level of participation in the Drupal project, as well as to the number of Acquia employees that hold leadership positions, some members of the Drupal community have suggested that Acquia wields its influence and power to control the future of Drupal for its own commercial benefit. But neither of us believe that Acquia should contribute less. Instead, we would like to see more companies provide more leadership to Drupal and meaningfully contribute on Drupal.org.

Who is sponsoring the top 30 contributors?

Rank Username Issues Volunteer Sponsored Not specified Sponsors
1 dawehner 560 84.1% 77.7% 9.5% Drupal Association (182), Chapter Three (179), Tag1 Consulting (160), Cando (6), Acquia (4), Comm-press (1)
2 DamienMcKenna 448 6.9% 76.3% 19.4% Mediacurrent (342)
3 alexpott 409 0.2% 97.8% 2.2% Chapter Three (400)
4 Berdir 383 0.0% 95.3% 4.7% MD Systems (365), Acquia (9)
5 Wim Leers 382 31.7% 98.2% 1.8% Acquia (375)
6 jhodgdon 381 5.2% 3.4% 91.3% Drupal Association (13), Poplar ProductivityWare (13)
7 joelpittet 294 23.8% 1.4% 76.2% Drupal Association (4)
8 heykarthikwithu 293 99.3% 100.0% 0.0% Valuebound (293), Drupal Bangalore Community (3)
9 mglaman 292 9.6% 96.9% 0.7% Commerce Guys (257), Bluehorn Digital (14), Gaggle.net, Inc. (12), LivePerson, Inc (11), Bluespark (5), DPCI (3), Thinkbean, LLC (3), Digital Bridge Solutions (2), Matsmart (1)
10 drunken monkey 248 75.4% 55.6% 2.0% Acquia (72), StudentFirst (44), epiqo (12), Vizala (9), Sunlime IT Services GmbH (1)
11 Sam152 237 75.9% 89.5% 10.1% PreviousNext (210), Code Drop (2)
12 borisson_ 207 62.8% 36.2% 15.9% Acquia (67), Intracto digital agency (8)
13 benjy 206 0.0% 98.1% 1.9% PreviousNext (168), Code Drop (34)
14 edurenye 184 0.0% 100.0% 0.0% MD Systems (184)
15 catch 180 3.3% 44.4% 54.4% Third and Grove (44), Tag1 Consulting (36), Drupal Association (4)
16 slashrsm 179 12.8% 96.6% 2.8% Examiner.com (89), MD Systems (84), Acquia (18), Studio Matris (1)
17 phenaproxima 177 0.0% 94.4% 5.6% Acquia (167)
18 mbovan 174 7.5% 100.0% 0.0% MD Systems (118), ACTO Team (43), Google Summer of Code (13)
19 tim.plunkett 168 14.3% 89.9% 10.1% Acquia (151)
20 rakesh.gectcr 163 100.0% 100.0% 0.0% Valuebound (138), National Virtual Library of India (NVLI) (25)
21 martin107 163 4.9% 0.0% 95.1%
22 dsnopek 152 0.7% 0.0% 99.3%
23 mikeryan 150 0.0% 89.3% 10.7% Acquia (112), Virtuoso Performance (22), Drupalize.Me (4), North Studio (4)
24 jhedstrom 149 0.0% 83.2% 16.8% Phase2 (124), Workday, Inc. (36), Memorial Sloan Kettering Cancer Center (4)
25 xjm 147 0.0% 81.0% 19.0% Acquia (119)
26 hussainweb 147 2.0% 98.6% 1.4% Axelerant (145)
27 stefan.r 146 0.7% 0.7% 98.6% Drupal Association (1)
28 bojanz 145 2.1% 83.4% 15.2% Commerce Guys (121), Bluespark (2)
29 penyaskito 141 6.4% 95.0% 3.5% Lingotek (129), Cocomore AG (5)
30 larowlan 135 34.1% 63.0% 16.3% PreviousNext (85), Department of Justice & Regulation, Victoria (14), amaysim Australia Ltd. (1), University of Adelaide (1)

We observe that the top 30 contributors are sponsored by 45 organizations. This kind of diversity is aligned with our desire not to see Drupal controlled by a single organization. The top 30 contributors and the 45 organizations are from many different parts in the world and work with customers large or small. We could still benefit from more diversity, though. The top 30 lacks digital marketing agencies, large system integrators and end-users -- all of whom could contribute meaningfully to making Drupal for them and others.

Evolving the credit system

The credit system gives us quantifiable data about where our community's contributions come from, but that data is not perfect. Here are a few suggested improvements:

  1. We need to find ways to recognize non-code contributions as well as code contributions outside of Drupal.org (i.e. on GitHub). Lots of people and organizations spend hundreds of hours putting together local events, writing documentation, translating Drupal, mentoring new contributors, and more—and none of that gets captured by the credit system.
  2. We'd benefit by finding a way to account for the complexity and quality of contributions; one person might have worked several weeks for just one credit, while another person might have gotten a credit for 30 minutes of work. We could, for example, consider the issue credit data in conjunction with Git commit data regarding insertions, deletions, and files changed.
  3. We could try to leverage the credit system to encourage more companies, especially those that do not contribute today, to participate in large-scale initiatives. Dries presented some ideas two years ago in his DrupalCon Amsterdam keynote and Matthew has suggested other ideas, but we are open to more suggestions on how we might bring more contributors into the fold using the credit system.
  4. We could segment out organization profiles between end users and different kinds of service providers. Doing so would make it easier to see who the top contributors are in each segment and perhaps foster more healthy competition among peers. In turn, the community could learn about the peculiar motivations within each segment.

Like Drupal the software, the credit system on Drupal.org is a tool that can evolve, but that ultimately will only be useful when the community uses it, understands its shortcomings, and suggests constructive improvements. In highlighting the organizations that sponsor work on Drupal.org, we hope to provoke responses that help evolve the credit system into something that incentivizes business to sponsor more work and that allows more people the opportunity to participate in our community, learn from others, teach newcomers, and make positive contributions. We view Drupal as a productive force for change and we wish to use the credit system to highlight (at least some of) the work of our diverse community of volunteers, companies, nonprofits, governments, schools, universities, individuals, and other groups.

Conclusion

Our data shows that Drupal is a vibrant and diverse community, with thousands of contributors, that is constantly evolving and improving the software. While here we have examined issue credits mostly through the lens of sponsorship, in future analyses we plan to consider the same issue credits in conjunction with other publicly-disclosed Drupal user data, such as gender identification, geography, seasonal participation, mentorship, and event attendance.

Our analysis of the Drupal.org credit data concludes that most of the contributions to Drupal are sponsored. At the same time, the data shows that volunteer contribution remains very important to Drupal.

As a community, we need to understand that a healthy Open Source ecosystem is a diverse ecosystem that includes more than traditional Drupal agencies. The traditional Drupal agencies and Acquia contribute the most but we don't see a lot of contribution from the larger digital marketing agencies, system integrators, technology companies, or end-users of Drupal—we believe that might come as these organizations build out their Drupal practices and Drupal becomes more strategic for them.

To grow and sustain Drupal, we should support those that contribute to Drupal, and find ways to get those that are not contributing involved in our community. We invite you to help us figure out how we can continue to strengthen our ecosystem.

We hope to repeat this work in 1 or 2 years' time so we can track our evolution. Special thanks to Tim Lehnen (Drupal Association) for providing us the credit system data and supporting us during our research.

Drupal 8.2, now with more outside-in

Over the weekend, Drupal 8.2 beta was released. One of the reasons why I'm so excited about this release is that it ships with "more outside-in". In an "outside-in experience", you can click anything on the page, edit its configuration in place without having to navigate to the administration back end, and watch it take effect immediately. This kind of on-the-fly editorial experience could be a game changer for Drupal's usability.

When I last discussed turning Drupal outside-in, we were still in the conceptual stages, with mockups illustrating the concepts. Since then, those designs have gone through multiple rounds of feedback from Drupal's usability team and a round of user testing led by Cheppers. This study identified some issues and provided some insights which were incorporated into subsequent designs.

Two policy changes we introduced in Drupal 8 — semantic versioning and experimental modules — have fundamentally changed Drupal's innovation model starting with Drupal 8. I should write a longer blog post about this, but the net result of those two changes is ongoing improvements with an easy upgrade path. In this case, it enabled us to add outside-in experiences to Drupal 8.2 instead of having to wait for Drupal 9. The authoring experience improvements we made in Drupal 8 are well-received, but that doesn't mean we are done. It's exciting that we can move much faster on making Drupal easier to use.

In-place block configuration

As you can see from the image below, Drupal 8.2 adds the ability to trigger "Edit" mode, which currently highlights all blocks on the page. Clicking on one — in this case, the block with the site's name — pops out a new tray or sidebar. A content creator can change the site name directly from the tray, without having to navigate through Drupal's administrative interface to theme settings as they would have to in Drupal 7 and Drupal 8.1.

Making adjustments to menus

In the second image, the pattern is applied to a menu block. You can make adjustments to the menu right from the new tray instead of having to navigate to the back end. Here the content creator changes the order of the menu links (moving "About us" after "Contact") and toggles the "Team" menu item from hidden to visible.

In-context block placement

In Drupal 8.1 and prior, placing a new block on the page required navigating away from your front end into the administrative back end and noting the available regions. Once you discover where to go to add a block, which can in itself be a challenge, you'll have to learn about the different regions, and some trial and error might be required to place a block exactly where you want it to go.

Starting in Drupal 8.2, content creators can now just click "Place block" without navigating to a different page and knowing about available regions ahead of time. Clicking "Place block" will highlight the different possible locations for a block to be placed in.

Next steps

These improvements are currently tagged "experimental". This means that anyone who downloads Drupal 8.2 can test these changes and provide feedback. It also means that we aren't quite satisfied with these changes yet and that you should expect to see this functionality improve between now and 8.2.0's release, and even after the Drupal 8.2.0 release.

As you probably noticed, things still look pretty raw in places; as an example, the forms in the tray are exposing too many visual details. There is more work to do to bring this functionality to the level of the designs. We're focused on improving that, as well as the underlying architecture and accessibility. Once we feel good about how it all works and looks, we'll remove the experimental label.

We deliberately postponed most of the design work to focus on introducing the fundamental concepts and patterns. That was an important first step. We wanted to enable Drupal developers to start experimenting with the outside-in pattern in Drupal 8.2. As part of that, we'll have to determine how this new pattern will apply broadly to Drupal core and the many contributed modules that would leverage it. Our hope is that once the outside-in work is stable and no longer experimental, it will trickle down to every Drupal module. At that point we can all work together, in parallel, on making Drupal much easier to use.

Users have proven time and again in usability studies to be extremely "preview-driven", so the ability to make quick configuration changes right from their front end, without becoming an expert in Drupal's information architecture, could be revolutionary for Drupal.

If you'd like to help get these features to stable release faster, please join us in the outside-in roadmap issue.

Thank you

I'd also like to thank everyone who contributed to these features and reviewed them, including Bojhan, yoroy, pwolanin, andrewmacpherson, gtamas, petycomp, zsofimajor, SKAUGHT, nod_, effulgentsia, Wim Leers, catch, alexpott, and xjm.

And finally, a special thank you to Acquia's outside-in team for driving most of the design and implementation: tkoleary, webchick, tedbow, Gábor Hojtsy, tim.plunkett, and drpal.

Acquia's outside in team
Acquia's outside-in team celebrating that the outside-in patch was committed to Drupal 8.2 beta. Go team!

Drupal goes to Rio

Rio olympic stadium

As the 2016 Summer Olympics in Rio de Janeiro enters its second and final week, it's worth noting that the last time I blogged about Drupal and the Olympics was way back in 2008 when I called attention to the fact that Nike was running its sponsorship site on Drupal 6 and using Drupal's multilingual capabilities to deliver their message in 13 languages.

While watching some track and field events on television, I also spent a lot of time on my laptop with the NBC Olympics website. It is a site that has run on Drupal for several years, and this year I noticed they took it up a notch and did a redesign to enhance the overall visitor experience.

Last week NBC issued a news release that it has streamed over one billion minutes of sports via their site so far. That's a massive number!

I take pride in knowing that an event as far-reaching as the Olympics is being delivered digitally to a massive audience by Drupal. In fact, some of the biggest sporting leagues around the globe run their websites off of Drupal, including NASCAR, the NBA, NFL, MLS, and NCAA. Massive events like the Super Bowl, Kentucky Derby, and the Olympics run on Drupal, making it the chosen platform for global athletic organizations.

Rio website

Rio press release

Update on August 24: This week, the NBC Sports Group issued a press release stating that the Rio 2016 Olympics was the most successful media event in history! Digital coverage across NBCOlympics.com and the NBC Sports app set records, with 3.3 billion total streaming minutes, 2.71 billion live streaming minutes, and 100 million unique users. According to the announcement, live streaming minutes for the Rio games nearly doubled that of all Olympic games combined, and digital coverage amassed 29 percent more unique users than the London Olympics four years prior. Drupal was proud to be a part of the largest digital sporting event in history. Looking forward to breaking more records in the years to come!

Setting higher standards for corporate contributions to Drupal

Last week I made a comment on Twitter that I'd like to see Pantheon contribute more to Drupal core. I wrote that in response to the announcement that Pantheon has raised a $30 million Series C. Pantheon has now raised $50 to $60 million dollars of working capital (depending on Industry Ventures' $8.5M) and is in a special class of companies. This is an amazing milestone. Though it wasn't meant that way, Pantheon and Acquia compete for business and my Tweet could be read as a cheap attack on a competitor, and so it resulted in a fair amount of criticism. Admittedly, Pantheon was neither the best nor the only example to single out. There are many companies that don't contribute to Drupal at all – and Pantheon does contribute to Drupal in a variety of ways such as sponsoring events and supporting the development of contributed modules. In hindsight, I recognize that my tweet was not one of my best, and for that I apologize.

Having said that, I'd like to reiterate something I've said before, in my remarks at DrupalCon Amsterdam and many times on this blog: I would like to see more companies contribute more to Drupal core – with the emphasis on "core". Drupal is now relied upon by many, and needs a strong base of commercial contributors. We have to build Drupal together. We need a bigger and more diverse base of organizations taking on both leadership and contribution.

Contribution to Drupal core is the most important type of contribution in terms of the impact it can make. It touches every aspect of Drupal and all users who depend on it. Long-term and full-time contribution to core is not within everyone's reach. It typically requires larger investment due to a variety of things: the complexity of the problems we are solving, our need for stringent security and the importance of having a rigorous review-process. So much is riding on Drupal for all of us today. While every module, theme, event and display of goodwill in our community is essential, contributions to core are quite possibly the hardest and most thankless, but also the most rewarding of all when it comes to Drupal's overall progress and success.

I believe we should have different expectations for different organizations based on their maturity, their funding, their profitability, how strategic Drupal is for them, etc. For example, sponsoring code sprints is an important form of contribution for small or mid-sized organizations. But for any organization that makes millions of dollars with Drupal, I would hope for more.

The real question that we have to answer is this: at what point should an organization meaningfully contribute to Drupal core? Some may say "never", and that is their Open Source right. But as Drupal's project lead it is also my right and responsibility to encourage those who benefit from Drupal to give back. It should not be taboo for our community to question organizations that don't pull their weight, or choose not to contribute at all.

For me, committing my workdays and nights to Drupal isn't the exhausting part of my job. It's dealing with criticism that comes from false or incomplete information, or tackling differences in ideals and beliefs. I've learned not to sweat the small stuff, but it's on important topics like giving back that my emotions and communication skills get tested. I will not apologize for encouraging organizations to contribute to Drupal core. It's a really important topic and one that I'm very passionate about. I feel good knowing that I'm pushing these conversations from inside the arena rather than from the sidelines, and for the benefit of the Drupal project at large.

City of Boston launches Boston.gov on Drupal

Yesterday the City of Boston launched its new website, Boston.gov, on Drupal. Not only is Boston a city well-known around the world, it has also become my home over the past 9 years. That makes it extra exciting to see the city of Boston use Drupal.

As a company headquartered in Boston, I'm also extremely proud to have Acquia involved with Boston.gov. The site is hosted on Acquia Cloud, and Acquia led a lot of the architecture, development and coordination. I remember pitching the project in the basement of Boston's City Hall, so seeing the site launched less than a year later is quite exciting.

The project was a big undertaking as the old website was 10 years old and running on Tridion. The city's digital team, Acquia, IDEO, Genuine Interactive, and others all worked together to reimagine how a government can serve its citizens better digitally. It was an ambitious project as the whole website was redesigned from scratch in 11 months; from creating a new identity, to interviewing citizens, to building, testing and launching the new site.

Along the way, the project relied heavily on feedback from a wide variety of residents. The openness and transparency of the whole process was refreshing. Even today, the city made its roadmap public at http://roadmap.boston.gov and is actively encouraging citizens to submit suggestions. This open process is one of the many reasons why I think Drupal is such a good fit for Boston.gov.

Boston gov tell us what you think

More than 20,000 web pages and one million words were rewritten in a more human tone to make the site easier to understand and navigate. For example, rather than organize information primarily by department (as is often the case with government websites), the new site is designed around how residents think about an issue, such as moving, starting a business or owning a car. Content is authored, maintained, and updated by more than 20 content authors across 120 city departments and initiatives.

Boston gov tools and apps

The new Boston.gov is absolutely beautiful, welcoming and usable. And, like any great technology endeavor, it will never stop improving. The City of Boston has only just begun its journey with Boston.gov - I’m excited see how it grows and evolves in the years to come. Go Boston!

Boston gov launch event
Boston gov launch event
Boston gov launch event
Last night there was a launch party to celebrate the launch of Boston.gov. It was an honor to give some remarks about this project alongside Boston Mayor Marty Walsh (pictured above), as well as Lauren Lockwood (Chief Digital Officer of the City of Boston) and Jascha Franklin-Hodge (Chief Information Officer of the City of Boston).

A roadmap for making Drupal more API-first

In one of my recent blog posts, I articulated a vision for the future of Drupal's web services, and at DrupalCon New Orleans, I announced the API-first initiative for Drupal 8. I believe that there is considerable momentum behind driving the web services initiative. As such, I want to provide a progress report, highlight some of the key people driving the work, and map the proposed vision from the previous blog post onto a rough timeline.

Here is a bird's-eye view of the plan for the next twelve months:

8.2 (Q4 2016) 8.3 (Q2 2017) Beyond 8.3 (2017+)
New REST API capabilities
Waterwheel initial release
New REST API capabilities
JSON API module
GraphQL module?
Entity graph iterator?

New REST API capabilities

Wim Leers (Acquia) and Daniel Wehner (Chapter Three) have produced a comprehensive list of the top priorities for the REST module. We're introducing significant REST API advancements in Drupal 8.2 and 8.3 in order to improve the developer experience and extend the capabilities of the REST API. We've been focused on configuration entity support, simplified REST configuration, translation and file upload support, pagination, and last but not least, support for user login, logout and registration. All this work starts to address differences between core's REST module and various contributed modules like Services and RELAXed Web Services. More details are available in my previous blog post.

Many thanks to Wim Leers (Acquia), Daniel Wehner (Chapter Three), Ted Bowman (Acquia), Alex Pott (Chapter Three), and others for their work on Drupal core's REST modules. Though there is considerable momentum behind efforts in core, we could always benefit from new contributors. Please consider taking a look at the REST module issue queue to help!

Waterwheel initial release

As I mentioned in my previous post, there has been exciting work surrounding Waterwheel, an SDK for JavaScript developers building Drupal-backed applications. If you want to build decoupled applications using a JavaScript framework (e.g. Angular, Ember, React, etc.) that use Drupal as a content repository, stay tuned for Waterwheel's initial release later this year.

Waterwheel aims to facilitate the construction of JavaScript applications that communicate with Drupal. Waterwheel's JavaScript library allows JavaScript developers to work with Drupal without needing deep knowledge of how requests should be authenticated against Drupal, what request headers should be included, and how responses are molded into particular data structures.

The Waterwheel Drupal module adds a new endpoint to Drupal's REST API allowing Waterwheel to discover entity resources and their fields. In other words, Waterwheel intelligently discovers and seamlessly integrates with the content model defined on any particular Drupal 8 site.

A wider ecosystem around Waterwheel is starting to grow as well. Gabe Sullice (Aten Design Group), creator of the Entity Query API module, has contributed an integration of Waterwheel which opens the door to features such as sorts, conditions and ranges. The Waterwheel team welcomes early adopters as well as those working on other REST modules such as JSON API and RELAXed or using native HTTP clients in JavaScript frameworks to add their own integrations to the mix.

Waterwheel is the currently the work of Matt Grill (Acquia) and Preston So (Acquia), who are developing the JavaScript library, and Ted Bowman (Acquia), who is working on the Drupal module.

JSON API module

In conjunction with the ongoing efforts in core REST, parallel work is underway to build a JSON API module which embraces the JSON API specification. JSON API is a particular implementation of REST that provides conventions for resource relationships, collections, filters, pagination, and sorting, in addition to error handling and full test coverage. These conventions help developers build clients faster and encourages reuse of code.

Thanks to Mateu Aguiló Bosch, Ed Faulkner and Gabe Sullice (Aten Design Group), who are spearheading the JSON API module work. The module could be ready for production use by the end of this year and included as an experimental module in core by 8.3. Contributors to JSON API are meeting weekly to discuss progress moving forward.

Beyond 8.3: GraphQL and entity graph iterator

While these other milestones are either certain or in the works, there are other projects gathering steam. Chief among these is GraphQL, which is a query language I highlighted in my Barcelona keynote and allows for clients to tailor the responses they receive based on the structure of the requests they issue.

One of the primary outcomes of the New Orleans web services discussion was the importance of a unified approach to iterating Drupal's entity graph; both GraphQL and JSON API require such an "entity graph iterator". Though much of this is still speculative and needs greater refinement, eventually, such an "entity graph iterator" could enable other functionality such as editable API responses (e.g. aliases for custom field names and timestamp formatters) and a unified versioning strategy for web services. However, more help is needed to keep making progress, and in absence of additional contributors, we do not believe this will land in Drupal until after 8.3.

Thanks to Sebastian Siemssen, who has been leading the effort around this work, which is currently available on GitHub.

Validating our work and getting involved

In order to validate all of the progress we've made, we need developers everywhere to test and experiment with what we're producing. This means stretching the limits of our core REST offerings, trying out JSON API for your own Drupal-backed applications, reporting issues and bugs as you encounter them, and participating in the discussions surrounding this exciting vision. Together, we can build towards a first-class API-first Drupal.

Special thanks to Preston So for contributions to this blog post and to Wim Leers for feedback during its writing.

Drupal is for ambitious digital experiences

What feelings does the name Drupal evoke? Perceptions vary from person to person; where one may describe it in positive terms as "powerful" and "flexible", another may describe it negatively as "complex". People describe Drupal differently not only as a result of their professional backgrounds, but also based on what they've heard and learned.

If you ask different people what Drupal is for, you'll get many different answers. This isn't a surprise because over the years, the answers to this fundamental question have evolved. Drupal started as a tool for hobbyists building community websites, but over time it has evolved to support large and sophisticated use cases.

Perception is everything

Perception is everything; it sets expectations and guides actions and inactions. We need to better communicate Drupal's identity, demonstrate its true value, and manage its perceptions and misconceptions. Words do lead to actions. Spending the time to capture what Drupal is for could energize and empower people to make better decisions when adopting, building and marketing Drupal.

Truth be told, I've been reluctant to define what Drupal is for, as it requires making trade-offs. I have feared that we would make the wrong choice or limit our growth. Over the years, it has become clear that not defining what Drupal is used for leaves more people confused even within our own community.

For example, because Drupal evolved from a simple tool for hobbyists to a more powerful digital experience platform, many people believe that Drupal is now "for the enterprise". While I agree that Drupal is a great fit for the enterprise, I personally never loved that categorization. It's not just large organizations that use Drupal. Individuals, small startups, universities, museums and non-profits can be equally ambitious in what they'd like to accomplish and Drupal can be an incredible solution for them.

Defining what Drupal is for

Rather than using "for the enterprise", I thought "for ambitious digital experiences" was a good phrase to describe what people can build using Drupal. I say "digital experiences" because I don't want to confine this definition to traditional browser-based websites. As I've stated in my Drupalcon New Orleans keynote, Drupal is used to power mobile applications, digital kiosks, conversational user experiences, and more. Today I really wanted to focus on the word "ambitious".

"Ambitious" is a good word because it aligns with the flexibility, scalability, speed and creative freedom that Drupal provides. Drupal projects may be ambitious because of the sheer scale (e.g. The Weather Channel), their security requirements (e.g. The White House), the number of sites (e.g. Johnson & Johnson manages thousands of Drupal sites), or specialized requirements of the project (e.g. the New York MTA powering digital kiosks with Drupal). Organizations are turning to Drupal because it gives them greater flexibility, better usability, deeper integrations, and faster innovation. Not all Drupal projects need these features on day one -- or needs to know about them -- but it is good to have them in case you need them later on.

"Ambitious" also aligns with our community's culture. Our industry is in constant change (responsive design, web services, social media, IoT), and we never look away. Drupal 8 was a very ambitious release; a reboot that took one-third of Drupal's lifespan to complete, but maneuvered Drupal to the right place for the future that is now coming. I have always believed that the Drupal community is ambitious, and believe that attitude remains strong in our community.

Last but not least, our adopters are also ambitious. They are using Drupal to transform their organizations digitally, leaving established business models and old business processes in the dust.

I like the position that Drupal is ambitious. Stating that Drupal is for ambitious digital experiences however is only a start. It only gives a taste of Drupal's objectives, scope, target audience and advantages. I think we'd benefit from being much more clear. I'm curious to know how you feel about the term "for ambitious digital experiences" versus "for the enterprise" versus not specifying anything. Let me know in the comments so we can figure out how to collectively change the perception of Drupal.

PS: I'm borrowing the term "ambitious" from the Ember.js community. They use the term in their tagline and slogan on their main page.

The long path to being understood

I sent an internal note to all of Acquia's 700+ employees today and decided to cross-post it to my blog because it contains a valuable lesson for any startup. One of my personal challenges — both as an Open Source evangelist/leader and entrepreneur — has been to learn to be comfortable with not being understood. Lots of people didn't believe in Open Source in Drupal's early days (and some still don't). Many people didn't believe Acquia could succeed (and some still don't). Something is radically different in software today, and the world is finally understanding and validating that some big shifts are happening. In many cases, an idea takes years to gain general acceptance. Such is the story of Drupal and Acquia. Along the way it can be difficult to deal with the naysayers and rejections. If you ever have an idea that is not understood, I want you to think of my story.

Team,

This week, Acquia got a nice mention on Techcrunch in an article written by Jake Flomenberg, a partner at Accel Partners. For those of you who don't know Accel Partners, they are one of the most prominent venture capital investors and were early investors in companies like Facebook, Dropbox, Slack, Etsy, Atlassian, Lynda.com, Kayak and more.

The article, called "The next wave in software is open adoption software", talks about how the enterprise IT stack is being redrawn atop powerful Open Source projects like MongoDB, Hadoop, Drupal and more. Included in the article is a graph that shows Acquia's place in the latest wave of change to transform the technology landscape, a place showing our opportunity is bigger than anything before as the software industry migrated from mainframes to client-server, then SaaS/PaaS and now - to what Flomenberg dubs, the age of Open Adoption Software.

Waves of software adoption

It's a great article, but it isn't new to any of us per se – we have been promoting this vision since our start nine years ago and we have seen over and over again how Open Source is becoming the dominant model for how enterprises build and deliver IT. We have also shown that we are building a successful technology company using Open Source.

Why then do I feel compelled to share this article, you ask? The article marks a small but important milestone for Acquia.

We started Acquia to build a new kind of company with a new kind of business model, a new innovation model, all optimized for a new world. A world where businesses are moving most applications into the cloud, where a lot of software is becoming Open Source, where IT infrastructure is becoming a metered utility, and where data-driven services make or break business results.

We've been steadily executing on this vision; it is why we invest in Open Source (e.g. Drupal), cloud infrastructure (e.g. Acquia Cloud and Site Factory), and data-centric business tools (e.g. Acquia Lift).

In my 15+ years as an Open Source evangelist, I've argued with thousands of people who didn't believe in Open Source. In my 8+ years as an entrepreneur, I've talked to thousands of business people and dozens of investors who didn't understand or believe in Acquia's vision. Throughout the years, Tom and I have presented Acquia's vision to many investors – some have bought in and some, like Accel, have not (for various reasons). I see more and more major corporations and venture capital firms coming around to Open Source business models every day. This trend is promising for new Open Source companies; I'm proud that Acquia has been a part of clearing their path to being understood.

When former skeptics become believers, you know you are finally being understood. The Techcrunch article is a small but important milestone because it signifies that Acquia is finally starting to be understood more widely. As flattering as the Techcrunch article is, true validation doesn't come in the form of an article written by a prominent venture capitalist; it comes day-in and day-out by our continued focus and passion to grow Drupal and Acquia bit by bit, one successful customer at a time.

Building a new kind of company like we are doing with Acquia is the harder, less-traveled path, but we always believed it would be the best path for our customers, our communities, and ultimately, our world. Success starts with building a great team that not only understands what we do, but truly believes in what we do and remains undeterred in its execution. Together, we can build this new kind of company.

--
Dries Buytaert
Founder and Project Lead, Drupal
Co-founder and Chief Technology Officer, Acquia

Gotthard tunnel website using Drupal

The Gotthard Base Tunnel, under construction for the last 17 years, was officially opened last week. This is the world's longest and deepest railroad tunnel, spanning 57 kilometers from Erstfeld to Bolio, Switzerland, underneath the Swiss Alps. To celebrate its opening, Switzerland also launched a multi-lingual multimedia website to celebrate the project's completion. I was excited to see they chose to build their site on Drupal 8! The site is a fitting digital tribute to an incredible project and launch event. Congratulations to the Gotthard Base Tunnel team!

Gottardo

Advancing Drupal's web services

In an earlier blog post, I looked at the web services solutions available in Drupal 8 and compared their strengths and weaknesses. That blog post was intended to help developers choose between different solutions when building Drupal 8 sites. In this blog post, I want to talk about how to advance Drupal's web services beyond Drupal 8.1 for the benefit of Drupal core contributors, module creators and technical decision-makers.

I believe it is really important to continue advancing Drupal's web services support. There are powerful market trends that oblige us to keep focused on this: integration with diverse systems having their own APIs, the proliferation of new devices, the expanding Internet of Things (IoT), and the widening adoption of JavaScript frameworks. All of these depend to some degree on robust web services.

Moreover, newer headless content-as-a-service solutions (e.g. Contentful, Prismic.io, Backand and CloudCMS) have entered the market and represent a widening interest in content repositories enabling more flexible content delivery. They provide content modeling tools, easy-to-use tools to construct REST APIs, and SDKs for different programming languages and client-side frameworks.

In my view, we need to do the following, which I summarize in each of the following sections: (1) facilitate a single robust REST module in core; (2) add functionality to help web services modules more easily query and manipulate Drupal's entity graph; (3) incorporate GraphQL and JSON API out of the box; and (4) add SDKs enabling easy integration with Drupal. Though I shared some of this in my DrupalCon New Orleans keynote, I wanted to provide more details in this blog post. I'm hoping to discuss this and revise it based on feedback from you.

One great REST module in core

While core REST can be enabled with only a few configuration changes, the full extent of possibilities in Drupal is only unlocked either when leveraging modules which add to or work alongside core REST's functionality, such as Services or RELAXed, or when augmenting core REST's capabilities with additional resources to interact with (by providing corresponding plugins) or using other custom code.

Having such disparate REST modules complicates the experience. These REST modules have overlapping or conflicting feature sets, which are shown in the following table.

Feature Core REST RELAXed Services Ideal core REST
Content entity CRUD Yes Yes Yes Yes
Configuration entity CRUD Create resource plugin (issue) Create resource plugin Yes Yes
Custom resources Create resource plugin Create resource plugin Create Services plugin Possible without code
Custom routes Create resource plugin or Views REST export (GET) Create resource plugin Configurable route prefixes Possible without code
Translations Not yet (issue) Yes Create Services plugin Yes
Revisions Create resource plugin Yes Create Services plugin Yes
File attachments Create resource plugin Yes Create Services plugin Yes
Authenticated user resources (log in/out, password reset) Not yet (issue) No User login and logout Yes

I would like to see a convergence where all of these can be achieved in Drupal core with minimal configuration and minimal code.

Working with Drupal's entity graph

Recently, a discussion at DrupalCon New Orleans with key contributors to the core REST modules, maintainers of important contributed web services modules, and external observers led to a proposed path forward for all of Drupal's web services.

Web services entity graph
A visual example of an entity graph in Drupal.

Buried inside Drupal is an "entity graph" over which different API approaches like traditional REST, JSON API, and GraphQL can be layered. These varied approaches all traverse and manipulate Drupal's entity graph, with differences solely in the syntax and features made possible by that syntax. Unlike core's REST API which only returns a single level (single entity or lists of entities), GraphQL and JSON API can return multiple levels of nested entities as the result of a single query. To better understand what this means, have a look at the GraphQL demo video I shared in my DrupalCon Barcelona keynote.

What we concluded at DrupalCon New Orleans is that Drupal's GraphQL and JSON API implementations require a substantial amount of custom code to traverse and manipulate Drupal's entity graph, that there was a lot of duplication in that code, and that there is an opportunity to provide more flexibility and simplicity. Therefore, it was agreed that we should first focus on building an "entity graph iterator" that can be reused by JSON API, GraphQL, and other modules.

This entity graph iterator would also enable manipulation of the graph, e.g. for aliasing fields in the graph or simplifying the structure. For example, the difference between Drupal's "base fields" and "configured fields" is irrelevant to an application developer using Drupal's web services API, but Drupal's responses leak this internal distinction by prefixing configured fields with field_ (see the left column in the table below). By the same token, all fields, even if they carry single values, expose the verbosity of Drupal's typed data system by being presented as arrays (see the left column in the table below). While there are both advantages and disadvantages to exposing single-value fields as arrays, many developers prefer more control over the output or the ability to opt into simpler outputs.

A good Drupal entity graph iterator would simplify the development of Drupal web service APIs, provide more flexibility over naming and structure, and eliminate duplicate code.

Current core REST (shortened response) Ideal core REST (shortened response)
{
  "nid": [
    {
      "value": "2"
    }
  ],
  "title": [
    {
      "value": "Lorem ipsum"
    }
  ],
  "field_product_number": [
    {
      "value": "35"
    }
  ],
  "field_image": [
    {
      "target_id": "2",
      "alt": "Image",
      "title": "Hover text",
      "width": "210",
      "height": "281",
      "url": "http://site.com/x.jpg"
    }
  ]
}
{
  "nid": "2"
  "title": "Lorem ipsum",
  "product_number": {
    "value": 35
  },
  "image": {
    "target_id": 2,
    "alt": "Image",
    "title": "Hover text",
    "width": 210,
    "height": 281,
    "url": "http://site.com/x.jpg"
  }
}

GraphQL and JSON API in core

We should acknowledge simultaneously that the wider JavaScript community is beginning to embrace different approaches, like JSON API and GraphQL, which both enable complex relational queries that require fewer requests between Drupal and the client (thanks to the ability to follow relationships, as mentioned in the section concerning the entity graph).

While both JSON API and GraphQL are preferred over traditional REST due to their ability to provide nested entity relationships, GraphQL goes a step further than JSON API by facilitating explicitly client-driven queries, in which the client dictates its data requirements.

I believe that GraphQL and JSON API in core would be a big win for those building decoupled applications with Drupal, and these modules can use existing foundations in Drupal 8 such as the Serialization module. Furthermore, Drupal's own built-in JavaScript-driven UIs could benefit tremendously from GraphQL and JSON API. I'd love to see them in core rather than as contributed modules, as we could leverage them when building decoupled applications backed by Drupal or exchanging data with other server-side implementations. We could also "eat our own dog food" by using them to power JavaScript-driven UIs for block placement, media management, and other administrative interfaces. I can even see a future where Views and GraphQL are closely integrated.

Web services rest json grapql
A comparison of different API approaches for Drupal 8, with amended and simplified payloads for illustrative purposes.

SDKs to consume web services

While a unified REST API and support for GraphQL and JSON API would dramatically improve Drupal as a web services back end, we need to be attentive to the needs of consumers of those web services as well by providing SDKs and helper libraries for developers new to Drupal.

An SDK could make it easy to retrieve an article node, modify a field, and send it back without having to learn the details of Drupal's particular REST API implementation or the structure of Drupal's underlying data storage. For example, this would allow front-end developers to not have to deal with the details of single- versus multi-value fields, optional vs required fields, validation errors, and so on. As an additional example, incorporating user account creation and password change requests into decoupled applications would empower front-end developers building these forms on a decoupled front end such that they would not need to know anything about how Drupal performs user authentication.

As starting points for JavaScript applications, native mobile applications, and even other back-end applications, these SDKs could handle authenticating against the API and juggling of the correct routes to resources without the front-end developer needing an understanding of those nuances.

In fact, at Acquia we're now in the early stages of building the first of several SDKs for consuming and manipulating data via Drupal 8's REST API. Waterwheel (previously Hydrant), a new generic helper library intended for JavaScript developers building applications backed by Drupal, is the work of Acquia's Matt Grill and Preston So, and it is already seeing community contributions. We're eager to share our work more widely and welcome new contributors.

Conclusion

I believe that it is important to have first-class web services in Drupal out of the box in order to enable top-notch APIs and continue our evolution to become API-first.

In parallel with our ongoing work on shoring up our REST module in core, we should provide the underpinnings for even richer web services solutions in the future. With reusable helper functionality that operates on Drupal's entity graph available in core, we open the door to GraphQL, JSON API, and even our current core REST implementation eventually relying on the same robust foundation. Both GraphQL and JSON API could also be promising modules in core. Last but not least, SDKs like Hydrant that empower developers to work with Drupal without learning its complexities will further advance our web services.

Collectively, these tracks of work will make Drupal uniquely compelling for application developers within our own community and well beyond.

Special thanks to Preston So for contributions to this blog post and to Moshe Weitzman, Kyle Browning, Kris Vanderwater, Wim Leers, Sebastian Siemssen, Tim Millwood, Ted Bowman, and Mateu Aguiló Bosch for their feedback during its writing.

Updates from Dries straight to your mailbox