The Drupal community is very special because of its culture of adapting to change, determination and passion, but also its fun and friendship. It is a combination that is hard to come by, even in the Open Source world. Our culture enabled us to work through really long, but ground-breaking release cycles, which also prompted us to celebrate the release of Drupal 8 with 240 parties around the world.
Throughout Drupal's 15 years history, that culture has served us really well. As the larger industry around us continues to change -- see my DrupalCon New Orleans keynote for recent examples -- we have been able to evolve Drupal accordingly. Drupal has not only survived massive changes in our industry; it has also helped drive them. Very few open source projects are 15 years old and continue to gain momentum.
Drupal 8 is creating new kinds of opportunities for Drupal. For example, who could have imagined that Lufthansa would be using Drupal 8 to build its next-generation in-flight entertainment system? Drupal 8 changes the kind of end-user experiences people can build, how we think about Drupal, and what kind of people we'll attract to our community. I firmly believe that these changes are positive for Drupal, increase Drupal's impact on the world, and grow the opportunity for our commercial ecosystem.
To seize the big opportunity ahead of us and to adjust to the changing environment, it was the Drupal Association's turn to adapt and carefully realign the Drupal Association's strategic focus.
The last couple of years the Drupal Association invested heavily in Drupal.org to support the development and the release of Drupal 8. Now Drupal 8 is released, the Drupal Association's Board of Directors made the strategic decision to shift some focus from the "contribution journey" to the "evaluator's adoption journey" -- without compromising our ability to build and maintain the Drupal software. The Drupal Association will reduce its efforts on Drupal.org's collaboration tools and expand its efforts to grow Drupal's adoption and to build a larger ecosystem of technology partners.
We believe this is not only the right strategic focus at this point in Drupal 8's lifecycle, but also a necessary decision. While the Drupal Association's revenues continued to grow at a healthy pace, we invested heavily, and exhausted our available reserves supporting the Drupal 8 release. As a result, we have to right-size the organization, balance our income with our expenses, and focus on rebuilding our reserves.
In a blog post today, we provide more details on why we made these decisions and how we will continue to build a healthy long-term organization. The changes we made today help ensure that Drupal will gain momentum for decades to come. We could not make this community what it is without the participation of each and every one of you. Thanks for your support!
This is a time of transition for the Drupal Association. As you might have read on the Drupal Association blog, Holly Ross, our Executive Director, is moving on. Megan Sanicki, who has been with the Drupal Association for almost 6 years, and was working alongside Holly as the Drupal Association's COO, will take over Holly's role as the Executive Director.
Open source stewardship is not easy but in the 3 years Holly was leading the Drupal Association, she lead with passion, determination and transparency. She operationalized the Drupal Association and built a team that truly embraces its mission to serve the community, growing that team by over 50% over three years of her tenure. She established a relationship with the community that wasn't there before, allowing the Drupal Association to help in new ways like supporting the Drupal 8 launch, providing test infrastructure, implementing the Drupal contribution credit system, and more. Holly also matured our DrupalCon, expanding its reach to more users with conferences in Latin America and India. She also executed the Drupal 8 Accelerate Fund, which allowed direct funding of key contributors to help lead Drupal 8 to a successful release.
Holly did a lot for Drupal. She touched all of us in the Drupal community. She helped us become better and work closer together. It is sad to see her leave, but I'm confident she'll find success in future endeavors. Thanks, Holly!
Megan, the Drupal Association staff and the Board of Directors are committed to supporting the Drupal project. In this time of transition, we are focused on the work that Drupal Association must do and looking at how to do that in a sustainable way so we can support the project for many years to come.
Last year around this time, I wrote that The Big Reverse of Web would force a major re-architecture of the web to bring the right information, to the right person, at the right time, in the right context. I believe that conversational interfaces like Amazon Echo are further proof that the big reverse is happening.
New user experience and distribution platforms only come along every 5-10 years, and when they do, they cause massive shifts in the web's underlying technology. The last big one was mobile, and the web industry adapted. Conversational interfaces could be the next user experience and distribution platform – just look at Amazon Echo (aka Alexa), Facebook's messenger or Microsoft's Conversation-as-a-Platform.
Today, hardly anyone questions whether to build a mobile-optimized website. A decade from now, we might be saying the same thing about optimizing digital experiences for voice or chat commands. The convenience of a customer experience will be a critical key differentiator. As a result, no one will think twice about optimizing their websites for multiple interaction patterns, including conversational interfaces like voice and chat. Anyone will be able to deliver a continuous user experience across multiple channels, devices and interaction patterns. In some of these cross-channel experiences, users will never even look at a website. Conversational interfaces let users disintermediate the website by asking anything and getting instant, often personalized, results.
To prototype this future, my team at Acquia built a fully functional demo based on Drupal 8 and recorded a video of it. In the demo video below, we show a sample supermarket chain called Gourmet Market. Gourmet Market wants their customers to not only shop online using their website, but also use Echo or push notifications to do business with them.
We built an Alexa integration module to connect Alexa to the Gourmet Market site and to answer questions about sale items. For example, you can speak the command: "Alexa, ask Gourmet Market what fruits are on sale today". From there, Alexa would make a call to the Gourmet Market website, finding what is on sale in the specified category and pull only the needed information related to your ask.
On the website's side, a store manager can tag certain items as "on sale", and Alexa's voice responses will automatically and instantly reflect those changes. The marketing manager needs no expertise in programming -- Alexa composes its response by talking to Drupal 8 using web service APIs.
The demo video also shows how a site could deliver smart notifications. If you ask for an item that is not on sale, the Gourmet Market site can automatically notify you via text once the store manager tags it as "On Sale".
From a technical point of view, we've had to teach Drupal how to respond to a voice command, otherwise known as a "Skill", coming into Alexa. Alexa Skills are fairly straightforward to create. First, you specify a list of "Intents", which are basically the commands you want users to run in a way very similar to Drupal's routes. From there, you specify a list of "Utterances", or sentences you want Echo to react to that map to the Intents. In the example of Gourmet Market above, the Intents would have a command called
GetSaleItems. Once the command is executed, your Drupal site will receive a webhook callback on
/alexa/callback with a payload of the command and any arguments. The Alexa module for Drupal 8 will validate that the request really came from Alexa, and fire a Drupal Event that allows any Drupal module to respond.
It's exciting to think about how new user experiences and distribution platforms will change the way we build the web in the future. As I referenced in Drupalcon New Orleans keynote, the Drupal community needs to put some thought into how to design and build multichannel customer experiences. Voice assistance, chatbots or notifications are just one part of the greater equation. If you have any further thoughts on this topic, please share them in the comments.
DrupalCon New Orleans comes at an important time in the history of Drupal. Now that Drupal 8 has launched, we have a lot of work to do to accelerate Drupal 8's adoption as well as plan what is next.
In my keynote presentation, I shared my thoughts on where we should focus our efforts in order for Drupal to continue its path to become the leading platform for assembling the world's best digital experiences.
Based on recent survey data, I proposed key initiatives for Drupal, as well as shared my vision for building cross-channel customer experiences that span various devices, including conversational technologies like Amazon Echo.
There is much more data hidden in the raw survey results, so if you'd like to do your own analysis, you can download a copy of the raw survey results (CSV format or XLS format) and look at the raw data yourself. I anonymized the data by removing the names, e-mail addresses and IP address information. If you decide to analyze the raw data, consider sharing your findings with all of us.
Take a look, and as always feel free to leave your opinions in the comments!
Today's third-party applications increasingly depend on web services to retrieve and manipulate data, and Drupal offers a range of web services options for API-first content delivery. For example, a robust first-class web services layer is now available out-of-the-box with Drupal 8. But there are also new approaches to expose Drupal data, including Services and newer entrants like RELAXed Web Services and GraphQL.
The goal of this blog post is to enable Drupal developers in need of web services to make an educated decision about the right web services solution for their project. This blog post also sets the stage for a future blog post, where I plan to share my thoughts about how I believe we should move Drupal core's web services API forward. Getting aligned on our strengths and weaknesses is an essential first step before we can brainstorm about the future.
The Drupal community now has a range of web services modules available in core and as contributed modules sharing overlapping missions but leveraging disparate mechanisms and architectural styles to achieve them. Here is a comparison table of the most notable web services modules in Drupal 8:
|Content entity CRUD||Yes||Yes||Yes|
|Configuration entity CRUD||Create resource plugin (issue)||Create resource plugin||Yes|
|Custom resources||Create resource plugin||Create resource plugin||Create Services plugin|
|Custom routes||Create resource plugin or Views REST export (GET)||Create resource plugin||Configurable route prefixes|
|Renderable objects||Not applicable||Not applicable||Yes (no contextual blocks or views)|
|Translations||Not yet (issue)||Yes||Create Services plugin|
|Revisions||Create resource plugin||Yes||Create Services plugin|
|File attachments||Create resource plugin||Yes||Create Services plugin|
|Shareable UUIDs (GET)||Yes||Yes||Yes|
|Authenticated user resources (log in/out, password reset)||Not yet (issue)||No||User login and logout|
Core RESTful Web Services
Thanks to the Web Services and Context Core Initiative (WSCCI), Drupal 8 is now an out-of-the-box REST server with operations to create, read, update, and delete (CRUD) content entities such as nodes, users, taxonomy terms, and comments. The four primary REST modules in core are:
- Serialization is able to perform serialization by providing normalizers and encoders. First, it normalizes Drupal data (entities and their fields) into arrays with a particular structure. Any normalization can then be sent to an encoder, which transforms those arrays into data formats such as JSON or XML.
- RESTful Web Services allows for HTTP methods to be performed on existing resources including but not limited to content entities and views (the latter facilitated through the “REST export" display in Views) and custom resources added through REST plugins.
- HAL builds on top of the Serialization module and adds the Hypertext Application Language normalization, a format that enables you to design an API geared toward clients moving between distinct resources through hyperlinks.
- Basic Auth allows you to include a username and password with request headers for operations requiring permissions beyond that of an anonymous user. It should only be used with HTTPS.
Core REST adheres strictly to REST principles in that resources directly match their URIs (accessible via a query parameter, e.g.
?_format=json for JSON) and in the ability to serialize non-content into JSON or XML representations. By default, core REST also includes two authentication mechanisms: basic authentication and cookie-based authentication.
While core REST provides a range of features with only a few steps of configuration there are several reasons why other options, available as contributed modules, may be a better choice. Limitations of core REST include the lack of support for configuration entities as well as the inability to include file attachments and revisions in response payloads. With your help, we can continue to improve and expand core's REST support.
RELAXed Web Services
As I highlighted in my recent blog post about improving Drupal's content workflow, RELAXed Web Services, is part of a larger suite of modules handling content staging and deployment across environments. It is explicitly tied to the CouchDB API specification, and when enabled, will yield a REST API that operates like the CouchDB REST API. This means that CouchDB integration with client-side libraries such as PouchDB and Hood.ie makes possible offline-enabled Drupal, which synchronizes content once the client regains connectivity. Moreover, people new to Drupal with exposure to CouchDB will immediately understand the API, since there is robust documentation for the endpoints.
RELAXed Web Services depends on core's REST modules and extends its functionality by adding support for translations, parent revisions (through the Multiversion module), file attachments, and especially cross-environment UUID references, which make it possible to replicate content to Drupal sites or other CouchDB compatible services. UUID references and revisions are essential to resolving merge conflicts during the content staging process. I believe it would be great to support translations, parent revisions, file attachments, and UUID references in core's RESTful web services — we simply didn't get around to them in time for Drupal 8.0.0.
Since RESTful Web Services are now incorporated into Drupal 8 core, relevant contributed modules have either been superseded or have gained new missions in the interest of extending existing core REST functionality. In the case of Services, a popular Drupal 7 module for providing Drupal data to external applications, the module has evolved considerably for its upcoming Drupal 8 release.
With Services in Drupal 8 you can assign a custom name to your endpoint to distinguish your resources from those provisioned by core and also provision custom resources similar to core's RESTful Web Services. In addition to content entities, Services supports configuration entities such as blocks and menus — this can be important when you want to build a decoupled application that leverages Drupal's menu and blocks system. Moreover, Services is capable of returning renderable objects encoded in JSON, which allows you to use Drupal's server-side rendering of blocks and menus in an entirely distinct application.
At the time of this writing, the Drupal 8 version of Services module is not yet feature-complete: there is no test coverage, no content entity validation (when creating or modifying), no field access checking, and no CSRF protection, so caution is important when using Services in its current state, and contributions are greatly appreciated.
GraphQL, originally created by Facebook to power its data fetching, is a query language that enables fewer queries and limits response bloat. Rather than tightly coupling responses with a predefined schema, GraphQL overturns this common practice by allowing for the client's request to explicitly tailor a response so that the client only receives what it needs: no more and no less. To accomplish this, client requests and server responses have a shared shape. It doesn't fall into the same category as the web services modules that expose a REST API and as such is absent from the table above.
GraphQL shifts responsibility from the server to the client: the server publishes its possibilities, and the client publishes its requirements instead of receiving a response dictated solely by the server. In addition, information from related entities (e.g. both a node's body and its author's e-mail address) can be retrieved in a single request rather than successive ones.
Typical REST APIs tend to be static (or versioned, in many cases, e.g.
/api/v1) in order to facilitate backwards compatibility for applications. However, in Drupal's case, when the underlying content model is inevitably augmented or otherwise changed, schema compatibility is no longer guaranteed. For instance, when you remove a field from a content type or modify it, Drupal's core REST API is no longer compatible with those applications expecting that field to be present. With GraphQL's native schema introspection and client-specified queries, the API is much less opaque from the client's perspective in that the client is aware of what response will result according to its own requirements.
I'm very bullish on the potential for GraphQL, which I believe makes a lot of sense in core in the long term. I featured the project in my Barcelona keynote (demo video), and Acquia also sponsored development of the GraphQL module (Drupal 8 only) following DrupalCon Barcelona. The GraphQL module, created by Sebastian Siemssen, now supports read queries, implements the GraphiQL query testing interface, and can be integrated with Relay (with some limitations).
For most simple REST API use cases, core REST is adequate, but core REST can be insufficient for more complex use cases. Depending on your use case, you may need more off-the-shelf functionality without the need to write a resource plugin or custom code, such as support for configuration entity CRUD (Services); for revisions, file attachments, translations, and cross-environment UUIDs (RELAXed); or for client-driven queries (GraphQL).
Special thanks to Preston So for contributions to this blog post and to Moshe Weitzman, Kyle Browning, Kris Vanderwater, Wim Leers, Sebastian Siemssen, Tim Millwood and Ted Bowman for their feedback during its writing.