Lamborghini using Drupal

Several people have commented on social media that last week's TAG Heuer post was a subtle (or not so subtle) gift request. In all honesty, it wasn't. That changes this week as I just learned that Lamborghini relaunched its website using Drupal 8. If anyone wants to gift a Lamborghini, don't hesitate to. Joking aside, it's great to see another famous brand use Drupal 8, including Drupal 8's new multi-lingual features. Check it out at https://www.lamborghini.com!
Lamborghini using drupal

How to move an entire government to a new digital platform

In this era of Open Government, constituents expect to be offered great services online. This can require moving an entire government to a new digital platform in order to deliver ambitious digital experiences that support the needs of citizens. It takes work, but many governments from the United States to Australia have demonstrated that with the right technology and strategy in place, governments can successfully adopt a new platform. Unfortunately this is not always the case.

How not to do it: Canada.ca

In 2014, the Government of Canada began a project to move all of its web pages onto a single site, Canada.ca. A $1.54 million contract for a content management system was awarded to a proprietary vendor in 2015. Fast forward to today, and the project is a year behind schedule and 10x over budget. The contract is now approaching $10 million. As only 0.05% of the migration to Canada.ca has been completed, many consider the current project to be disservice to its citizens.

I find the impending outcomes of this project to be disheartening as current timelines suggest that the migration will continue to be delayed, run over budget, and strain taxpayers. While I hope that Canada.ca will develop into a valuable resource for its citizens, I agree with Tom Cochran, Acquia's Chief Digital Strategist for Public Sector -- who ran digital platforms at the White House and U.S. Department of State -- that the prospects for Canada.ca are dim given the way the project was designed and is being implemented.

The root of Canada.ca's problem appears to be the decision to migrate 1,500 departments and 17 million pages into a single site. I'm guessing that the goal of having a single site is to offer citizens a central entry point to connect with their government. A single site strategy can be extremely effective, for example the City of Boston's single site is home to over 200,000 web page spanning 120 city departments, and offers a truly user-centric experience. With 17 million pages to migrate, Canada.ca is eighty-five times bigger than Boston.gov. A project of this magnitude should have considered using a multi-site approach where different agencies and departments have their own sites, but can use a common platform, toolset and shared infrastructure.

While difficulties with Canada.ca may have started with the ambitious attempt to move every department to a single domain, the complexities of this strategy are likely amplified through the implementation of a single-source proprietary solution. I find it unfortunate that Canada's procurement models did not favor Open Source. The Canadian government has a tenured history of utilizing Open Source, and there is a lot of existing Drupal talent in the country. In rejecting an open platform, the Canadian Government lost the opportunity to engage a skilled community of native, Open Source developers.

How to do it: Australian Government

Transforming an entire nation's digital strategy is challenging, but other public sector leaders have proven it is possible. Take the Australian Government. In 2015, John Sheridan, Sharyn Clarkson and their team in the Department of Finance moved their department's site from a legacy environment to Drupal and the cloud. The Department of Finance's success has grown into the Drupal distribution govCMS, which is currently supporting over 52 government agencies across 6 jurisdictions in Australia. Much like Canada.ca, the goal of govCMS is to provide citizens with a more intuitive platform to engage with their government.

The guiding principle of govCMS is to govern but to not seek control. Each government department requires flexibility to service the needs of their particular audiences. While single-site solutions do work as umbrellas for some organizations, the City of Boston being a great example, most large (government) organizations that have a state-of-the-art approach follow a hub and spoke model where different sites share code, templates and infrastructure. While sharing is strongly encouraged it is not required. This allows each department to independently innovate and adopt the platform how they choose.

The Open Source nature of govCMS has encouraged both innovation and collaboration across many Australian departments. One of the most remarkable examples of this is that a federal agency and a state agency coordinated their development efforts to build a data visualization capability on an open data CKAN repository. The Department of Environment initiated the development of the CKAN module necessary to pull and analyze data from a variety of departments. The Victorian Department of Premier and Cabinet recognized that they too could utilize the module to propel their budget report and aided in the co-development of the govCMS CKAN. This is an incredible example of how Open Source allows agencies to extend functionality across departments, regardless of vendor involvement. By setting up a model which removed the barriers to share, govCMS has provided Australia the freedom to truly innovate.

Seeing is believing: shifting the prevailing mindset

A distributed model using multiple sites to leverage an Open Source platform where infrastructure, code and templates are shared allows for governance and innovation to co-exist. I've written about this model here in a post about Loosening control the Open Source Way. I believe that a multi-site approach based on Open Source is the only way to move an entire government to a new digital strategy and platform.

It can be incredibly hard for organizations to understand this. After all, this is not about product features, technical capabilities or commercial support, but about a completely different way of working and innovating. It's a hard sell because we have to change the lens through which organizations see the world; away from procuring proprietary software that provides perceived safety and control, to a world that allows frictionless innovation and sharing through the loosening of control without losing control. For us to successfully market and sell the innovation that comes out of Drupal, Open Source and cloud, we have to shift how people think and challenge the prevailing model.

In many ways, organizations have to see it to believe it. What is exciting about the Australian government is that it helps others see the potential of a decentralized service model predicated on Open Source software with a Drupal distribution at its heart. The Australian government has created an ecosystem of frictionless sharing that is cheaper, faster, and enables better results.

What is next for Canada?

It’s difficult for me to see a light at the end of the tunnel for Canadian citizens. While the Canadian government can stay the course -– and all indications so far are that they will -- that path has a high price tag, long delays and slow innovation. An alternative would be for Ottawa to hit the pause button and reassess their strategy. They could look externally to how governments in Washington, Canberra, and countless others approached their mission to support the digital needs of its citizens. I know that there are countless Drupal experts working both within the government and at dozens of Drupal agencies throughout Canada that are eager to show their government a better way forward.

TAG Heuer using Drupal

Growing up my dad had a watch from TAG Heuer. As a young child, I always admired his watch and wished that one day I'd have one as well. I still don't have a TAG Heuer watch, however I just found out that TAG Heuer relaunched its website using Drupal 8 and that is pretty cool too.

TAG Heuer's new website integrates Drupal 8 with their existing Magento commerce platform to provide a better, more content-rich shopping experience. It is a nice example of the "Content for Commerce"-opportunity that I wrote about a month ago. Check it out at https://www.tagheuer.com!

Tag heuer using drupal

Buytaert.net now on Drupal 8

Some of my readers may have noticed that I recently upgraded this blog from Drupal 7 to Drupal 8. You might be wondering why it took me so long, and the truth is that while I love working on my site, it's hard to justify spending much time on it.

I did most of the work on my commute to work, cramped in a small seat on the train to the tune of the rumbling train carriage. I finally pushed the site live during Thanksgiving weekend. Nothing like seeing the Drupal migration tool run on your computer to the smell of the turkey cooking in the oven. The comfortable atmosphere of Thanksgiving -- and the glass of wine that was sitting on my desk -- made it all the more fun.

While my blog has been around for more than a decade, it isn't very complex; I have approximately 1,250 blog posts and 8,000 photos. All of the photos on my website are managed by my Album module, a custom Drupal module whose first version I wrote about ten years ago. Since its inception, I've upgraded the module to each major Drupal version from Drupal 5 to 7, and now to Drupal 8.

Upgrading my little website was relatively simple, involving porting my custom Album module, porting my theme to use Twig, and migrating the data and configuration. Though I've reviewed and committed many patches that went into Drupal 8, this was the first time I'm actually undertaking a Drupal 8 module, theme, and migration from scratch.

Each of the steps in upgrading a Drupal site have undergone substantial changes in the transition from Drupal 7 to Drupal 8. Building a module, for instance, is completely different because of the new introduction of object-oriented programming paradigms, the new routing system, and more. Writing a module in Drupal 8 is so much nicer than writing a module in Drupal 7; my code is better organized, more reusable, and better tested. Theming has been completely reinvented to use Twig. I found Twig much more pleasant to use than PHPTemplate in Drupal 7, and I'm very happy with how the theming process has improved. That said, I've to give a special shoutout to Preston So for helping me with my theme; I was painfully reminded I'm still not a designer or a CSS expert.

Overall, porting my site was a fun experience and I'm excited to be using Drupal 8. There is so much power packed into Drupal 8. Now that I've experienced Drupal 8 development from the other side of the equation, I believe even more in the decisions we made. While there is always room for improvement, Drupal 8 is a giant leap forward for Drupal.

Improving Drupal 8's API-first: JSON API and OAuth2?

Among the most important initiatives for Drupal 8's future is the "API-first initiative", whose goal is to improve Drupal's web services capabilities for building decoupled applications. In my previous blog posts, I evaluated the current state of, outlined a vision for, and mapped a path forward for Drupal's web services solutions.

In the five months since my last update, web services in Drupal have moved forward substantially in functionality and ease of use. This blog post discusses the biggest improvements.

Api first building blocks
An overview of the key building blocks to create web services in Drupal. Out of the box, Drupal core can expose raw JSON structures reflecting its internal storage, or it can expose them in HAL. Note: Waterwheel has an optional dependency on JSON API if JSON API methods are invoked through Waterwheel.js.

Core REST improvements

With the release of Drupal 8.2, the core REST API now supports important requirements for decoupled applications. First, you can now retrieve configuration entities (such as blocks and menus) as REST resources. You can also conduct user registration through the REST API. The developer experience of core REST has improved as well, with simplified REST configuration and better response messages and status codes for requests causing errors.

Moreover, you can now access Drupal content from decoupled applications and sites hosted on other domains thanks to opt-in cross-origin resource sharing (CORS) support. This is particularly useful if you have a JavaScript application hosted directly on AWS, for instance, while your Drupal repository is hosted on a separate platform. Thanks to all this progress, you can now build more feature-rich decoupled applications with Drupal.

All of these improvements are thanks to the hard work of Wim Leers (Acquia), Ted Bowman (Acquia), Daniel Wehner (Chapter Three), Clemens Tolboom, José Manuel Rodríguez Vélez, Klaus Purer, James Gilliland (APQC), and Gabe Sullice (Aten Design Group).

JSON API as an emerging standard

JSON API is a specification for REST APIs in JSON which offers several compelling advantages over the current core REST API. First, JSON API provides a standard way to query not only single entities but also all relationships tied to that entity and to perform query operations through query string parameters (for example, listing all tags associated with an article). Second, JSON API allows you to fetch lists of content entities (including filtering, sorting and paging), whereas the only options for this currently available in core are to issue multiple requests, which is undesirable for performance, or to query Drupal views, which require additional work to create.

Moreover, JSON API is increasingly common in the JavaScript community due to its adoption by developers creating REST APIs in JSON and by members of both the Ruby on Rails and Ember communities. In short, the momentum outside of Drupal currently favors JSON API over HAL. It's my belief that JSON API should become an experimental core module, and it may one day potentially even supersede HAL, Views-based REST endpoints, and more. Though Views REST exports will always be valuable for special needs, JSON API is suitable for more common tasks due to query operations needing no additional configuration. All this being said, we should discuss and evaluate the implications of prioritizing JSON API.

Thanks to the efforts of Mateu Aguiló Bosch (Lullabot) and Gabe Sullice (Aten Design Group), the JSON API module in Drupal 8 is quickly approaching a level of stability that could put it under consideration as a core experimental module.

OAuth2 bearer token authentication

One of the issues facing many decoupled applications today is the lack of a robust authentication mechanism when working with Drupal's REST API. Currently, core REST only has two options available out of the box, namely cookie-based authentication, which is unhelpful for decoupled applications on domains other than the Drupal site, and basic authentication, which is less secure than other mechanisms.

Due to its wide acceptance, OAuth2 seems a logical next step for Drupal sites that need to authenticate requests. Because it is more secure than what is available in core REST's basic authentication, OAuth2 would help developers build more secure decoupled Drupal architectures and allow us to deprecate the other less secure approaches.

It would make sense to see an OAuth2 solution such as Simple OAuth, the work of Mateu Aguiló Bosch (Lullabot), as an experimental core module, so that we can make REST APIs in Drupal more secure.

Waterwheel, Drupal's SDK ecosystem

The API-first initiative's work goes beyond making improvements on the Drupal back end. Acquia released Waterwheel.js 1.0 as open-source, the first iteration of a helper SDK for JavaScript developers. The Waterwheel.js SDK helps JavaScript developers perform requests against Drupal without requiring extensive knowledge of how Drupal's REST API functions. For example, the Waterwheel module allows you to benefit from resource discovery, which makes your JavaScript application aware of Drupal's data model. Waterwheel.js also integrates easily with the JSON API contributed module.

Thanks to Kyle Browning (Acquia), the Waterwheel.swift SDK allows developers of applications for Apple devices such as iPads, iPhones, Apple Watches, and Apple TVs to more quickly build their Drupal-powered applications. To learn more about the Waterwheel ecosystem, check the blog posts by Preston So (Acquia).

Conclusion

Thanks to the hard work of the many contributors involved, the API-first initiative is progressing with great momentum. In this post, we ended with the following conclusions:

  • The JSON API and Simple OAuth modules could be candidates for inclusion in Drupal 8 core — this is something we should discuss and evaluate.
  • We should discuss where support for HAL and JSON API stops and starts, because it helps us focus our time and effort.

If you are building decoupled applications with Drupal 8, we would benefit significantly from your impressions of the core REST API and JSON API implementations now available. What features are missing? What queries are difficult or need work? How can we make the APIs more suited to your requirements? Use the comments section on this blog post to tell us more about the work you are doing so we can learn from your experiences.

Of course, if you can help accelerate the work of the API-first initiative by contributing code, reviewing and testing patches, or merely by participating in the discussions within the issues, you will not only be helping improve Drupal itself; you'll be helping improve the experience of all developers who wish to use Drupal as their ideal API-first back end.

Special thanks to Preston So for contributions to this blog post and to Wim Leers, Angie Byron, Ted Bowman and Chris Hamper for their feedback during the writing process.