Drupal 8 turns one!

Tomorrow is the one year anniversary of Drupal 8. On this day last year we celebrated the release of Drupal 8 with over 200 parties around the world. It's a project we worked on for almost five years, bringing the work of more than 3,000 contributors together to make Drupal more flexible, innovative, scalable, and easier to use.

To celebrate tomorrow's release-versary, I wanted to look back at a few of the amazing Drupal 8 projects that have launched in the past year.

1. NBA.com

The NBA is one of the largest professional sports leagues in the United States and Canada. Millions of fans around the globe rely on the NBA's Drupal 8 website to livestream games, read stats and standings, and stay up to date on their favorite team. Drupal 8 will bring you courtside, no matter who you're rooting for.

2. Nasdaq

Nasdaq Corporate Solutions has selected Drupal 8 as the basis for its next generation Investor Relations Website Platform. IR websites are where public companies share their most sensitive and critical news and information with their shareholders, institutional investors, the media and analysts. With Drupal 8, Nasdaq Corporate Solutions will be providing companies with the most engaging, secure, and innovative IR websites to date.

3.Hubert Burda Media

For more than 100 years, Hubert Burda Media has been Germany's premier media company. Burda is using Drupal 8 to expand their traditional business of print publishing to reach more than 52 million readers online. Burda didn't stop there, the media company also open sourced Thunder, a distribution for professional publishers built on Drupal 8.

4. Jurassic World

Drupal 8 propels a wide variety of sites, some of Jurassic proportion. Following the release of the blockbuster film, Jurassic World built its digital park on Drupal 8. Jurassic World offers fans games, video, community forums, and even interactive profiles all of the epic dinosaurs found on Isla Nublar.

5. WWF

The World Wide Fund for Nature has been a leading conservation organization since its founding in 1961. WWF's mission is to protect our planet and Drupal 8 is on their team. WWF UK uses Drupal 8 to engage the community, enabling users to adopt, donate and join online. From pole to pole, Drupal 8 and WWF are making an impact.

6. YMCA Greater Twin Cities

The YMCA is one the leading non-profit organizations for youth development, healthy living, and social responsibility. The YMCA serves more than 45 million people in 119 countries. The team at YMCA Greater Twin Cities turned to Drupal 8 to build OpenY, a platform that allows YMCA members to check in, set fitness goals, and book classes. They even hooked up Drupal to workout machines and wearables like Fitbit, which enables visitors to track their workouts from a Drupal 8 powered mobile app. The team at Greater Twin Cities also took advantage of Drupal 8's built-in multilingual capabilities so that other YMCAs around the world can participate. The YMCA has set a new personal record, and is a great example of what is possible with Drupal 8.

7. Jack Daniels

The one year anniversary of Drupal 8 is cause for celebration, so why not raise a glass? You might try Jack Daniels and their Drupal 8 website. Jack Daniels has been making whiskey for 150 years and you can get your fill with Drupal 8.

8. Al Jazeera Media Network

Al Jazeera is the largest news organization focused on the Middle East, and broadcasts news and current affairs 24 hours a day, 7 days a week. Al Jazeera required a platform that could unify several different content streams and support a complicated editorial workflow, allowing network wide collaboration and search. Drupal 8 allowed Al Jazeera to do that and then some. Content creators can now easily deliver critical news to their readers in real time.

9. Alabama.gov

From Boston to LA and even Australia, Drupal is supporting the digital needs of governments around the globe. Alabama is leading the way with Drupal 8. Alabama.gov puts its citizens first, and demonstrates how open source can change the way the public sector engages online.

10. Box

Box has been a leader in the technology industry since its founding in 2005. Box takes advantage of Drupal 8 and the improved features made available right out-of-the-box. Bad puns aside, companies like Box are using Drupal 8's new features and improved user interface to build the best digital experiences yet.

11. Habitat for Humanity

The historic nonprofit Habitat for Humanity doesn't just build houses for those in need; they build habitat.org on Drupal 8. Habitat for Humanity provides affordable housing for communities in over 70 countries around the world. You can discover their impact through the "Where we Build” interactive map, donate, and volunteer all on their Drupal 8 site.

12. Obermeyer

Obermeyer and Drupal 8 will take you into new territory. The ski wear company offers seamless end to end commerce integration, providing both new and loyal customers a great shopping experience. Let Obermeyer's Drupal 8 integration with Drupal Commerce keep you warm because winter is coming ...

Happy 1st birthday Drupal 8!

I can't think of a better way to celebrate Drupal 8's one year anniversary than by sharing some incredible experiences that are being created with Drupal 8. Whether the project is big or small, features dinosaurs, or spreads awareness for an important cause, I'm proud to say that Drupal 8 is supporting an amazing array of projects. In my recent keynote at DrupalCon Dublin, I explained why the why of Drupal is so important. After one year of Drupal 8, it's clear how powerful our collective purpose, projects, and passions can be.

Thank you to everyone who has continued to contribute to Drupal 8! I can't wait for another year of exciting projects. Special thanks to Paul Johnson for crowdsourcing great examples that I wouldn't have known about otherwise.

Content and Commerce: a big opportunity for Drupal

Last week Acquia announced a partnership with Magento. I wanted to use this opportunity to explain why I am excited about this. I also want to take a step back and share what I see is a big opportunity for both Drupal, Acquia and commerce platforms.

State of the commerce market

First, it is important to understand what is one of the most important market trends in online commerce: consumers are demanding better experiences when they shop online. In particular, commerce teams are looking to leverage vastly greater levels of content throughout the customer's shopping journey - editorials, lookbooks, tutorials, product demonstration videos, mood videos, testimonials, etc.

At the same time, commerce platforms have not added many tools for rich content management. Instead they have been investing in capabilities needed to compete in the commerce market; order management systems (OMS), omnichannel shopping (point of sale, mobile, desktop, kiosk, etc), improved product information management (PIM) and other vital commerce capabilities. The limited investment in content management capabilities has left merchants looking for better tools to take control of the customer experience, something that Drupal addresses extremely well.

To overcome the limitations that today's commerce platforms have with building content-rich shopping experiences, organizations want to integrate their commerce platform with a content management system (CMS). Depending on the situation, the combined solution is architected for either system to be "the glass", i.e. the driver of the shopping experience.

Lush.com is a nice example of a content-rich shopping experience built with Drupal and Drupal Commerce.

Drupal's unique advantage for commerce

Drupal is unique in its ability to easily integrate into ambitious commerce architectures in precisely the manner the brand prefers. We are seeing this first hand at Acquia. We have helped many customers implement a "Content for Commerce" strategy where Acquia products and Drupal were integrated with an existing commerce platform. Those integrations spanned commerce platforms including IBM WebSphere Commerce, Demandware, Oracle/ATG, SAP/hybris, Magento and even custom transaction platforms. Check out Quicken (Magento), Puma (Demandware), Motorola (Broadleaf Commerce), Tesla (custom to order a car, and Shopify to order accessories) as great examples of Drupal working with commerce platforms.

We've seen a variety of approaches to "Content for Commerce" but one thing that is clear is that a best-of-breed approach is preferred. The more complex demands may end up with IBM WebSphere Commerce or SAP/hybris. Less demanding requirements may be solved with Commerce Tools, Elastic Path or Drupal Commerce, while Magento historically has fit in between.

Additionally, having to rip and replace an existing commerce platform is not something most organizations aspire to do. This is true for smaller organizations who can't afford to replace their commerce platform, but also for large organizations who can't afford the business risk to forklift a complex commerce backend. Remember that commerce platforms have complex integrations with ERP systems, point-of-sales systems, CRM systems, warehousing systems, payment systems, marketplaces, product information systems, etc. It's often easier to add a content management system than to replace everything they have in place.

This year's "State of Retailing Online" series asked retailers and brands to prioritize their initiatives for the year. Just 16% of respondents prioritized a commerce re-platform project while 41-59% prioritized investments to evolve the customer experience including content development, responsive design and personalization. In other words, organizations are 3 times more likely to invest in improving the shopping experience than in switching commerce platforms.

The market trends, customer use cases and survey data make me believe that (1) there are hundreds of thousands of existing commerce sites that would prefer to have a better shopping experience and (2) that many of those organizations prefer to keep their commerce backend untouched while swapping out the shopping experience.

Acquia's near-term commerce strategy

There is a really strong case to be made for a best-of-breed approach where you choose and integrate the best software from different vendors. Countless point solutions exist that are optimized for narrow use cases (e.g. mobile commerce, marketplaces and industry specific solutions) as well as solutions optimized for different technology stacks (e.g. Reaction Commerce is JavaScript-based, Magento is PHP-based, Drupal Commerce is Drupal-based).

A big part of Acquia's commerce strategy is to focus on integrating Drupal with multiple commerce platforms, and to offer personalization through Lift. The partnership with Magento is an important part of this strategy, and one that will drive adoption of both Drupal and Magento.

There are over 250,000 commerce sites built with Magento and many of these organizations will want a better shopping experience. Furthermore, given the consolidation seen in the commerce platform space, there are few, proven enterprise solutions left on the market. This has increased the market opportunity for Magento and Drupal. Drupal and Magento are a natural fit; we share the same technology stack (PHP, MySQL) and we are both open source (albeit using different licenses). Last but not least, the market is pushing us to partner; we've seen strong demand for Drupal-Magento integration.

We're keen to partner with other commerce platforms as well. In fact, Acquia has existing partnerships with SAP/hybris, Demandware, Elastic Path and Commerce Tools.

Conclusion

Global brands are seeing increased opportunity to sell direct to consumers and want to build content-rich shopping journeys, and merchants are looking for better tools to take control of the customer experience.

Most organizations prefer best of breed solutions. There are hundreds of thousands of existing commerce sites that would like to have more differentiation enabled by a stronger shopping experience, yet leave their commerce capabilities relatively untouched.

Drupal is a great fit. It's power and flexibility allow it to be molded to virtually any systems architecture, while vastly improving the content experience of both authors and customers along the shopping journey. I believe commerce is evolving to be the next massive use case for Drupal and I'm excited to partner with different commerce platforms.

Special thanks to Tom Erickson and Kelly O'Neill for their contributions to this blog post.

A plan for media management in Drupal 8

Today, when you install Drupal 8.2, the out-of-the-box media handling is very basic. For example, you can upload and insert images in posts using a WYSIWYG editor, but there is no way to reuse files across posts, there is no built-in media manager, no support for "remote media" such as YouTube videos or tweets, etc. While all of these media features can be added using contributed modules, it is not ideal.

This was validated by my "State of Drupal 2016 survey" which 2,900 people participated in; the top two requested features for the content creator persona are richer image and media integration and digital asset management (see slide 44 of my DrupalCon New Orleans presentation).

This led me to propose a "media initiative" for Drupal 8 at DrupalCon New Orleans. Since then a dedicated group of people worked on a plan for the Drupal 8 media initiative. I'm happy to share that we now have good alignment for that initiative. We want to provide extensible base functionality for media handling in core that supports the reuse of media assets, media browsing, and remote media, and that can be cleanly extended by contributed modules for various additional functionality and integrations. That is a mouthful so in this blog post, I'll discuss the problem we're trying to solve and how we hope to address that in Drupal 8.

Problem statement

While Drupal core provides basic media capabilities, contributed modules have to be used to meet the media management requirements of most websites. These contributed modules are powerful — look at Drupal's massive adoption in the media and entertainment market — but they are also not without some challenges.

First, it is hard for end-users to figure out what combination of modules to use. Even after the right modules are selected, the installation and configuration of various modules can be daunting. Fortunately, there are a number of Drupal distributions that select and configure various contributed modules to offer better out-of-the-box experience for media handling. Acquia maintains the Lightning distribution as a general purpose set of components including media best practices. Hubert Burda Media built the Thunder distribution and offers publishers strong media management capabilities. MD Systems created the NP8 distribution for news publishers which also bundles strong media features. While I'm a big believer in Drupal distributions, the vast majority of Drupal sites are not built with one of these distributions. Incorporating some of these media best practices in core would make them available to all end-users.

Second, the current situation is not ideal for module developers either. Competing solutions and architectures exist for how to store media data and how to display a library of the available media assets. The lack of standardization means that developers who build and maintain media-related modules must decide which of the competing approaches to integrate with, or spend time and effort integrating with all of them.

The current plan

In a way, Drupal's media management today is comparable to the state of multilingual in Drupal 7; it took 22 or more contributed modules to make Drupal 7 truly multilingual and some of those provided conflicting solutions. Multilingual in Drupal 7 was challenging for both end-users and developers. We fixed that in Drupal 8 by adding a base layer of services in Drupal 8 core, while contributed modules still cover the more complex scenarios. That is exactly what we hope to do with media in a future version of Drupal 8.

The plan for the Drupal 8 media initiative is to provide extensible base functionality for media handling in core that supports the reuse of media assets, media browsing, and remote media, and that can be cleanly extended by contributed modules for various additional functionality and integrations.

In order to do so, we're introducing a media entity type which supports plugins for various media types. We're currently aiming to support images and YouTube videos in core, while contributed modules will continue to provide more, like audio, Facebook, Twitter, etc. To facilitate media reuse, WYSIWYG image embedding will be rebuilt using media entities and a media library will be included to allow selecting from pre-existing media.

We consider this functionality to be the minimum viable product for media in Drupal 8 core. The objective is to provide a simple media solution to make Drupal 8 easy to use out of the box for basic use cases. This would help users of sites large and small.

Media library prototype
A work-in-progress prototype of the proposed media library.

Expected timeline and call for help

We believe this could be achieved in a relatively short time — to be included in Drupal 8.3 or Drupal 8.4 as experimental modules. To help make this happen, we are looking for organizations to help fund two dedicated code sprints. The existing contributors are doing an amazing job but dedicated in-person sprints would go a long way to make the plans actually happen. If you are willing to help fund this project, let me know! Looking to help with the implementation itself? The media team meets at 2pm UTC every Wednesday. I also recommend you follow @drupalmedia for updates.

I tried to make a list of all people and organizations to thank for their work on the media initiative but couldn't. The Drupal 8 initiative borrows heavily from years of hard work and learnings on media related modules from many people and organizations. In addition, there are many people actively working on various aspects of the Drupal 8 media initiative. Special thanks to everyone who has contributed now and in the past. Also thank you to Gábor Hojtsy, Alex Bronstein and Janez Urevc for their contributions to this blog post.

Nasdaq using Drupal 8 for new Investor Relations websites

Nasdaq using drupal

I wanted to share the exciting news that Nasdaq Corporate Solutions has selected Acquia and Drupal 8 as the basis for its next generation Investor Relations Website Platform. About 3,000 of the largest companies in the world use Nasdaq's Corporate Solutions for their investor relations websites. This includes 78 of the Nasdaq 100 Index companies and 63% of the Fortune 500 companies.

What is an IR website? It's a website where public companies share their most sensitive and critical news and information with their shareholders, institutional investors, the media and analysts. This includes everything from financial results to regulatory filings, press releases, and other company news. Examples of IR websites include http://investor.starbucks.com, http://investor.apple.com and http://ir.exxonmobil.com -- all three companies are listed on Nasdaq.

All IR websites are subject to strict compliance standards, and security and reliability are very important. Nasdaq's use of Drupal 8 is a fantastic testament for Drupal and Open Source. It will raise awareness about Drupal across financial institutions worldwide.

In their announcement, Nasdaq explained that all the publicly listed companies on Nasdaq are eligible to upgrade their sites to the next-gen model "beginning in 2017 using a variety of redesign options, all of which leverage Acquia and the Drupal 8 open source enterprise web content management (WCM) system."

It's exciting that 3,000 of the largest companies in the world, like Starbucks, Apple, Amazon, Google and ExxonMobil, are now eligible to start using Drupal 8 for some of their most critical websites. If you want to learn more, consider attending Acquia Engage in a few weeks, as Nasdaq's CIO, Brad Peterson, will be presenting.

Personalization takes flight with the new Acquia Lift

Last week, we launched a new version of Acquia Lift, our personalization tool. Acquia Lift learns about your visitors' interests, preferences and context and uses that information to personalize and contextualize their experience. After more than a year of hard work, Acquia Lift has many new and powerful capabilities. In this post, I want to highlight some of the biggest improvements.

Intuitive user experience

To begin, Acquia Lift's new user interface is based on the outside-in principle. In the case of Acquia Lift, this means that the user interface primarily takes the form of a sidebar that can slide out from the edge of the page when needed. From there, users can drag and drop content into the page and get an instant preview of how the content would look. From the sidebar, you can also switch between different user segments to preview the site for different users. Personalization rules can be configured as A/B tests, and all rules affecting a certain area of a page can easily be visualized and prioritized in context. The new user interface is a lot more intuitive.

Unifying content and customer data

Having a complete view of the customer is one of the core ideas of personalization. This means being able to capture visitor profiles and behavioral data, as well as implicit interests across all channels. Acquia Lift also makes it possible to segment and target audiences in real time based on their behaviors and actions. For example, Acquia Lift can learn that someone is more interested in "tennis" than "soccer" and will use that information to serve more tennis news.

It is equally important to have a complete view of the content and experiences that you can deliver to those customers. The latest version of Acquia Lift can aggregate content from any source. This means that the Acquia Lift tray shows you content from all your sites and not just the site you're on. You can drag content from an ecommerce platform into a Drupal site and vice versa. The rendering of the content can be done inside Drupal or directly from the content's source (in this case the ecommerce platform). A central view of all your organization's content enables marketers to streamline the distribution process and deliver the most relevant content to their customers, regardless of where that content was stored originally.

Content can also be displayed in any number of ways. Just as content in Drupal can have different "display modes" (i.e. short form, long form, hero banner, sidebar image, etc), content in Acquia Lift can also be selected for the right display format in addition to the right audience. In fact, when you connect a Drupal site to Acquia Lift, you can simply configure which "entities" should be indexed inside of Acquia Lift and which "display modes" should be available, allowing you to reuse all of your existing content and configurations. Without this capability, marketers are forced to duplicate the same piece of content in different platforms and in several different formats for each use. Building a consistent experience across all channels in a personalized way then becomes incredibly difficult to manage. The new capabilities of Acquia Lift remedy this pain point.

The best for Drupal, and beyond

We've always focused on making Acquia Lift the best personalization solution for Drupal, but we realize that customers have other technology in place. The latest version of Acquia Lift can be installed on any Drupal or non-Drupal website through a simple JavaScript tag (much like the way you might install Google Analytics). So whether it's a legacy system, a JavaScript application, a decoupled Drupal build with custom front end, or a non-Drupal commerce site, they can all be personalized and connected with Acquia Lift.

In addition, we've also taken an API-first approach. The new version of Acquia Lift comes with an open API, which can be used for tracking events, retrieving user segments in real time, and showing decisions and content inside of any application. Developers can now use this capability to extend beyond the Lift UI and integrate behavioral tracking and personalization with experiences beyond the web, such as mobile applications or email.

I believe personalization and contextualization are becoming critical building blocks in the future of the web. Earlier this year I wrote that personalization is one of the most important trends in how digital experiences are being built today and will be built in the future. Tools like Acquia Lift allow organizations to better understand their customer's context and preferences so they can continue to deliver the best digital experiences. With the latest release of Acquia Lift, we've taken everything we've learned in personalization over the past several years to build a tool that is both flexible and easy to use. I'm excited to see the new Acquia Lift in the hands of our customers and partners.

State of Drupal presentation (September 2016)

DrupalCon Dublin marked my 28th DrupalCon and #Driesnote presentation. You can watch a recording of my keynote (starting at 23:55) or download a copy of my slides here (185.4 MB).

The first half of the presentation I provided a technical update on Drupal 8. I showcased some of the big changes in Drupal 8.2 such as the settings tray, REST API improvements, migration tool improvements, and easier to use block placement. I also talked about how we've transformed Drupal 8 for continuous innovation. I'm super excited about our improved development process and release cycle, as it helps us ship innovative updates to Drupal 8 faster and with a much easier upgrade path.

The second half of the talk focused on "The why" of Drupal, and asked an important question for all of us to think about: what is Drupal's collective purpose? In addition to me talking about my own purpose, my team interviewed Drupal people around the world about their passion and purpose.


I featured a lot of interviews with Drupalists. If you're interested in viewing their individual videos, they're now available on my YouTube channel:

Keynote
© Paul Johnson

Special thanks to Ash for the amazing design work on my slides and helping with the interviews, and Paul Johnson for the nice photo of me listening to Zsófi.

Drupal's collective purpose

Purpose reflecting on journey

When I was on vacation in Italy this summer, I had no internet, which gave me a lot of time to think. Some of that time was spent reflecting on why I do what I do. I have been working on Drupal for over 15 years and on Acquia for almost 10 years. The question of what gives me meaning and purpose has changed drastically over that time.

Evolving purpose

I started Drupal because I wanted to build a website for myself and a few friends. In the early days of Drupal, I was obsessed with the code and architecture of Drupal.

Purpose blog post

As I wrote in 2006: "I focused completely and utterly on creating fewer and fewer lines of more elegant code.". I wanted Drupal to be pure. I wanted the code to be perfect. For Drupal to be architected in the right way, I had to rewrite it multiple times and strip away anything that wasn't necessary – I couldn't imagine preserving backwards compatibility as it meant we had to drag along a lot of historical baggage. My mission in the early days was to keep the platform fast, clean and on the leading edge of technology.

As time passed and Drupal started growing, my role evolved. More people became involved with Drupal, and I thought more about scaling the community, including our tools, processes and culture. I started to focus on building the Drupal Association, promoting Drupal, handling trademark issues, and last but not least, setting the overall direction of the project. In the process, I started to worry less about achieving that perfect vision and more about the health of the community and collaborating on a shared vision.

While I miss programming, I have come to accept that I can't do everything. Every day when I wake up, I decide where I want to focus my energy. My guiding principle at this time in my life is to optimize for impact. That means enabling others versus doing much programming myself.

Meaningful moments: part I

Purpose moments part

While in Italy, I decided to make a list of the moments in Drupal's history that stand out as particularly meaningful or purposeful. I started to discover some patterns in these moments, and ended up sorting them into two groups. Here is the first set:

  • When people find Drupal, and it gives them a better career path and ultimately changes their life. I got goosebumps when almost 3,000 people stood up at DrupalCon San Francisco when I asked "Please stand up if Drupal changed your life". I often talk to people that went on to make a full-time living with Drupal – or even start a Drupal business – to provide better lives for their families. Some of these stories, such as Vijaya Chandran Mani's, are deeply impactful.
  • Seeing how Drupal is used for aid relief, like in the aftermath of the 2013 tornado in Moore, Oklahoma. Members of the Drupal community worked throughout the night to create a website for victims to help each other.
  • Seeing how Drupal has made a meaningful impact on the Open Web movement. Over the last 10 years, millions of people have created Drupal sites that express their creative freedom and individuality. In recent years, I've become concerned about the Open Web's future and have spoken out on how the Drupal community is uniquely positioned to help preserve the open web. I believe it's an important mission that we should all embrace, so the original integrity and freedom of the Open Web remains intact for our children and grandchildren.

Purpose radiating circles

All of these moments suggest that my purpose is self-transcendent – I get meaning when my work matters more to others than it does to me. Organized into radiating circles, the impact on each of these groups gives me purpose: individual Drupalists, the Drupal community, Drupal end users, and the open web. This is why I've become so passionate about things like usability, internationalization and accessibility over the years.

I know it's not just me; my team interviewed many other people that have the same feelings of finding meaning when their work results in life-changing outcomes. One great example is "Franck" Seferiba Salif Soulama, who hopes that training more young people in Drupal can lift people from Burkina Faso, Africa out of poverty. He wants to provide them job opportunities so they don't have to leave their country. Other examples are Drew Gorton or Ronan Dowling. There are many people like Franck, Drew or Ronan around the world that have a positive domino effect on others.

Meaningful moments: part II

Purpose moments part

The second group of moments I wrote down weren't necessarily self-transcendent, but still gave me purpose. Here are a few examples:

  • Fundraising after the great server meltdown. In 2005, we had to raise money to buy new infrastructure for Drupal.org. We nearly had to shut down Drupal.org and could have lost everything. While it was a difficult time, this moment was especially meaningful as it helped us come together as a community.
  • Having to ask individuals to leave the project or change their behavior because their values weren't aligned with the project. While providing critique or removing someone from the project has never been never easy, I'm proud of the times we stand up for our values.
  • Getting Drupal 8 over the finish line after 4.5 years of hard work. At times, many people doubted our progress, questioned whether we were making the right decisions, and even left our project. While the development process wasn't always fun in the moment, when we did release parties around the world, we all felt a real sense of accomplishment. In the long run, we built something that will keep Drupal relevant for many years to come.

Purpose not just positive

Many of us find meaning when the hard and uncomfortable work results in life-changing outcomes for others. Not only does this type of work provide purpose, some people believe it is the recipe for success. For example, Angela Lee Duckworth's TED talk on grit applies directly to the work that is done by Drupal's maintainers.

Purpose applaud the maintainers

How do we scale purpose?

Hearing all of these inspirational stories makes me think: How we can attract more people to the project, but do so in a way that ensures we share our core values (like giving back)? While there are no straightforward answers to this question, there are many organizations that are doing great things in this area.

One example is the Drupal Campus Ambassador Program which hopes to appoint ambassadors in every university in India to introduce more students to Drupal and help them with their job search. While at Drupalcon India earlier this year, I met Rakesh James, who has personally trained 600 people on Drupal!

Another example is the Drupal apprenticeship program in the UK, which focuses on recruiting new talent to the Drupal community. Participants get an extensive Drupal bootcamp to help them with their job search. Many of these apprentices are disadvantaged young people who have great talent and aptitude, but might be lacking the traditional route or access to a meaningful career path.

I'd love to take programs like these global – they instill our values, culture and a sense of purpose to many new people. If you know of similar initiatives, or have ideas to share, please do so in the comments section.

Purpose sets drupal apart

Based on my own introspection, and hearing from amazing Drupalists from around the world, I truly believe that Drupal is fueled by a collective sense of purpose that sets us apart from other open source software communities and organizations. We need to keep this purpose in mind when we make decisions, especially when the going gets tough. What is your sense of purpose? And how can we scale it around the world?

Acquia a leader in 2016 Gartner Magic Quadrant for Web Content Management

This week Acquia was named a leader in the Gartner Magic Quadrant for Web Content Management. We're quite excited about it at Acquia as it's the third year in a row that we received this recognition from Gartner.

Organizations planning large WCM deployments rely on reports such as the Gartner Magic Quadrant for Web Content Management to give them a comprehensive view of the competitive landscape. Acquia's strong placement will cause many organizations to give Drupal a closer look for their content management projects. Acquia is the only open source company included among the six leaders, and as CMSWire's Dom Nicastro reported, "Acquia made the biggest positive move" of all the solutions ranked in Gartner's report.

What I found especially interesting in this year's report was their observation that the WCM market is beginning to split in two subcategories: the first subcategory treats the web as an ever more important channel for marketing and communications; the second subcategory is focused on more profound digital transformations, where WCM is a strategic technology to create new business models and build new digital experiences.

Drupal is used for both. Johnson & Johnson, MillerCoors and Motorola are examples of brands that are creating digital experiences to support their marketing and communications. Look to the City of Boston's work to reimagine citizen engagement, or the innovation at Hubert Burda Media to transform the publishing industry for the next digital age as examples of the latter.

The transformation of Drupal 8 for continuous innovation

In the past, after every major release of Drupal, most innovation would shift to two areas: (1) contributed modules for the current release, and (2) core development work on the next major release of Drupal. This innovation model was the direct result of several long-standing policies, including our culture of breaking backward compatibility between major releases.

In many ways, this approach served us really well. It put strong emphasis on big architectural changes, for a cleaner, more modern, and more flexible codebase. The downsides were lengthy release cycles, a costly upgrade path, and low incentive for core contributors (as it could take years for their contribution to be available in production). Drupal 8's development was a great example of this; the architectural changes in Drupal 8 really propelled Drupal's codebase to be more modern and flexible, but also came at the cost of four and a half years of development and a complex upgrade path.

As Drupal grows — in lines of code, number of contributed modules, and market adoption — it becomes harder and harder to rely purely on backward compatibility breaks for innovation. As a result, we decided to evolve our philosophy starting after the release of Drupal 8.

The only way to stay competitive is to have the best product and to help people adopt it more seamlessly. This means that we have to continue to be able to reinvent ourselves, but that we need to make the resulting changes less scary and easier to absorb. We decided that we wanted more frequent releases of Drupal, with new features, API additions, and an easy upgrade path.

To achieve these goals, we adopted three new practices:

  1. Semantic versioning: a major.minor.patch versioning scheme that allows us to add significant, backwards-compatible improvements in minor releases like Drupal 8.1.0 and 8.2.0.
  2. Scheduled releases: new minor releases are timed twice a year for predictability. To ensure quality, each of these minor releases gets its own beta releases and release candidates with strict guidelines on allowed changes.
  3. Experimental modules in core: optional alpha-stability modules shipped with the core package, which allow us to distribute new functionality, gather feedback, and iterate faster on the modules' planned path to stability.

Now that Drupal 8 has been released for about 10 months and Drupal 8.2 is scheduled to be released next week, we can look back at how this new process worked. Drupal 8.1 introduced two new experimental modules (the BigPipe module and a user interface for data migration), various API additions, and usability improvements like spell checking in CKEditor. Drupal 8.2 further stabilizes the migration system and introduces numerous experimental alpha features, including significant usability improvements (i.e. block placement and block configuration), date range support, and advanced content moderation — among a long list of other stable and experimental improvements.

It's clear that these regular feature updates help us innovate faster — we can now add new capabilities to Drupal that previously would have required a new major version. With experimental modules, we can get features in users' hands early, get feedback quickly, and validate that we are implementing the right things. And with the scheduled release cycle, we can deliver these improvements more frequently and more predictably. In aggregate, this enables us to innovate continuously; we can bring more value to our users in less time in a sustainable manner, and we can engage more developers to contribute to core.

It is exciting to see how Drupal 8 transformed our capabilities to continually innovate with core, and I'm looking forward to seeing what we accomplish next! It also raises questions about what this means for Drupal 9 — I'll cover that in a future blog post.

Can Drupal outdo native applications?

I've made no secret of my interest in the open web, so it won't come as a surprise that I'd love to see more web applications and fewer native applications. Nonetheless, many argue that "the future of the internet isn't the web" and that it's only a matter of time before walled gardens like Facebook and Google — and the native applications which serve as their gatekeepers — overwhelm the web as we know it today: a public, inclusive, and decentralized common good.

I'm not convinced. Native applications seem to be winning because they offer a better user experience. So the question is: can open web applications, like those powered by Drupal, ever match up to the user experience exemplified by native applications? In this blog post, I want to describe inversion of control, a technique now common in web applications and that could benefit Drupal's own user experience.

Native applications versus web applications

Using a native application — for the first time — is usually a high-friction, low-performance experience because you need to download, install, and open the application (Android's streamed apps notwithstanding). Once installed, native applications offer unique access to smartphone capabilities such as hardware APIs (e.g. microphone, GPS, fingerprint sensors, camera), events such as push notifications, and gestures such as swipes and pinch-and-zoom. Unfortunately, most of these don't have corresponding APIs for web applications.

A web application, on the other hand, is a low-friction experience upon opening it for the first time. While native applications can require a large amount of time to download initially, web applications usually don't have to be installed and launched. Nevertheless, web applications do incur the constraint of low performance when there is significant code weight or dozens of assets that have to be downloaded from the server. As such, one of the unique challenges facing web applications today is how to emulate a native user experience without the drawbacks that come with a closed, opaque, and proprietary ecosystem.

Inversion of control

In the spirit of open source, the Drupal Association invited experts from the wider front-end community to speak at DrupalCon New Orleans, including from Ember and Angular. Ed Faulkner, a member of the Ember core team and contributor to the API-first initiative, delivered a fascinating presentation about how Drupal and Ember working in tandem can enrich the user experience.

One of Ember's primary objectives is to demonstrate how web applications can be indistinguishable from native applications. And one of the key ideas of JavaScript frameworks like Ember is inversion of control, in which the client side essentially "takes over" from the server side by driving requirements and initiating actions. In the traditional page delivery model, the server is in charge, and the end user has to wait for the next page to be delivered and rendered through a page refresh. With inversion of control, the client is in charge, which enables fluid transitions from one place in the web application to another, just like native applications.

Before the advent of JavaScript and AJAX, distinct states in web applications could be defined only on the server side as individual pages and requested and transmitted via a round trip to the server, i.e. a full page refresh. Today, the client can retrieve application states asynchronously rather than depending on the server for a completely new page load. This improves perceived performance. I discuss the history of this trend in more detail in this blog post.

Through inversion of control, JavaScript frameworks like Ember provide much more than seamless interactions and perceived performance enhancements; they also offer client-side storage and offline functionality when the client has no access to the server. As a result, inversion of control opens a door to other features requiring the empowerment of the client beyond just client-driven interactions. In fact, because the JavaScript code is run on a client such as a smartphone rather than on the server, it would be well-positioned to access other hardware APIs, like near-field communication, as web APIs become available.

Inversion of control in end user experiences

When a user clicks a teaser image on the homepage of an Ember-enhanced Drupal.com, the page seamlessly transitions into the full content page for that teaser, with the teaser image as a reference point, even though the URL changes.

In response to our recent evaluation of JavaScript frameworks and their compatibility with Drupal, Ed applied the inversion of control principle to Drupal.com using Ember. Ed's goal was to enhance Drupal.com's end user experience with Ember to make it more application-like, while also preserving Drupal's editorial and rendering capabilities as much as possible.

Ed's changes are not in production on Drupal.com, but in his demo, clicking a teaser image causes it to "explode" to become the hero image of the destination page. Pairing Ember with Drupal in this way allows a user to visually and mentally transition from a piece of teaser content to its corresponding page via an animated transition between pages — all without a page refresh. The animation is very impressive and the animated GIF above doesn't do it full justice. While this transition across pages is similar to behavior found in native mobile applications, it's not currently possible out of the box in Drupal without extensive client-side control.

Rather than the progressively decoupled approach, which embeds JavaScript-driven components into a Drupal-rendered page, Ed's implementation inverts control by allowing Ember to render what is emitted by Drupal. Ember maintains control over how URLs are loaded in the browser by controlling URLs under its responsibility; take a look at Ed's DrupalCon presentation to better understand how Drupal and Ember interact in this model.

These impressive interactions are possible using the Ember plugin Liquid Fire. Fewer than 20 lines of code were needed to build the animations in Ed's demo, much like how SDKs for native mobile applications provide easy-to-implement animations out of the box. Of course, Ember isn't the only tool capable of this kind of functionality. The RefreshLess module for Drupal by Wim Leers (Acquia) also uses client-side control to enable navigating across pages with minimal server requests. Unfortunately, RefreshLess can't tap into Liquid Fire or other Ember plugins.

Inversion of control in editorial experiences

In CardStack Editor, an editorial interface with transitions and animations is superimposed onto the content page in a manner similar to outside-in, and the editor benefits from an in-context, in-preview experience that updates in real time.

We can apply this principle of inversion of control not only to the end user experience but also to editorial experiences. The last demos in Ed's presentation depict CardStack Editor, a fully decoupled Ember application that uses inversion of control to overlay an administrative interface to edit Drupal content, much like in-place editing.

CardStack Editor communicates with Drupal's web services in order to retrieve and manipulate content to be edited, and in this example Drupal serves solely as a central content repository. This is why the API-first initiative is so important; it enables developers to use JavaScript frameworks to build application-like experiences on top of and backed by Drupal. And with the help of SDKs like Waterwheel.js (a native JavaScript library for interacting with Drupal's REST API), Drupal can become a preferred choice for JavaScript developers.

Inversion of control as the rule or exception?

Those of you following the outside-in work might have noticed some striking similarities between outside-in and the work Ed has been doing: both use inversion of control. The primary purpose of our outside-in interfaces is to provide for an in-context editing experience in which state changes take effect live before your eyes; hence the need for inversion of control.

Thinking about the future, we have to answer the following question: does Drupal want inversion of control to be the rule or the exception? We don't have to answer that question today or tomorrow, but at some point we should.

If the answer to that question is "the rule", we should consider embracing a JavaScript framework like Ember. The constellation of tools we have in jQuery, Backbone, and the Drupal AJAX framework makes using inversion of control much harder to implement than it could be. With a JavaScript framework like Ember as a standard, implementation could accelerate by becoming considerably easier. That said, there are many other factors to consider, including the costs of developing and hosting two codebases in different languages.

In the longer term, client-side frameworks like Ember will allow us to build web applications which compete with and even exceed native applications with regard to perceived performance, built-in interactions, and a better developer experience. But these frameworks will also enrich interactions between web applications and device hardware, potentially allowing them to react to pinch-and-zoom, issue native push notifications, and even interact with lower-level devices.

In the meantime, I maintain my recommendation of (1) progressive decoupling as a means to begin exploring inversion of control and (2) a continued focus on the API-first initiative to enable application-like experiences to be developed on Drupal.

Conclusion

I'm hopeful Drupal can exemplify how the open web will ultimately succeed over native applications and walled gardens. Through the API-first initiative, Drupal will provide the underpinnings for web and native applications. But is it enough?

Inversion of control is an important principle that we can apply to Drupal to improve how we power our user interactions and build robust experiences for end users and editors that rival native applications. Doing so will enable us to enhance our user experience long into the future in ways that we may not even be able to think of now. I encourage the community to experiment with these ideas around inversion of control and consider how we can apply them to Drupal.

Special thanks to Preston So for contributions to this blog post and to Angie Byron, Wim Leers, Kevin O'Leary, Matt Grill, and Ted Bowman for their feedback during its writing.

Subscribe to Drupal

Updates from Dries straight to your mailbox