We truly live in miraculous times. Open Source is at the core of the largest organizations in the world. Open Source is changing lives in emerging countries. Open Source has changed the tide of governments around the world. And yet, Open Source can be really difficult. Open Source can be largely a thankless job. It is hard to find volunteers, it is hard to find organizations to donate time or money, it is hard to organize the community, it is hard to learn, it is hard to attract full-time contributors, and more. As the project lead for Drupal, one of the largest Open Source projects/communities in the world, I live these challenges every day. In this blog post, I will analyze the challenge with scaling Open Source communities and recommend a solution for how to build very large Open Source communities.
Open Source projects are public goods
In economic terms, for something to be a "public good", it needs to match two criteria:
- non-excludability - it is impossible to prevent anyone from consuming that good, and
- non-rivalry - consumption of this good by anyone does not reduce the benefits available to others.
Examples of public goods include street lighting, national defense, public parks, basic education, the road system, etc. By that definition, Open Source software is also a "public good": we can't stop anyone from using Open Source software, and one person benefiting from Open Source software does not reduce the benefits available to others.
The realization that Open Source is a public good is a helpful one because there has been a lot of research about how to maintain and scale public goods.
Public goods and the free-rider problem
The biggest problem with public goods is the "free rider problem". A free rider is someone who uses a public good but who does not pay anything (or pay enough) towards its cost or production. If the maintainers of a public good do not address the free-rider problem it can lead to the non-production or under-production of a public good. This is generally known as the "Tragedy of the Commons".
In Open Source, a free-rider is someone who uses an Open Source software project without contributing to it. If too few people or organizations contribute to the project, the project can become unhealthy, and ultimately could cease to exist.
The free-rider problem is typical for public goods and does not usually arise with private businesses. For example, community-maintained software like Drupal may have many free riders but proprietary competitors like Adobe or Sitecore have no problem excluding those who will not pay a license fee.
To properly understand the free-rider problem and public good provision, we need to understand both self-interest theory and the theory of collective action. I'll discuss both theories and apply them to Open Source.
Open Source contributors do amazing things. They contribute to fixing the hardest problems, they share their expertise, and more. Actions like these are often described as altruistic, in contrast to the pursuit of self-interest. In reality, generosity is often driven by some level of self-interest: we provide value to others when it benefits ourselves.
Many reasons exist why people contribute to Open Source projects; people contribute because they enjoy being part of a community of like-minded people, to hone their technical skills, to get recognition, to try and make a difference in the world, because they are paid to, or for different forms of "social capital". Often we contribute because by improving the world we are living in, we are making our world better too.
Modern economics suggest that both individuals and organizations tend to act in their own self-interest, bound by morals, ethics, the well-being of future generations and more. The theory of self-interest goes back to the writings of the old Greeks, is championed by early modern economists, and is still being adhered to by late-modern economists. It follows from the theory of self-interest that we'd see more individuals and organizations contribute if they received more benefits.
While contributing to Open Source clearly has benefits, it is not obvious if the benefits outweigh the cost. If we can increase the benefits, there is no doubt we can can attract more contributors.
Collective action theory
The theory of self-interest also applies to groups of individuals. In his seminal work on collective action and public goods, economist Mancur Olson shows that the incentive for group action diminishes as group size increases. Large groups are less able to act in their common interest than small ones because (1) the complexity increases and (2) the benefits diminish.
We see this first hand in Open Source projects. As an Open Source project grows, aspects of the development, maintenance and operation have to be transferred from volunteers to paid workers. Linux is a good example. Without Red Hat, IBM and Dell employing full-time Linux contributors, Linux might not have the strong market share it has today.
The concept of major public goods growing out of volunteer and community-based models is not new to the world. The first trade routes were ancient trackways, which citizens later developed on their own into roads suited for wheeled vehicles in order to improve commerce. Transportation was improved for all citizens, driven by the commercial interest of some. Today, we certainly appreciate that full-time government workers maintain the roads. Ditto for the national defense system, basic education, etc.
The theory of collective action also implies that as an Open Source project grows, we need to evolve how we incent contributors or we won't be able to attract either part-time volunteers or full-time paid contributors.
Solutions for the free-rider problem and collective action problem exist, and this is where Open Source can learn from public goods theory and research. The most common solution for the free-rider problem is taxation; the government mandates all citizens to help pay for the production of the public good. Taxpayers help pay for our basic education system, the road system and national defense for example. Other solutions are privatization, civic duty or legislation. These solutions don't apply to Open Source.
I believe the most promising solution for Open Source is known as "privileged groups". Privileged groups are those who receive "selective benefits". Selective benefits are benefits that can motivate participation because they are available only to those who participate. The study of collective action shows that public goods are still produced when a privileged group benefits more from the public good than it costs them to produce it.
In fact, prominent "privileged groups" examples exist in the Open Source community; Automattic is a privileged group in the WordPress community as it is in a unique position to make many millions of dollars from WordPress.com. Mozilla Corporation, the for-profit subsidiary of the Mozilla Foundation, is a privileged group as it is in a unique position to get paid millions of dollars by Google. As a result, both Automattic and Mozilla Corporation are willing to make significant engineering investments in WordPress and Mozilla, respectively. Millions of people in the world benefit from that every day.
Drupal is different from Automattic and Mozilla in that no single organization benefits uniquely from contributing. For example, my company Acquia currently employs the most full-time contributors to Drupal but does not receive any exclusive benefits in terms of monetizing Drupal. While Acquia does accrue some value from hiring the Drupal contributors that it does, this is something any company can do.
Better incentives for Drupal contributors
It's my belief that we should embrace the concept of "privileged groups" and "selective benefits" in the Drupal community to help us grow and maintain the Drupal project. Furthermore, I believe we should provide "selective benefits" in a way that encourages fairness and equality, and doesn't primarily benefit any one particular organization.
From the theory of self-interest it follows that to get more paid core contributors we need to provide more and better benefits to organizations that are willing to let their employees contribute. Drupal agencies are looking for two things: customers and Drupal talent.
Many organizations would be eager to contribute more if, in return, they were able to attract more customers and/or Drupal talent. Hence, the "selective benefits" that we can provide them are things like:
- Organizational profile pages on drupal.org with badges or statistics that prominently showcase their contributions,
- Advertising on the drupal.org in exchange for fixing critical bugs in Drupal 8 (imagine we rewarded each company that helped fix a critical Drupal 8 bug 10,000 ad views on the front page of drupal.org),
- Better visibility on Drupal.org's job board for those trying to hire Drupal developers,
- The ability to sort the marketplace by contributions, rather than just alphabetically
I'm particularly excited about providing ads in exchange for contributing. Contributing to Drupal now becomes a marketing expense; the more you contribute, the more customers you can gain from drupal.org. We can even direct resources; award more ad views in exchange for fixing UX problems early in the development cycle, but award critical bugs and beta blockers later in the development cycle. With some relatively small changes to drupal.org, hiring a full-time core developer becomes a lot more interesting.
By matching the benefits to the needs of Drupal agencies, we candirect more resources towards Drupal development. I also believe this system to be fair; all companies can choose to contribute to Drupal 8 and earn advertising credits, and all participants are rewarded equally. We can turn Drupal.org into a platform that encourages and directs participation from a large number of organizations.
Systems like this are subject to gaming but I believe these challenges can be overcome. Any benefit is better than almost no benefit. In general, it will be interesting to see if fairness and heterogeneity will facilitate or impede contribution compared to Open Source projects like WordPress and Mozilla, where some hold unique benefits. I believe that if all participants benefit equally from their contributions, they have an incentive to match each other's contributions and it will facilitate the agreement and establishment of a contribution norm that fosters both cooperation and coordination, while minimizing gaming of the system. In contrast, when participants benefit very differently, like with WordPress and Mozilla, this decreases the willingness to cooperate, which, in turn, could have detrimental effects on contributions. While not necessarily the easiest path, I believe that making the system fair and heterogeneous is the "Drupal way" and that it will serve us in the long term.
There are plenty of technical challenges ahead of us that we need to work on, fun ideas that we should experiment with, and more. With some relatively small changes, we could drastically change the benefits of contributing to Drupal. Better incentives mean more contributors, and more contributors mean that we can try more things and do things better and faster. It means we can scale Drupal development to new heights and with that, increase Open Source's impact on the world.
(I talked about this in my DrupalCon Amsterdam keynote. If you're hungry for more, I recommend that you check out my slides.)
I spent the last week in Japan. The goal was two-fold: meet with the Drupal community to understand how we can grow Drupal in Japan, and evaluate the business opportunity to incorporate an Acquia subsidiary in Japan (we already offer Acquia Cloud in Japan using Amazon's Tokyo data center).
I presented at two Drupal meetups in Japan; spent the week meeting with members of the Drupal community, Drupal agencies, large system integrators (IBM, Accenture, Hitachi, Fujitsu, Ci&T and SIOS) and the Japanese government. In between meetings, I enjoyed the amazing food that Japan has to offer.
The community in Japan is healthy; there are some noteworthy Japanese Drupal sites and there are passionate leaders that organize meetups and conferences. The Japanese Drupal community is bigger than the Chinese Drupal community but compared to North America and Europe, the Japanese Drupal community is relatively small; the largest Drupal agency I met with employs 20 developers.
The large system integrators, with the exception of Ci&T, have not done any Drupal projects in Japan. We're way behind our competitors like Sitecore, Adobe Experience Manager and SDL in this regard. All of them enabled the large system integrators to sell and use their products. It was great to meet with all the system integrators to make them aware of Drupal, and the potential it could have to their business. It's clear the large system integrators could benefit from an Open Source platform that allows them to move faster and integrate with more systems.
The biggest challenge is the lack of Japanese documentation; both marketing materials as well as developer documentation. Most of the Japanese do not have much confidence in their English speaking ability and struggle to use Drupal or to participate on drupal.org. My recommendation for the Japanese Drupal community is to organize regular translation sprints. Translating one or more of the best-selling English Drupal books to Japanese could also be a game-changer for the community.
Another problem has been the historic challenges with drupal.jp. The anonymous owner of the domain drupal.jp claims that drupal.jp is the official Drupal site in Japan (it's not officially approved) and runs it without much regard or consultation with the broader Japanese Drupal community. I promised the Japanese community to help fix this.
I returned from my trip feeling that the Japanese market offers a great opportunity for Drupal. Japan is the world's third-largest economy, after the United States and China. With continued leadership, Drupal could be huge in Japan. I’d love that, as I would like to go back and visit Japan again.
I just spent the past week in China, and I thought I'd share a few reflections on the state of Drupal in China.
First, let me set the stage. There are 1.35 billion people living in China; that is almost 20 percent of the world's population. Based on current trends, China's economy will overtake the US within the next few years. At that point, the US economy will no longer be the largest economy in the world. China's rapid urbanization is what has led to the country's impressive economic growth over the past couple of decades and it doesn't look like it is going to stop anytime soon. To put that in perspective: China currently produces and uses 60 percent of the world's cement.
In terms of Drupal, the first thing I learned is that "Drupal" sounds like "the pig is running" ("Zhu Pao") in Chinese. Contrary to a pig's rather negative reputation in the West, many Chinese developers find that cute. A pig is a more honorable sign in Chinese astrology and culture. Phew!
In terms of adoption, it feels like the Drupal community in China is about 8 to 10 years behind compared to North America or Europe. That isn't a surprise, as Open Source software is a more recent phenomenon in China than it is in North America or Europe.
Specifically, there are about 5 Drupal companies in Shanghai (population of 21 million people), 3 Drupal companies in Beijing (population of 23 million people) and 5 Drupal companies in Hong Kong (population of 7 million people). The largest Drupal companies in China have about 5 Drupal developers on staff. Four of the 5 Shanghai companies are subsidiaries from European Drupal companies. The exception is Ci&T, which has 40 Drupal developers in China. Ci&T is a global systems integrator with several thousand employees worldwide, so unlike the other companies I met, they are not a pure Drupal play. Another point of reference is that the largest Drupal event in China attracted 200 to 300 attendees.
Given that China has 4 times the population of the US, or 2 times the population of Europe, what are we missing? In talking to different people, it appears the biggest barrier to adoption is language. The problem is that Chinese Drupal documentation is limited; translation efforts exist but are slow. The little documentation that is translated is often outdated and spread out over different websites. Less than 20 percent of the Chinese Drupal developers have an account on Drupal.org, simply because they are not fluent enough in the English language. Most Drupal developers hang out on QQ, an instant messaging tool comparable to Skype or IRC. I saw QQ channels dedicated to Drupal with a couple thousand of Drupal developers.
There is no prominent Chinese content management system; most people appear to be building their websites from scratch. This gap could provide a big opportunity for Drupal. China's urbanization equals growth -- and lots of it. Like the rest of the economy, Drupal and Open Source could be catching up fast, and it might not take long before some of the world's biggest Drupal projects are delivered from China.
Supporting Drupal's global growth is important so I'd love to improve Drupal's translation efforts and make Drupal more inclusive and more diverse. Drupal 8's improved multilingual capabilities should help a lot, but we also have to improve the tools and processes on Drupal.org to help the community maintain multi-lingual documentation. Discussing this with both the Drupal Association and different members of our community, it's clear that we have a lot of good ideas on what we could do but lack both the funding and resources to make it happen faster.
After spending the summer in Boston, I'm ready to fly across the world ... literally, as I'm leaving on a two week trip to China and Japan later this week. I'm very excited about it as I've never had the opportunity to see either of these countries.
I will arrive in Beijing on Saturday, September 6th, for the Young Global Leaders Annual Summit. A private path from where I'll be staying leads to a non-restored section of the epic Great Wall of China. Exploring this truly untouched piece of Chinese history still in its original landscape should be a special experience! Stay tuned for photos.
Following my time in Beijing, I'll transfer to Tianjin to attend Summer Davos. In addition to that, we're organizing a meetup with the local Drupal community - https://groups.drupal.org/node/434658. If you are in the area on September 10th, please stop by for a drink. I'd love to meet you and learn about the state of Drupal in China.
I'll end my two week trip in Tokyo, Japan. My time will be split between meeting the local Drupal community - https://groups.drupal.org/node/440198, understanding the adoption rate of Drupal in the Japanese market, and attending private meetings with digital agencies, Acquia partners and others to learn about the state of the web and digital in Japan.
Xièxiè and dōmo arigatō to those that have helped plan these events and gather the Drupal community for some fun evenings!
If you aren't able to make either Drupal meetup, feel free to leave your thoughts in the comments.
I have an idea but currently don't have the time or resources to work on it. So I'm sharing the idea here, hoping we can at least discuss it, and maybe someone will even feel inspired to take it on.
The idea is based on two predictions. First, I'm convinced that the future of web sites or web applications is component-based platforms (e.g. Drupal modules, WordPress plugins, etc). Second, I believe that the best way to deploy and use web sites or web applications is through a SaaS hosting environment (e.g. WordPress.com, DrupalGardens, SalesForce's Force.com platform, DemandWare's SaaS platform, etc). Specifically, I believe that in the big picture on-premise software is a "transitional state". It may take another 15 years, but on-premise software will become the exception rather than the standard. Combined, these two predictions present a future where we have component-based platforms running in SaaS environments.
To get the idea, imagine a WordPress.com, SquareSpace, Wix or DrupalGardens where you can install every module/plugin available, including your own custom modules/plugins, instead of being limited to those modules/plugins manually approved by their vendors. This is a big deal because one of the biggest challenges with running web sites or web applications is that almost every user wants to extend or customize the application beyond what is provided out of the box.
Web applications have to be (1) manageable, (2) extensible, (3) customizable and (4) robust. The problem is that we don't have a programming language or an execution runtime that is able to meet all four of these requirements in the context of building and running dynamic component-based applications.
What we really need here is an execution runtime that allows you to install untrusted components and guarantee application robustness at the same time. Such technology would be a total game-changer as we could build unlimited customizable SaaS platforms that leverage the power of community innovation. You'd be able to install any Drupal module on DrupalGardens, any plugin on WordPress.com or custom code on Squarespace or Wix. It would fundamentally disrupt the entire industry and would help us achieve the assembled web dream.
I've been giving this some thought, and what I think we need is the ability to handle each HTTP request in a micro-kernel-like environment where each software component (i.e. Drupal module, WordPress plugin) runs in its own isolated process or environment and communicates with the other components through a form of inter-process communication (i.e. think remote procedure calls or web service calls). It is a lot harder to implement than it sounds as the inter-process communication could add huge overhead (e.g. we might need fast or clever ways to safely share data between isolated components without having to copy or transfer a lot of data around). Alternatively, virtualization technology like Docker might help us move in this direction as well. Their goal of a lightweight container is a step towards micro-services but it is likely to have more communication overhead. In both scenarios, Drupal would look a lot like a collection of micro web services (Drupal 10 anyone?).
Once we have such a runtime, we can implement and enforce governance and security policies for each component (e.g. limit its memory usage, limit its I/O, security permission, but also control access to the underlying platform like the database). We'd have real component-based isolation along with platform-level governance: (1) manageable, (2) extensible, (3) customizable and (4) robust.
Food for thought and discussion?