A better runtime for component-based web applications

I have an idea but currently don't have the time or resources to work on it. So I'm sharing the idea here, hoping we can at least discuss it, and maybe someone will even feel inspired to take it on.

The idea is based on two predictions. First, I'm convinced that the future of web sites or web applications is component-based platforms (e.g. Drupal modules, WordPress plugins, etc). Second, I believe that the best way to deploy and use web sites or web applications is through a SaaS hosting environment (e.g. WordPress.com, DrupalGardens, SalesForce's Force.com platform, DemandWare's SaaS platform, etc). Specifically, I believe that in the big picture on-premise software is a "transitional state". It may take another 15 years, but on-premise software will become the exception rather than the standard. Combined, these two predictions present a future where we have component-based platforms running in SaaS environments.

To get the idea, imagine a WordPress.com, SquareSpace, Wix or DrupalGardens where you can install every module/plugin available, including your own custom modules/plugins, instead of being limited to those modules/plugins manually approved by their vendors. This is a big deal because one of the biggest challenges with running web sites or web applications is that almost every user wants to extend or customize the application beyond what is provided out of the box.

Web applications have to be (1) manageable, (2) extensible, (3) customizable and (4) robust. The problem is that we don't have a programming language or an execution runtime that is able to meet all four of these requirements in the context of building and running dynamic component-based applications.

Neither PHP, JavaScript, Ruby, Go or Java allow us to build truly robust applications as the runtimes don't provide proper resource isolation. Often all the components (i.e. Drupal modules, WordPress plugins) run in the same memory space. In the Java world you have Enterprise Java Beans or OSGi which add some level of isolation and management, but it still doesn't provide full component-level isolation or component-level fault containment. As a result, it is required that one component pretty much trusts the other components installed on the system. This means that usually one malfunctioning component can corrupt the other component's data or functional logic, or that one component can harm the performance of the entire platform. In other words, you have to review, certify and test components before installing them on your platform. As a result, most SaaS vendors won't let you install untrusted or custom components.

What we really need here is an execution runtime that allows you to install untrusted components and guarantee application robustness at the same time. Such technology would be a total game-changer as we could build unlimited customizable SaaS platforms that leverage the power of community innovation. You'd be able to install any Drupal module on DrupalGardens, any plugin on WordPress.com or custom code on Squarespace or Wix. It would fundamentally disrupt the entire industry and would help us achieve the assembled web dream.

I've been giving this some thought, and what I think we need is the ability to handle each HTTP request in a micro-kernel-like environment where each software component (i.e. Drupal module, WordPress plugin) runs in its own isolated process or environment and communicates with the other components through a form of inter-process communication (i.e. think remote procedure calls or web service calls). It is a lot harder to implement than it sounds as the inter-process communication could add huge overhead (e.g. we might need fast or clever ways to safely share data between isolated components without having to copy or transfer a lot of data around). Alternatively, virtualization technology like Docker might help us move in this direction as well. Their goal of a lightweight container is a step towards micro-services but it is likely to have more communication overhead. In both scenarios, Drupal would look a lot like a collection of micro web services (Drupal 10 anyone?).

Once we have such a runtime, we can implement and enforce governance and security policies for each component (e.g. limit its memory usage, limit its I/O, security permission, but also control access to the underlying platform like the database). We'd have real component-based isolation along with platform-level governance: (1) manageable, (2) extensible, (3) customizable and (4) robust.

Food for thought and discussion?

Open Source and social capital

The notion that people contributing to Open Source don't get paid is false. Contributors to Open Source are compensated for their labor; not always with financial capital (i.e. a paycheck) but certainly with social capital. Social capital is a rather vague and intangible concept so let me give some examples. If you know someone at a company where you are applying for a job and this connection helps you get that job, you have used social capital. Or if you got a lead or a business opportunity through your network, you have used social capital. Or when you fall on hard times and you rely on friends for emotional support, you're also using social capital.

The term "social" refers to the fact that the value is in the network of relationships; they can't be owned like personal assets. Too many people believe that success in life is based on the individual, and that if you do not have success in life, there is no one to blame but yourself. The truth is that individuals who build and use social capital get better jobs, better pay, faster promotions and are more effective compared to peers who are not tapping the power of social capital. As shown in the examples, social capital also translates into happiness and well-being.

Most Open Source contributors benefit from social capital but may not have stopped to think about it, or may not value it appropriately. Most of us in the Open Source world have made friendships for life, have landed jobs because of our contributions, others have started businesses together, and for others it has provided an important sense of purpose. Once you become attuned to spotting social capital being leveraged, you see it everywhere, every day. I could literally write a book filled with hundreds of stories about how contributing to Open Source changed people's lives -- I love hearing these stories.

Social capital is a big deal; it is worth understanding, worth talking about, and worth investing in. It is key to achieving personal success, business success and even happiness.

To bean or not to bean?

Last weekend our Nespresso machine died in front of my eyes. Water started leaking from its base during use and it shorted the electricity. It was a painful death. I'm tempted to take it apart and try to repair it but it also brings up the question; what to buy next?

Part of me enjoys the convenience of the Nespresso machine, the other part of me is eager to buy my first "serious" espresso machine.

See, I'm a coffee lover. That is to say, like most of the people living in the US I have a coffee addiction, and have been brainwashed into spending more and more on my daily coffee intake. To make matters worse, we live in a society where we call the people who make great coffee "artists". I'd love to practice some coffee artistry myself and make that perfect barista-grade cup of coffee.

I did a little bit of research and picking an espresso machine is not easy. It turns out this is a complex space. The choices range from super-automatic machines (e.g. they do everything from grinding, dosing, tamping to brewing) to semi-automatic machines (e.g. you manually grind your own beans and tamp them) to manual machines (e.g. you control how long the brewing water sits over the bed of coffee, resting as it were at neutral or boiler pressure). There are even "coffee schools" that offer classes and certifications to become a professional barista.

While I love the smell of fresh ground coffee and an above perfection espresso, I also don't want to take 15 minutes to make a cup of coffee. I usually need my first cup of coffee to help me wake up and I'm often crunched for time, so I don't want it to be super complicated.

Espresso or Nespresso? To bean or not to bean? Help!

Drupal.com refresh launched

Back in the early days of Drupal, Drupal.com looked like this:

Drupal.com in 2005

Drupal.com as launched in 2005.

On August 14 2009, I relaunched Drupal.com to replace the oh-so-embarrassing placeholder page. The 2009 re-launch turned Drupal.com into a better spotlight for Drupal. It wasn't hard to beat the white page with a Druplicon logo.

Drupal.com in 2009

Drupal.com as launched in 2009.

What was a good spotlight five years ago though is no longer a good spotlight today. Five years later, Drupal.com didn't do Drupal justice. It didn't really explain what Drupal is, what you can use Drupal for, and more. Along with sub-optimal content, the site wasn't optimized for mobile use either.

Today, exactly five years later to the day, I'm excited to announce that I relaunched Drupal.com again:

Drupal com devices

Redesigning Drupal.com to make it more useful and current has been one of my New Year's resolutions for a number of years now. And as of today, I can finally strike that off my list.

The new Drupal.com has become richer in its content; you'll find a bit more information about Drupal to help people understand what Drupal is all about and how to get started with Drupal. On a desktop, on a tablet, on a phone, the site has become much easier to navigate and read.

I believe the new Drupal.com is a much better, more relevant showcase for Drupal. The goal is to update the site more regularly and to keep adding to it. My next step is to add more use cases and to include short demo videos of both the Drupal backend as well as the showcases. Drupal.com will become an increasingly helpful resource and starting point for people who are evaluating Drupal.

Drupal com mobile

The changes are not limited to content and look; Drupal.com also has a new engine as the site was upgraded from Drupal 6 to Drupal 8 alpha (don't try this at home). We're using Drupal 8 to push the boundaries of site building and responsive design and to uncover bugs and usability issues with Drupal 8. Because we're using an alpha version of Drupal 8, things might not function perfectly yet. We’d still love to hear feedback from designers and front end developers on how it’s working.

Amazon invests in Acquia

I'm happy to share news that Amazon has joined the Acquia family as our newest investor. This investment builds on the recent $50 million financing round that Acquia completed in May, which was led by New Enterprise Associates (NEA).

Acquia is the largest provider of Drupal infrastructure in the world. We run on more than 8,000 AWS instances and serve more than 27 billion hits a month or 333 TB of bandwidth a month. Working with AWS has been an invaluable part of our success story, and today's investment will further solidify our collaboration.

We did not disclose the amount of the investment in today's news announcement.

Help me write my DrupalCon Amsterdam keynote

For my DrupalCon Amsterdam keynote, I want to try something slightly different. Instead of coming up with the talk track myself, I want to "crowdsource" it. In other words, I want the wider Drupal community to have direct input on the content of the keynote. I feel this will provide a great opportunity to surface questions and ideas from the people who make Drupal what it is.

In the past, I've done traditional surveys to get input for my keynote and I've also done keynotes that were Q&A from beginning to end. This time, I'd like to try something in between.

I'd love your help to identify the topics of interests (e.g. scaling our community, future of the web, information about Drupal's competitors, "headless" Drupal, the Drupal Association, the business of Open Source, sustaining core development, etc). You can make your suggestions in the comments of this blog post or on Twitter (tag them with @Dries and #driesnote). I'll handpick some topics from all the suggestions, largely based on popularity but also based on how important and meaty I think the topic is.

Then, in the lead-up to the event, I'll create discussion opportunities on some or all of the topics so we can dive deeper on them together, and surface various opinions and ideas. The result of those deeper conversations will form the basis of my DrupalCon Amsterdam keynote.

So what would you like me to talk about? Suggest your topics in the comments of this blog post or on Twitter by tagging your suggestions with #driesnote and/or @Dries. Thank you!

The business behind Open Source

A few days ago, I sat down with Quentin Hardy of The New York Times to talk Open Source. We spoke mostly about the Drupal ecosystem and how Acquia makes money. As someone who spent almost his entire career in Open Source, I'm a firm believer in the fact that you can build a high-growth, high-margin business and help the community flourish. It's not an either-or proposition, and Acquia and Drupal are proof of that.

Rather than an utopian alternate reality as Quentin outlines, I believe Open Source is both a better way to build software, and a good foundation for an ecosystem of for-profit companies. Open Source software itself is very successful, and is capable of running some of the most complex enterprise systems. But failure to commercialize Open Source doesn't necessarily make it bad.

I mentioned to Quentin that I thought Open Source was Darwinian; a proprietary software company can't afford to experiment with creating 10 different implementations of an online photo album, only to pick the best one. In Open Source we can, and do. We often have competing implementations and eventually the best implementation(s) will win. One could say that Open Source is a more "wasteful" way of software development. In a pure capitalist read of On the Origin of Species, there is only one winner, but business and Darwin's theory itself is far more complex. Beyond "only the strongest survive", Darwin tells a story of interconnectedness, or the way an ecosystem can dictate how an entire species chooses to adapt.

While it's true that the Open Source "business model" has produced few large businesses (Red Hat being one notable example), we're also evolving the different Open Source business models. In the case of Acquia, we're selling a number of "as-a-service" products for Drupal, which is vastly different than just selling support like the first generation of Open Source companies did.

As a private company, Acquia doesn't disclose financial information, but I can say that we've been very busy operating a high-growth business. Acquia is North America's fastest growing private company on the Deloitte Fast 500 list. Our Q1 2014 bookings increased 55 percent year-over-year, and the majority of that is recurring subscription revenue. We've experienced 21 consecutive quarters of revenue growth, with no signs of slowing down. Acquia's business model has been both disruptive and transformative in our industry. Other Open Source companies like Hortonworks, Cloudera and MongoDB seem to be building thriving businesses too.

Society is undergoing tremendous change right now -- the sharing and collaboration practices of the internet are extending to transportation (Uber), hotels (Airbnb), financing (Kickstarter, LendingClub) and music services (Spotify). The rise of the collaborative economy, of which the Open Source community is a part of, should be a powerful message for the business community. It is the established, proprietary vendors whose business models are at risk, and not the other way around.

Hundreds of other companies, including several venture backed startups, have been born out of the Drupal community. Like Acquia, they have grown their businesses while supporting the ecosystem from which they came. That is more than a feel-good story, it's just good business.

Pages

© 1999-2014 Dries Buytaert Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License.
Drupal is a Registered Trademark of Dries Buytaert.