Dries Buytaert

For four years now I've been an advisor for Akiban, a Boston start-up building a new class of NewSQL/NoSQL database. I'm excited that after 4 years of hard work, Akiban launched their first Drupal customer solution in the Acquia Cloud. A great opportunity to talk a bit more about what Akiban is doing, and why I'm excited to help their team.

The early phase strategy for Akiban is to augment existing deployments (for example MySQL) to enhance query performance among other capabilities. Our mutual customer was facing performance, concurrency and availability challenges with some custom Drupal report code. The report was built in Drupal as a module, and involved a series of complex joins making performance unpredictable, frequently resulting in slow query performance and periodically crashing the whole site. Using Akiban's database, the customer is realizing 66x performance improvement over their existing implementation, without any significant change to the Drupal application.

One of the core benefits of Akiban is query acceleration. The Akiban database can run along side of MySQL server in "augmentation mode" comparable to master-slave configuration. Akiban implemented a simple Drupal patch which allows the reporting queries to be redirected to the Akiban server. While Akiban’s solution requires data duplication, it also means that there is virtually no intrusion on the day-to-day running of the site.

The report module remains as originally designed but now the problem queries are redirected to the Akiban server. Akiban’s core technology is called Table Grouping. Table Grouping enables for the physical grouping of tables while preserving a logical layer allowing developers to continue to use SQL. This grouping eliminates complex traditional joins while preserving the use of ANSI SQL. In addition, Akiban can create cross-table indexes thus accelerating formerly slow queries. As a result, with the reporting queries now directed to Akiban server, the report performs 66x faster.

The Akiban team refers to Akiban server as a new class of database that accelerates SQL and NoSQL data by 10-100x, while allowing developers to access data in both traditional SQL and RESTful environments (SOAs). Compared to other database technologies, Table Grouping provides an innovative way to store and query structured and semi-structured data.

Akiban's Padraig O'Sullivan is working on a module for Drupal 7, and while there is still some work to be done to test and optimize it, he has already enabled Akiban to run as the source database for Drupal 8 in development. Something to keep an eye on. If you want to test out Akiban yourself, head over to akiban.com and download it.

The following blog post was published as a guest blog post on Forbes.com. I wrote it after Al Jazeera successfully moved some of their Drupal sites from their traditional hosting company to Acquia Hosting (now called Acquia Managed Cloud) to help them survive a 2,000% traffic increase as a result of the crises in the Middle East. The blog post provides real proof of how the Cloud helped one of the largest news organizations in the world survive one of the largest political events in the world. A fascinating story for Drupal!

Over the past decade, the Web has completely transformed how people create and consume information. We have all witnessed firsthand how the free flow of information is impacting the way individuals and companies communicate and how the rules of governance are changing for entire nations. Now, we’re all participating and reporting on events as they happen, and from where they happen.

There is no better example of that than the most recent events in the Middle East. And one organization, Al Jazeera, the world’s largest news organization solely focused on the Middle East, was right in the middle of the incredible broadcast and social media storm that instantly developed. Throughout the ordeal, Al Jazeera effectively leveraged the power of the cloud to stay on the air and scale its reach and performance. If events of the past few months are any indication, there are lessons here for other content-driven companies to consider for their own online operations.

Al Jazeera’s English operations broadcasts news and current affairs 24 hours a day, 7 days a week with more than 1,000 staff members from more than 50 nations. Quite literally, Al Jazeera provides the world with a front seat on the Middle East stage. It broadcasts from centers in Doha, the capital city of the state of Qatar, Kuala Lumpur, London and Washington.

Al Jazeera’s live blog site is powered by Drupal, a free, open source social publishing platform that enables content-driven organizations to publish content and build communities quickly and easily. Drupal is used by many of the world’s most prominent organizations including the White House, the World Economic Forum, Intel, The Economist and Turner Broadcasting.

Al Jazeera’s English live blog site was a vital source for breaking news in Egypt. Bloggers were posting updates from the epicenter of the crisis and social media was often the only means of communication both inside and outside of the country. During the crisis, traffic to the Al Jazeera web site increased 1,000% and traffic to the live blog spiked 2,000%. This dilemma, normally a good one for news organizations, caused unpredictable performance and excessive page load times for site visitors.

From an infrastructure standpoint, Al Jazeera had historically hosted its blog with a traditional provider but had increasingly suffered a variety of scalability issues brought on by surging demand – unacceptable for Al Jazeera or any similar content business. What might have been just a typical technical nuisance on a mundane news day quickly became unsustainable when Egypt erupted.

Al Jazeera faced a mission-critical problem that needed a real-time solution. Where could it find performance hosting and support immediately and within a reasonable cost? Would it be secure and private? What about reliable? The answer: The cloud, the various data access, storage and hosting services available remotely over the Internet. Much discussed but often not fully appreciated by the business community, cloud services enable custom sites to perform well under varying, and sometimes severe, traffic conditions. Moving to a Drupal-supported cloud option allowed Al Jazeera to scale up quickly, dynamically render their content faster, and achieve a higher level of site reliability – issues that previously overwhelmed its physical hardware environments.

By leveraging Drupal and turning to the cloud, the Al Jazeera technical team demonstrated how to rapidly turn a seemingly disastrous situation into a net positive business decision going forward. Fast forward a few weeks, and the demands on Al Jazeera’s Web infrastructure have only increased with new crises across the region. The difference is the organization is now able to better handle these unforeseen demands and focus on the core business, reporting the news as it happens.

It has been only 18 months since we launched Acquia Hosting. Today, Acquia Hosting is serving 2.2 billion page views a month for several hundred customers and growing rapidly. In those 18 months, we also built a lot of tools to help organizations manage their entire Drupal workflow process from development to staging and production. The combined result is that we've built one of the world's best managed cloud environments for Drupal, and have proven that we can scale any Drupal site to meet the needs of the largest enterprises.

During this time, we received a lot of demand for a lower-priced single-server version of Acquia Hosting that developers could use to build sites that didn’t require multi-server high-availability. We held off offering a single-server version of our offering so we could focus on solving the largest Drupal scalability issues for the most demanding enterprises. We're proud to announce that we're ready, and that we will offer a developer-focused solution in the next few weeks. We're ready to show you at DrupalCon Chicago and will launch it shortly thereafter.

Like Acquia Hosting, the new single-server offering will come with an optimized stack for Drupal, including Varnish, Nginx, APC and Memcache, as well as all the tools to manage the lifecycle of your website. This new hosting offering will be called Acquia Dev Cloud. It will be great for people that want to build Drupal sites without the need for high-availability or help from Acquia to help scale and maintain their site. In many ways, Dev Cloud is the logical next step for Acquia Hosting, and helps us close the chasm between Drupal Gardens and Acquia Hosting.

One of the key features of Dev Cloud is the ability to run multiple separate sites on a single server. This has been the most common request from our partners since we launched Acquia Hosting. Because many of the sites our partners build are small, it’s more cost effective to run multiple sites on a single server and partners wanted this option on our hosting platform.

It was also clear that 'Acquia Hosting' was no longer the best name -- or maybe it never was. The more features we added, the more we grew to dislike it as the less it reflected what we had build. The amount of developer tools and the level of service and support we provide to help customers scale their Drupal sites is unheard of. None of that is reflected in the name 'Acquia Hosting'. Too many people had preconceived notions of what 'hosting' meant, and what we offer goes well beyond traditional hosting. It is really Platform-as-a-Service (or PaaS). Hence, we decided to rename 'Acquia Hosting' to 'Acquia Managed Cloud' to better reflect that fact.

Acquia Managed Cloud is for business-critical sites that require a fully redundant, highly available environment that is managed 24x7 by Acquia staff. Acquia Dev Cloud is for developers that want all the great features of Acquia Managed Cloud, without the support and guarantees provided by our team of Drupal experts.

We're very excited about these changes as they get us a step closer to completing our vision for Acquia. It will be easy to migrate from Drupal Gardens to Dev Cloud, or from Dev Cloud to Managed Cloud.

Dev Cloud will be available for purchase as part of an Acquia Network subscription. We are introducing a new Developer Subscription package that includes access to Dev Cloud, self-service support tools, our new Acquia Library knowledge base and Acquia Network services including Acquia Search, Mobify, and New Relic. This will start at a developer friendly price of less than $200 / month including a Dev Cloud server. If you are looking for support ticket access to Acquia’s support team, you can also use Dev Cloud with our Professional Subscription.

In my Acquia 2010 retrospective, I promised to write a bit more about Acquia's product strategy. This blog post provides a high level view of the vision that we've been working towards for the last 3 years, and explains how Acquia can help simplify your web strategy.

The web: it's currently a mess

Ten years ago, the average organization had one website. Since then, doing business through the web has become more complex and have introduced a diverse set of needs. If you're like most organizations the number of sites you have is large and continues to grow at a rapid clip.

For most organizations, one tool could not historically get the job done, so they kept multiple tools in their toolbox – whether they intended to or not. The situation can be quite a mess, and is unfortunately a common scenario in many enterprises.

Each site has unique needs

Most of these sites are vastly different in terms of scale, functionality, complexity and longevity. Some sites are under continuous development while other sites are only around for a couple of weeks or months. Some of the websites are owned by the company's IT department and hosted internally, while other websites may be owned by their marketing department and hosted externally. As a result, the level of investment and the time to market requirements are usually very different.

Standardize on Drupal to save costs

CIOs – facing cost-cutting pressures and the need to streamline their resources – are now addressing the reality of running twenty different content management systems on twenty different stack configurations as an expensive, unnecessary burden for the organization. They have always known that there were cost savings to be made if they standardize on a single platform, but have never felt the confidence in a single platform to suit all of their needs across their organization.

Drupal has the required features to accomplish this today. This is more than a vision – it is reality. Every day, more organizations are standardizing on Drupal.

By standardizing on Drupal, organizations can reduce training costs, reduce maintenance costs, streamline security, and optimize internal resources – all without sacrificing quality or requirements. Standardizing on Drupal certainly doesn't mean every single system needs to be Drupal. Even going from 20 different systems to 10 or to 5 different systems still translates to dramatic cost savings. It goes without saying that you need to be smart about what makes sense to standardize on Drupal, and what not to standardize on Drupal. With our vast community of contributors, Drupal continues to become better and better and the feasibility for an organization to standardize on Drupal continues to improve over time.

Drupal distributions help adoption

Drupal distributions are an important part of helping organizations adopt Drupal. Drupal distributions are complete, ready-to-use solutions built on Drupal. Just install and go.

Drupal Commons is a Drupal distribution for social business software; it provides organizations a complete solution for forming collaborative communities. Similarly, Open Publish is a Drupal distribution optimized for news publishing. Acquia sees expansion of distributions as critically important to the future growth of Drupal. With that, we are acting as a software publishers for these and other distributions developed by partners within the Drupal community; supporting the marketing, promotion, support, and ongoing development of distributions to extend the capability of the companies who have incubated these incredible products.

Add the Acquia Network for support and cloud services

To help organizations adopt and standardize on Drupal, we created the Acquia Network to provide a suite of Drupal support, knowledge, and web development and maintenance tools to help build, manage and extend Drupal websites.

The Acquia Network is your connection to a team of Drupal experts, available 24x7, and backed by Acquia's engineering and professional services team. As an Acquia Network subscriber, you can submit help tickets, search our knowledge base and contribute in our subscriber forums.

The Acquia Network also provides you access to a number of cloud-based services. Services like heartbeat monitoring, software update management, and soon to be released integration with New Relic provide visibility into your site's performance and help with site management. Other services, like Acquia Search and Mollom, extend the functional capabilities of your sites.

We are in the middle of a massive redesign of the Acquia Network and many of the services you use through the Acquia Network today (including the Acquia Library, a broad collection of tips, tricks, how-to's, and resources for Drupal developers and site owners). Through the Acquia Network you will soon have the ability to easily access a growing list of third-party services, with many available at no additional charge. We already offer many third-party services (e.g. Mollom for spam filtering, New Relic for application profiling, etc), but we'll soon be opening up the Acquia Network as a ‘service delivery platform' and marketplace for additional services. In the works for release over the next few months are mobile design tools from Mobify, analytics, video services, marketing tools, and more.

Interested in adding your service to the Acquia Network? In the future, we will roll out APIs and infrastructure (e.g. billing) to enable other organizations to deliver their cloud-services to any Drupal site through the Acquia Network.

Add Acquia Hosting, a Drupal Platform-as-a-Service

For large websites that require custom code, high availability, on-demand elasticity or release management tools (i.e. staging and production workflows), we recommend Acquia Hosting, our Drupal-platform-as-a-service (Drupal PaaS).

Acquia Hosting is an extension of the Acquia Network, so if you need help scaling your site or debugging a problem, Acquia Client Advisors are always available to help. Through the Acquia Network, we also provide a number of Acquia Hosting specific e-services, including backups, database rollbacks, staging environments, version control for code management, and more.

Going forward you can expect even more developer tools and self-service tools to be added to Acquia Hosting, as well as more critical features for large scale sites, including improved security and code workflow options.

Add Drupal Gardens for rapid micro-site development

All sites are different. Not all your organization's website need the scale, functionality, complexity or longevity of your most important websites. A lot of times you have smaller sites that you may want to roll-out quickly, preferably without having to involve IT.

For that, we built Drupal Gardens, a Drupal-as-a-service platform that makes building Drupal websites as simple as point and click. Built on Drupal 7, Drupal Gardens brings the freedom and innovation you expect from open source without having to worry about installing, hosting or upgrading your Drupal site.

Our mission for Drupal Gardens is to allow site builders to go from design to online in minutes instead of days or weeks. To help, we provide an ever-growing library of site templates and themes to start from. We believe it will be the best platform for your smaller sites that complement your primary web properties.

For organizations that need to manage tens, if not hundreds, of small websites, we're building ‘Enterprise Drupal Gardens'. It provides site provisioning, site management, single sign-on, multi-site dashboards and organization wide templates and themes to maintain consistent branding.

Host your own sites, if you prefer

One of the biggest advantages of using Open Source software is that there are no limits to how you use the software. Some organizations prefer to host some of their own sites. The Acquia Network is able to plug in into your site, regardless of where it is hosted.

No lock in with "Open SaaS"

Almost all Software as a Service (SaaS) providers employ a proprietary model – they might allow you to export your data, but they usually don't allow you to export the underlying code. Users of Drupal Gardens are able to export their Drupal Gardens site – the code, the theme and data – and move of the platform to any Drupal hosting environment. By doing so, we provide people an easy on-ramp but we allow them to grow beyond the capabilities of Drupal Gardens without locking them in.

We call this "Open SaaS" or Software as a Service done right based on Open Source principles – it offers a much more secure and low-cost alternative to proprietary counterparts.

Conclusions

I've highlighted some of our key products and services in this blog post and will bring you a more detailed white paper focusing on Acquia's vision. Stay tuned!

During the past month, I've been trying New Relic on my personal blog. New Relic is a performance management tool. It can help to monitor, debug and optimize a site. It includes features like slow page request analysis, slow database query analysis, error tracking, scalability analysis, performance alerts, weekly e-mail reports, up-time monitoring and even very specialized features like Apache Solr profiling. It's a bundle of developer goodies that can be used on live production websites.

I'm a bit of a profiling geek. In my previous life, I spent a lot of time working on production profiling and performance optimization of Java applications. So when New Relic recently announced that they're working on support for PHP applications, I wanted to give it a try.

We began testing New Relic when the PHP version was still in beta. We believe it can make for a great service to add to Acquia Hosting and the Acquia Network. My site is hosted on Acquia Hosting which is why I have been testing it on my site. So far the testing has been good. We've helped New Relic fix a number of bugs during the beta test period, and in return New Relic has helped us to discover a performance problem in one of our deployment scripts on Acquia Hosting. Proof that New Relic is useful!

Now that we are past internal testing, I am excited to say that we will soon be bundling New Relic into the Acquia Network. Every Acquia customer will soon get access to a premium version of New Relic's monitoring tools at no additional charge -- whether you are hosting with us or not. Expect more detail from us soon on how to access New Relic as a subscriber.

New Relic requires a small 'agent' to be installed on your web server. The agent hooks into the PHP interpreter and sends the relevant information to a back-end for analysis and visualization. Because it hooks into the PHP interpreter, it can provide that deep, code-level visibility that is useful when debugging a performance problem. The New Relic agent also adds extra Drupal specific instrumentation -- and will probably add more over time.

I've only used it on my personal site, which currently runs on a regular Drupal 6. However, I'm quite keen to unleash it on a Drupal 7 installation to see what we can learn. In the mean time, I'm including some screenshots of what my personal blog looks like according to New Relic. If you're interested in improving the performance of your site and are a profiling geek like me, you should give New Relic a try.

The U.S. government recently launched a new Drupal website, RestoreTheGulf.gov, to provide information about the environmental catastrophe in the Gulf of Mexico from the oil spill earlier this year and announced plans to phase out the current disaster response.

The site runs on a fully managed Acquia Hosting cluster that is FISMA compliant, running on Amazon EC2. That is a pretty big deal as most government agencies are still wary about using the cloud.

I'm excited that the site is running Drupal and that it is hosted at Acquia -- not just because it's pushing adoption of Drupal in the cloud, but primarily because it is very important to restore the Gulf of Mexico. All small contributions help.

Government Acquisitions, an Ohio based company, is the prime contractor and worked with Acquia for support and hosting and with Siteworx (an Acquia partner) to build the site.

On our quest to offer the best possible hosting solution for large-scale Drupal sites here at Acquia, we recently added support for memcache to our Acquia Hosting platform.

For those that don't know memcache, it is a high-performance memory object caching system. Oftentimes it's used to speed up database-driven websites by caching data and objects in RAM. This is very effective in managing the load on your database, which for most web applications including Drupal, is the biggest performance bottleneck and risk to scalability.

Memcache is a natural addition to Acquia Hosting. The Memcache module for Drupal was originally written by Acquia's Robert Douglass, and continues to be maintained with Acquia's help. So it only makes sense for us to add support memcahe as a hosting option. Combined with all of the other high-performance and scalability features (e.g. Varnish, NginX, etc), it makes Acquia Hosting a very powerful offering.

We run memcache on the existing web servers to take advantage of spare free memory. As such, it is available for every Acquia Hosting customer at no additional cost. Memcache uses a client–server architecture so, if preferred, we can also spin up dedicated memcache servers for our customers. Drupal then talks to the memcache server over a network socket.

A number of Acquia Hosting customers are currently using it in production. Now I still need to take advantage of the new memcache support for my own site. Sounds like another weekend project. :-)