Acquia Search versus Drupal search

It's been several days since we launched Acquia Search commercially. After reviewing the press, articles, comments, and tweets, I wanted to address the question of why we seem to care so much about search and why we can't simply improve Drupal's built-in search module. These questions came up during the beta test period as well, and have even resonated with the WordPress community on Matt Mullenweg's blog. I feel they are important questions to address.

I've already partially answered these questions in two recent blog posts -- why Acquia Search matters for site administrators and why Acquia Search matters for site visitors -- but there is more to it.

First, at the end of the day, search is a hard but important problem. This is reflected by the size of the search market. Some have estimated the search market to be at least as big as the web content management market. The leading providers of site search technology such as Autonomy, FAST and Endeca have built large, successful businesses supplying search technology to the enterprise. Last year, FAST was acquired by Microsoft for $1.2 billion. Gartner forecasts that the enterprise search market will grow to more than $1.1 billion in total software revenue by 2011 (excluding professional service revenues). For many people in the Drupal community, these data points will probably come as a surprise.

Reality is that for a certain class of websites -- like intranets or e-commerce websites -- search can be the most important feature of the entire site. Faceted search can really increase your conversions if you have an e-commerce website, or can really boost the productivity of your employees if you have a large intranet. For those organizations, Drupal's built in search is simply not adequate. We invested in search because we believe that for many of these sites, enterprise-grade search is a requirement.

Secondly, why don't we just implement improvements in Drupal's core search module? As I've noted, search is a difficult problem -- it is hard for Drupal to compete with enterprise-grade search engines, to keep up with advances in search technology, and to do both while continuing to run in shared hosting environments. Instead, Acquia Search leverages the Open Source Lucene and Solr distributions from the Apache project.

The search module shipped with Drupal core has its purpose and target audience. It isn't right for everyone, just as Acquia Search is not for everyone. Both are important, not just for the Drupal community at large, but also for many of Acquia's own customers. Regardless, there is no question that we need to keep investing and improving Drupal's built-in search. The search module that is built into Drupal 7 already has improvements over the one in Drupal 6, in part because of Acquia's support of the Search sprint in Minnesota.

I'm hopeful that we can scale up our investments in Acquia Search as we grow the search component of our business. There is a lot more we can do, so I'd like to see us become active contributors to Apache Lucene and Solr, as well as continue to ramp our contributions to the different Apache Solr projects on drupal.org, as well as Drupal core's built-in search.

Comments

dalin (not verified):

Just like we include a fabulous JavaScript library instead of trying to write our own, it only makes sense to provide the same possibilities for search. Drupal is not the right tool to do every job. To think otherwise would be catastrophic.

July 7, 2009 - 07:58
Andy Forbes (not verified):

Dries:

If I had to pick the one thing that really slammed my last Drupal project, it was search - both from an indexing and searching perspective. My only question for you is why didn't you do this a year ago when I needed it? :-)

I think bifurcating Drupal search into a basic free search that works for most sites, and an enterprise class search product for sites that really need it is absolutely the way to go.

For those people that complain about what the Acquia search service costs - tell them to go get a quote from Google for indexing 1,000,000 pages and then see if they still complain.

Andy

July 7, 2009 - 23:02
wes (not verified):

Dries, I couldn't agree with you more. It's promising to know you take this sector so seriously. One question I have - can we add Acquia search to a vanilla Drupal install (assuming we pay the monthly fee)? I ask this because as a developer, sometimes I like a more fine grained control over what modules, etc. get installed... sometimes the whole Acquia package is overkill for certain sites, but a powerful search framework is needed. Thanks!

September 22, 2010 - 03:30
Dries:

You can add Acquia Search to a vanilla Drupal install as long as you sign up for an Acquia Network subscription and you install the Acquia Network Connector module for Drupal.

October 11, 2010 - 18:15

Add new comment

© 1999-2014 Dries Buytaert Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License.
Drupal is a Registered Trademark of Dries Buytaert.