DabbleDB = CCK + Views + 12 Duvels

Peter Van Dijck sent me a link to this DabbleDB demo, referring to it as "Drupal's CCK and Views after twelve Duvels". :)

Peter is right. DabbleDB is awesome, Drupal can do much of what DabbleDB can do, and has been able to do so for a long time. Witness this one year old Drupal screencast. Where Drupal fails short is in making this functionality easy to use.

I still believe that it is of strategic importance that we move more of the CCK and Views into Drupal's main distribution, as opposed to keeping them contributed modules that you need to install separately. And when we do, we need to look at tools like DabbleDB, as there is a lot to learn from their user interface and interaction design.

I wish we had more people willing to make this happen. I'll be the first in line to help, because together we can get the CCK and Views to the next level.

Update: DabbleDB was acquired by Twitter and the DabbleDB website no longer exists. The link to the original demo video broke but I updated it to point to a backup on YouTube. YouTube seems to have two good videos on DabbleDB that are worth watching: https://www.youtube.com/watch?v=6wZmYMWKLkY and https://www.youtube.com/watch?v=MCVj5RZOqwY.

Comments

Adrian (not verified):

I'm incredibly impressed by what Avi and the guys have done with DabbleDB.

The key to their power is their platform (smalltalk), which allows them to do a lot of awesome things we have to fight the LAMP stack tooth and nail for.

Instead of having to store stuff in a table, they have an object database, that gets stored to memory. This allows them to be far more flexible in their interpretation and manipulation of the data they have in their system.

I was amazed when they took a text field that used to contain comma separated values, into it's own "table", with a "join" on the report they were using, and be able to re-use it wherever.

CCK doesn't even allow you to change the field types, much less easily port the old freeform data into entirely new data structures.

There's a very big schism between how Drupal and DabbleDB approach their interfaces though.

DabbleDB is about manipulating and importing / exporting data. It's focus gives it incredible power. Drupal is far too tied to publishing of data, not manipulating it.

Panels + Views + CCK changes this somewhat, but it still feels tacked on. (This is not a slight to any of those modules, they are tacked on by definition, as they are contributed modules.)

In my opinion it would require a drastic change of approach to how we handle all data in Drupal to be able to truly match their functionality.

One thing though, in my experiences, DabbleDB didn't degrade well. It _required_ AJAX to operate. Which is something Drupal could not, and should not ever strive for.

June 18, 2007
Wim Mostrey (not verified):

It is possible. I've had the "pleasure" of importing tens of thousands of entries from Excel to Drupal, containing taxonomy, users, organizations, articles and books, each of them linked: books contained articles (which have the pdf-version attached), both of them having terms attached and were linked to users in organizations. I had to import them while keeping the relations intact and with the possibility to view all articles written by a certain user or all users in a certain organization. It was easier than I expected using CCK and Views and I see nothing in DabbleDB that could not be done by Drupal.

"CCK doesn't even allow you to change the field types, much less easily port the old freeform data into entirely new data structures." The GUI doesn't allow you to do that, but nothing restricts you from changing all this in your own module.

June 18, 2007
Dublin Drupaller (not verified):

I second that emotion (getting CCK into core), although I would like to see a views-lite.module rather than the full contributed version of views finding it's way into core with it.

Dub

June 18, 2007
Dries:

Yes, the goal is to get the core of Views and CCK into core, not the whole code base. For the CCK this would mean support for the most common field types such as numeric types, textfields, textareas and possibly dates. For Views this would mean support for common views such as the ability to list all the products in your portfolio or the ability to list all the employees in your company.

I think that as a community, we need to give a strong signal that we believe this is the way forward. Ultimately, that signal should translate to patches that make the CCK and Views accessible to the masses -- both in terms of availability and in terms of ease of use.

Frankly, we can't afford not to put more of this in Drupal core. It's a pity that we failed to extend core's support for custom content types in Drupal 6. (The Drupal 6 code freeze is two weeks from now, so technically, it can still happen. That would certainly take me by surprise.)

June 18, 2007
Jeff Eaton (not verified):

Frankly, we can't afford not to put more of this in Drupal core. It's a pity that we failed to extend core's support for custom content types in Drupal 6. (The Drupal 6 code freeze is two weeks from now, so technically, it can still happen. That would certainly take me by surprise.)

We've hit a point with CCK, I believe, that forces us to embark on a more thorough refactoring of core systems. That's where the work on node structuring and node rendering has been pushing -- pulling the 'old style' data storage mechanisms we currently use into the world of structured arrays, cleanly defined callbacks, renderable/validatable structures, etc.

A lot of the heavy lifting that CCK does under the core is about enforcing those structures on its field data. It's difficult to explain that sometimes, but it's important. Without that kind of fundamental underlying refactoring, we'll just build a two-headed architecture that uses 'old style' and 'new style' content types in a bafflingly weird mix. We already have themers and module builders trying to figure out why some fields are stored in $node->field and others are in $node->fields['fieldname']['value'], while others are in $node->content['fieldname']['#value']. Consolidating that muddle is just as important as getting the killer feature in our changelog. It requires pulling every module along to the 'new' way, though.

Ah, don't mind me. Just rambling. ;-)

June 18, 2007
Dries:

Eaton: I hear you, and you're spot on.

Thanks to your hard work, and that of others, we've made some progress, and we've more improvements waiting to be reviewed, tested and committed. We're fixing this from the inside out, and that is exactly the right approach to take.

It is just that we didn't make nearly as much CCK-progress as I hoped we'd make in this extended release cycle. And just to be 100% clear -- you're not the one to blame for that.

June 18, 2007
Boris Mann (not verified):

"I wish we had more people willing to make this happen"

...which really should read, "willing AND able". There are many willing, there are very few that have the time and/or the expertise to make this happen.

Views Light seems the smallest short term target.

And yeah, Avi and the guys are great. Vancouver wins again :P

June 18, 2007
ilabra (not verified):

Umm, I think I might be able to help here but only if we have the time and ability to deal with the complexity of these problems. An 80% solution might make sense when you doing features or UI but not when your dealing with issues that affect data integrity and manipulation.

June 18, 2007
Rodeo (not verified):

Als ik de populariteit van de CCK/Views-combo zie dan moet ik zeggen dat het me enigzins verbaasde dat die functie nog niet standaard in Drupal ingebakken zat ...

June 18, 2007
merlinofchaos (not verified):

I think the focus of dabbledb is a little different, as Adrian said, and this certainly changes things a little bit.

I do think the UI tools they either wrote for this or have available certainly helps. There's a lot of fun javascript wizardry there that Drupal is only now (thanks to the availability of jquery) starting to feel. And there are still not that many javascript wizards in Drupal, making the power available to everyone. So for the moment, we're still largely stuck coming up with our own ajaxy goodness tools.

Secondly, I've never been convinced that there IS such a thing as a Views lite. I know it's a popular topic and concept, but I layered Views as much as I could, and at the end of the day, there really isn't that much you can strip out of Views and have it still do the things that people really need it to do.

June 18, 2007
adrian (not verified):

I concur with Earl.

I don't believe there is such a thing as a views_light, and quite frankly, apart from the interface, views is actually very small for what it can do.

I think the split between views and views_ui is incredibly sensible, and I personally feel that views should become a library of sorts.

The same way modules provide forms by adding a _menu entry to do a drupal_get_form, they should do it to drupal_get_view.

We could possibly even introduce conditionally loaded '.view' files in module directories.

Views.inc would only be included when actually displaying a view, and the .view files would only be included when loading the view (or during module initialization for caching purposes).

Views would then also become a valid drupal_render element of their own. So you could include views with your forms such as for related nodes with checkboxes, updated during preview (or live with AHAH) because you can set arguments to fields in the current form, or as part of your output.

By default, you shouldn't be able to edit the views, but modules should be able to define them (and export should be able to export to a module that can be distributed).

If you want to disable a view, use the menu module to remove the entry.

If you want to edit views, or create your own views install the views_ui module (which should stay out of core, at least until it is incredibly simple to use).

June 18, 2007
Dries:

I agree with many of these ideas. The tracker module might be a good example of a module that does nothing but expose a view. I'm not saying that it should be possible to build such a complex view with a visual views builder, but it is something that could potentially benefit from tapping into the underlying architecture of Views. The point being: if we move the query builder to core, we can use it to refactor parts of core. Food for thought!

June 19, 2007
zoon_unit (not verified):

I know I'm speaking from the position of a neophyte on this stuff, so if my suggestions don't make sense, just ignore me. :-)

But it seems that a full install of Views into core makes the most sense anyway. It seems that many other core modules could be eliminated with full Views and CCK installed, such as the entire blocks section, for instance, managed by Views instead. Profile module could be replaced by CCK if profiles were made into nodes. Is it possible for Views to become the query "engine" for Core? That is, handling all or most queries through its interface?

I know one limitation would be that Views is unable to construct some of the more complex and efficient queries through its visual interface, but is it possible to allow a "human edited" query to be saved and executed from within Views? That way, Views limitations would be overridden by this approach while still making the query system more accessible to developers who haven't learned the ins and outs of the Drupal core API.

June 19, 2007
Dries:

I don't think that anyone is asking us to drop functionality. What people mean with a "views light", is a tool to build (1) simple views in a (2) simple manner.

  1. Simple views like a product portfolio, a directory of executive staff, a list of clients, or an overview of the hobby projects that people are working on. There are dozens of examples of "light-weight views", and I think that is what people want to see become part of Drupal core more than anything else.

    The ability to accommodate these use cases out of the box is key, but it doesn't mean that we should ditch the ability to provide filters, to export views as RSS or the ability to provide a calendar view.

  2. In a simple manner because the current View's configuration pages are complex and daunting to use (but incredibly powerful). I think this is an important reason why people demand a "views light". When users say "light" they don't necessarily mean "stripped down" -- it often means "easier to use".

The Views module has humongous potential, and can revolutionize the way people build websites. Let me help you untap that potential, and let us all join forces to execute that vision. An important part of that is moving more of CCK and Views to core. :)

The only reason to go with a stripped down version of Views, is to make it easier to integrate it into core, not to drop functionality.

June 19, 2007
Roel (not verified):

DabbleDB is an amazing tool. We're using it since March 2006 (private beta) and today we're pushing it's limits. We (AnySurfer) rely so heavily on it, it's scary! We're using DabbleDB as a CRM, a project management tool, a group calendar and much more. It allows us to work together from different offices and our data is always 'safe' (away from our own unreliable webserver ;)

It's funny that you compare Drupal + Views/CCK to what Dabble offers. Everybody sees something different in DabbleDB and I think that says a lot about it's versatilty. I've even read stories on dabble's user forum from people who use it as a 'real' web CMS (using the CSV exports and some PHP parsing) :) Although that doens't seem very practical to me...

The biggest differnce (for me) is that my Drupal experiments with CCK/Views seem to break quickly when I fiddle around too much (this says much more about my technical skills than it says about Drupal itself of course) :) With DabbleDB it's difficult to break things because it's so forgiving (and you can't really break because it's a hosted solution of courseĆ . With Dabble, I got results in a couple of hours, and with Drupal I've been wrestling for weeks. I know this is not a fair comparison, since every tool serves it purpose... but still.

June 18, 2007
Benj (not verified):

CCK + Views made me switch to Drupal last year because almost none of the applications around offered these functionalities. I spent so much time (and $$$) to find an application that would let me create my own fields and display them the way I wanted.

I went to setup a test account on DabbleDB.... sure I like how I can manipulate the database, and wish it would be that easy for me (or my visitors) to do so on Drupal... but I would not consider using DabbleDB over Drupal. Many reasons for this: no template, hosted solution, no file attachments, no CMS features... it is "only" a database manipulation tool.

I think version 7 of Drupal should focus exclusively on integrating CCK + Views as part of Drupal core (+ extending the functionalities of both). Indeed, Drupal + CCK + Views is not far from DabbleDB. Drupal can gain a big, HUGE popularity if the leader pushes it in the right direction.

Globally, I see two major Drupal development tracks:

  • Cache improvement for logged in users
  • Database manipulation / display / export

If a decisive decision is taken to have Drupal shine spectacularly in these two areas, it will stay on the edge. Drupal must be become renowned for its database capabilities.

Cordially,
Benj (brakkar)

June 19, 2007
Anonymous (not verified):

@Jeff Eaton:

We already have themers and module builders trying to figure out why some fields are stored in $node->field and others are in $node->fields['fieldname']['value'], while others are in $node->content['fieldname']['#value'].

+100

(And I love Drupal passionately.)

June 27, 2007
Jacques Mattheij (not verified):

It would be really good news is Drupal became more 'internally consistent', there are too many ways in which the same stuff is being done right now.

That leads to several kinds of inefficiencies, there is a lot of duplicated code as well as slow load times for nodes because the data that makes up a complete node has to come from several different sources.

It's not uncommon for a fairly simple node type to pull data from 3 or 4 tables when in reality a single table should really be the goal to strive for.

January 9, 2008

Pages

Updates from Dries straight to your mailbox