Engage / ArtifactView / Comments / Storing documents in couchdb and other things
Joan Garcia Vila
jgarciavila at uoc.edu
Tue Dec 22 11:05:38 UTC 2009
Hi all.Colin, it seems a good approach. I did some work on lucene-couchdb queries too (which was posted in the jira [the couchdb functions] and was tested with database.htm).I wish you a Merry Christmas and a very Happy New Year.Cheers,Joan.--> Missatge original de Colin Clark (colinbdclark at gmail.com) per a Joan García Vila enviat el 21/12/2009 20:10:05Hey all,[Joan, I know you're currently away from work due to an injury, so don't worry about the details of this. My main goal in responding to your questions (sorry for the delay!) is so that the whole community has clarity on our next steps in terms of data access.]I like the concept of the database.js implementation. Since it was written, we uncovered two JavaScript APIs that ship directly with CouchDB:http://titan.atrc.utoronto.ca:5984/_utils/script/couch.jshttp://titan.atrc.utoronto.ca:5984/_utils/script/jquery.couch.jsOne is a plain JavaScript API, and the other is designed as a jQuery plugin. They offer overlapping functionality, but in general a preference is emerging for the jQuery plugin--it's properly namespaced, and a bit better structured. Here's a thread on the fluid- work list from last week where Antranig and I discuss the issues:http://old.nabble.com/Fluid-Engage-nightly-build-broken-td26785515.html#a26832718I think we're always better to contribute back to existing projects rather than rolling our own where it's appropriate, so I'd like to suggest that instead of working on database.js, we contribute some patches back to Couch for their jQuery plugin. One area where we'll need to contribute back, in addition to the ones mentioned by Antranig, is support for Couch Lucene views, which we use frequently.Antranig also has some code in his FLUID-213 patch that illustrates some slightly higher-level ways of making Couch requests within Kettle:http://issues.fluidproject.org/secure/attachment/10833/ENGAGE-213-demonstration.patchThis covers the low-level issues of data access. I think our second goal is to provide nice abstractions so that every Engage service developer doesn't have to write the same data access boiler plate goal. In a recent development meeting, we talked a little bit about the notion of "data source specs," which would provide all of the information needed to retrieve and save data to/from Couch in a declarative way.This data source spec would include the URL in Couch that identifies the document or set of documents being used by a particular data feed and component. It would also include the requisite model mapping information required to correctly reshape the model into a form the component can suitably use. You can see a rough sketch of our current model mapping code in this file:http://source.fluidproject.org/svn/fluid/engage/fluid-engage-core/trunk/framework/js/Engage.jsTake a look at the fluid.engage.collections data structure for an example of the model maps we're currently using. This code needs quite a bit of work to be correctly parameterized and generalized, but the idea is to be able to have a document in Couch that represents the necessary transformations required on the model for a particular view context. The framework will include a number of "canned reshapers" for common mapping tasks, and museums could extend this with their own specific code if needed.This second layer of data access will take a bit longer for us to implement, so let's focus first on the task--contributing back to Couch's jquery.couch.js file based on our own needs.I hope this helps clarify the issue of data access in Engage a bit. This is all still very much in the concept stage, so no doubt it also opens up some new questions. Everyone, please don't hesitate to respond to this thread with thoughts, comments, and questions.ColinOn 1-Dec-09, at 9:31 AM, Joan Garcia Vila wrote:> Hi all.>> Recently I've posted an alpha implementation of the Database > component (which has to become, I suppose, a subcomponent of > ArtifactComments [server side]).> Needs a conceptual review.>> - Talking about storing data on couchdb, this has been my first > approach:>> // Artifact Comment example.> var artifactComment = {};> artifactComment.engage_type = "artifact-comment";> artifactComment.artifact_id = > "00311f0d3f5456c91934e472b880e9c5";> artifactComment.insertDate = "2009-11-30";> artifactComment.name = "Juan";> artifactComment.location = "Barcelona, Catalonia, Spain";> artifactComment.abuse = false;> artifactComment.info = "Barcelona is a beautiful > mediterranian city.";>> There are few essential key points to check (understand / validate / > talk about):>> 1.- This document has a field that identifies its type > ("engage_type").> If we store different object types inside the same database this > will help and give higher scores in lucene fulltext search if we add > the document type in the query> (Please see/run the example).>> 2 .- This document has a reference to another document. See field > "artifact_id". Important to have good views with high selectivity.> This will help us to create "in-server" high speed paged queries.> Views for each document type if we need this functionality.> (Please see/run the example).>> 3.- The object also gets its _id and _rev attributes after it has > been stored.> This is coherent with couchdb (object oriented database) and the key > is an uuid.> (Please see/run the example).>> 4.- Data attributes is stored as a fixed text with pattern "YYYY-MM- > DD".> This will help us to have date sorted field views.>> 5.- One important thing is that having the documents (2 diferent > types) inside the same database allows us to create fast views (see > Database.js).> We can also create "pseudo joins" [complex views using map/reduce > _design functions]>> - Other things that come to my mind, while developing the component, > are:> 1. - How do we know who we are?, I haven't see a login page in the > wireframes.> But there is behaviour that seems to depend on knowing who we are.> Like report abuse or delete, for an example.> makes sense?>> 2.- How do we know that the user is the moderator?> makes sense?>> Cheers,> joan>> ps: you can find (Database. [js/html] on scratchpad/bug-parade. > _______________________________________________________> fluid-work mailing list - fluid-work at fluidproject.org> To unsubscribe, change settings or access archives,> see http://fluidproject.org/mailman/listinfo/fluid-work---Colin ClarkTechnical Lead, Fluid Projecthttp://fluidproject.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://fluidproject.org/pipermail/fluid-work/attachments/20091222/3919c05f/attachment.html>