It's been a while since I last blogged about my experiments with Rails and the Dojo Toolkit. In fact, since that last blog post, I've migrated the Rails development environment to work against dojo-trunk by regularly performing a Subversion Update within my environment. Overall progress thus far include the following:

Modified the Rich Text Editor widget
I wanted to use the Rich Text Editor within a Rails form_for clause. In order to achieve that goal, this widget needed a small tweak to update the TextArea that it's attached to, before the Form is actually submitted.

The Rich Text Editor was also enhanced to expand to the full dimensions of its parent container, instead of the default behavior requiring you to specify a pixel-height.

A Form (Colored in Yellow) Containing Multiple Dojo Layout Objects
Dojo RichEditor stretches to fill its ContentPane Container

Created a new Dojo Form Class
Rails applications typically render content with “nested regions”, wrapped in DIV, SPAN, TR, or TD. This most often crops up with controller layouts that present an “outer shell” of the webpage, which include a <%= yield %> or <%= @content_for_layout %> to render the contents of the desired controller action. (index, new, show, etc.).

I ran into an interesting problem while using Rails with dojo: without the tweaks I made, a Form object cannot enclose multiple Dojo Layout objects like ContentPane. Thus, it was impossible to construct a form that wrapped a Top Toolbar (where Save and Cancel buttons live), and a main editing region below.

Created a new Data Store
This was the mother of all Rails and Dojo work. I wasn't happy with any of the stock Data Stores that come with the dojo toolkit. In fact, they all seemed to be more “proof of concept” that highlight specific features. It took a few weeks of work to construct a complete "Rails Compatible" dojo data store, but the end results are SOOOO totally worth it! With the addition a couple of enhanced Model adapters for Dojo Trees and Dojo Grids, I'm able to get Rails to present some fantastic GUI presentations of the underlying data.

Created a new Dojo Forest Store Model
While there's nothing wrong with the forest store model, out of the box, I wanted better handling for the tree's Node Labels.  The parent nodes are labeled to contain a count of child nodes in parentheses -- thus, if you create or drag nodes into an existing node, the parent's label should be updated with the new count.

Enhanced Tree Drag n' Drop Behavior: 
Upate Parent Label Counts on Drop (or Node Creation)
Before Image (Canada has Count = 2)

Enhanced Tree Drag n' Drop Behavior: 
Upate Parent Label Counts on Drop (or Node Creation)
After Image (Canada has Count = 3) 

Created a new Dojo Grid Model
While most of the dojo examples use static JSON data – either in a separate file or declared inline – I wanted to allow Dojo Grids (and Dojo Trees) to Lazy Load content from the Server. This was surprisingly straightforward. With this grid model between a Dojo Grid and the custom Rails Data Store described earlier, dojo grids lazy-load Rails-served data as rows scroll into focus. I also added a little something to allow Server-Side sorting and paging, along with the ability to specify additional query parameters like a Parent Node's ID.  With that enhancement, Rails and Dojo can render a Dojo Grid that ONLY contains the children of a parent node.

Dojo Grids with Lazy-Loading (on Scroll)
Also Includes Server-Side Sorting

Dojo Grids with Additional Server Query Filtering
(Selects all nodes that are children of current parent)

Next Steps
There's still a lot to do before the application I'm writing can be released to the world.  (Judging from the screenshots of the lookups, some of you probably already guessed what I'm working on. )  One rather large design change that I'm considering is to abandon the use of all Form Submits, and use the Dojo Data Store to exchange everything.  And before you dismiss me as being on crack, here's my reason for considering this route:  Ruby on Rails has TERRIBLE SUPPORT for Forms-Based Concurrency Handling!

Sure, Rails provides core support for Optimistic Revision (if your model includes the lock_revision magic field), but it leaves you to your own devices to detect the "409 Conflict" HTTP error.  You also receive an ugly "StaleObjectError" dump, and your Form-Submit data is pretty much lost unless you salvage them from the Rails Session.

Rails Conflict

Using the Dojo Data Store for Form Data Submission
If you're thinking this "problem" can be fixed by hacking the update action, I'd urge a bit of caution.  You cannot simply reload the updated model from the server, merge the form-submitted data, and save -- taking that approach achieves a "last in wins" effect, and defeats the purpose of revision locking.  To correctly handle an update conflict, you need a copy of the Original State (revision 0), the Current State (revision 1), and the Proposed State (revision X).  If the conflict resolution is done within the Rails Controller, you only have Revision 1 and Revision X available.

Thus, the plan is to use the custom Dojo Data Store to handle these conflicts.  The data store I wrote already maintains the "revision 0" and "revision X" states to handle the dojo.data.api.Write interface, which includes a revert() operation.  All that's needed is an HTTP Error trap that tests to see if we received a 409 Error, and issue a datastore  fetchItemByIdentity() within the data store to pull in the "revision 1" state.  Once all three are loaded, the data store can do a conflict-merge.  If the fields changed in "Revision 1" don't conflict with "Revision X", then FANTASTIC - just submit the changes and be on our way.  If a conflict is detected, though, the application should pop up a dialog-box -- "Revision X" is attempting to update a value that was already updated in "Revision 1".  The dialog should instruct the user to do cancel the current changeset.  Alternatively, a sophisticated application could reload the page with "revision 1" data, and highlight the conflicted fields for the user's review.

Doing the conflict-resolution within the Data Store also has a side benefit:  this little routine only needs to be written once, and will be available to ALL Rails on Dojo applications that are using this data store.

Gap Analysis of Dojo Toolkit

• Tree Drag n' Drop:  "drop" doesn't handle position amongst siblings - dropped items always end up at the bottom.
• Accordion Container:  I'd like a similar container to this that allows more than one pane to be "expanded".
• Dashboard Container: I'd really like to see a drag/drop "dashboard" container, where you can drag/drop "Widgets" into an X/Y grid position, like how Google Desktop works.

I'm sure there are other gaps I'll find along the way, but these are the ones that really stick out.  For the application I'm working on, these are pretty close to show-stoppers that I'll need to solve soon.

Stuff For My Todo List
I need to construct a Form Model that sits between a Rails/Dojo Form and an underlying Data Source.  This model basically holds a "focus item" within the Data Store for the Form to work against.  This model might also have functions like setFocus("identity-string"), gotoPrevious(), or gotoNext() to make it work more like a "data source cursor". 

I want the Rails/Dojo Form to render as normal, but the form should return false on submit to prevent the normal form-submit behavior.  Instead, a notification to the Form Model should fired, along with the corresponding values from the Form.  (using dojo.formToObject or some other formTo* variant, most likely..).  Once the datastore item is updated in the Form Model, issue a save() -- or not.  The actual save() action would depend on a cache setting within my data store implementation.  The Rails/Dojo form could be a "Detail Form" for a selected Grid Line Item, and all saves are cached until a save() button is clicked in the Grid.  (In this scenario, the "save button" on the Rails/Dojo form would really mean "save to data store cache").

Lots of design ideas to consider, and I'll get back to Rails/Dojo hacking in a week or two.  Right now, I'm researching git for the next phase of this application I'm writing.  I'm also looking for individuals or companies who may be interested in sponsoring this project to keep the momentum going.  Heck, even an "Atta Boy!" from the blogosphere would be encouraging motivation. 

I've been playing with the Dojo Toolkit (Release 1.0.2) for the past few weeks, and have made great progress in patching the Ruby on Rails core (2.0.2) to produce compatible data.  (side rant:  Hacking up Rails Core code is no fun at all -- not only do you have to find the issue and fix it, but you have to decide if your fix is specific to your own needs, or "General Purpose" enough for submission to the Rails Core team.)

*Grumble*  Anyways.  Despite not submitting patches, I've finally got a patched Rails Backend that works quite nicely with Dojo Toolkit.  In fact, it's kind of awesome to see the kinds of simple applications I can build with just plain old Dojo Toolkit, along with static JSON data islands.  And using the complete Dojo Toolkit definitely rocks my world much better than the (limited and outdated?) Dojo Toolbocks rails plugin.

The  dojo.data.ItemFileReadStore or dojox.data.QueryReadStore are perfectly fine for the small stuff (simple Tree Views, or for backing Data Grids), but I quickly ran into problems when I tried to do more complex Schemas -- particularly when I wanted to represent Schema Associations.

JSON: Type Information is Lost in Translation?!
As part of my system design, I am standardizing on exchanging JSON over the wire. There are compelling reasons to use JSON:  the declaration markup is much more compact, and it's wayyy faster and easier for JavaScript Engines to deserialize JSON into ready-to-use objects.

The downside to using JSON, though, is that you don't get type information in the output results. At least not by default.  That kind of sucks, because I'd like to represent Nested Associations in the output data. For example, let's look at the standard Dojo Continents/Countries example in XML:

<country>
  <area nil="true"/>
  <id type="integer">1</id>
  <name>Africa</name>
  <ntype>Continent</ntype>
  <population nil="true"/>
  <timezone nil="true"/>
  <children type="array">
    <child type="Country"> contents of child 1 XML trimmed by lalee </child>
    <child type="Country"> contents of child 2 XML trimmed by lalee </child>
  </children>
</country>

and the same data, in JSON:

  {  
  "name": "Africa", 
  "ntype": "Continent", 
  "timezone": null, 
  "id": 1, 
  "area": null, 
  "population": null, 
  "children": [{contents of child 1 JSON trimmed by lalee},{contents of child 2 JSON trimmed by lalee}] 
  }

Everyone seems to shrug this off as the price to pay for using a simplistic protocol. Umm.. no. There's no good reason why Schema Associations and Type Information (of associated records) can't be maintained and honored. In fact, a lot of what I plan to do with Django/Merb/Rails (backend) + Dojo Toolkit (frontend) absolutely depend on proper maintenance of Schema Associations.

If I were writing a Blog Server, I'd want to represent a Schema consisting of Posts, Comments, and Users. That way, a Post that is deep-fetched from a DataStore can provide data for the Post, displayed in the main area, as well as nested Comments, displayed in a DataGrid.