XSLDataGrid / Sarissa / AJAXSLT

Two weeks later …
With some tinkering and coding, now I’m really jazzed about the idea of clientside XSLT for widget creation. With Sarissa, it’s not slow at all. Hats off to Manos Batsis for Sarissa, he’s done some really cool work. And for those who want to do it on the server, being able to leverage the exact same code and technique seems really great to me. A dispatcher could even look at the bandwidth of the client and the version of the browser to decide how to optimally deliver the experience. To read about the advantages I see to this approach overall, check out a prior post Google’s AJAXSLT and UI Widgetry Thoughts. I’m getting most of the original DataGrid features working in this XSLT version, so check back as this demo page will get updated. I’d like to offer some profiling statistics too, but even with recordsets of 200 rows, I can barely tell the difference between the Sarissa client-side transform speed and the server/render speed.
References:
The XSL file: XSLDataGrid.xsl
Sarissa
Google’s AJAXSLT
Below is a test page to tinker with:

11 comments June 16th, 2006 Author: Lindsey Simon

Google’s AJAXSLT and UI Widgetry Thoughts

I’ve recently spent a little time tinkering with Google’s JavaScript XSLT library. It is available online at: goog-ajaxslt.sourceforge.net. I started to wonder if client-side XSLT would be one cool way to implement the OpenRico LiveGrid’s and/or my DataGrid’s functionality to xhtml tables. One of Richard’s goals for the OpenRico project is for it to be able to “fail gracefully” – in other words, to not be crippling to roll back out of; this is a principle I like too. The idea of taking a semantically correct representation of tabular data and then “live-ifying” it, and or adding certain kinds of dynamic functionality(reorderable columns, resizable columns, etc…) to it, is very, very appealing to me – an analogy being something like a taking a house frame and building everything else on top of and into it – and being able to go, “You know what, I don’t need a bathroom!” and voila, you can lose it and use that space for something else. Google’s XSLT via JavaScript/DOM looks like it will be too slow to do what I want, but it is a neat approach, at least in theory.

Why not CSS only?
One of the classic problems with tables is trying to achieve the scrolling table-body. It has been done with some quirky looking CSS (mostly for IE’s sake) but a CSS-only approach will never achieve some of the functionality I crave in a table beyond scrollability. (Until XForms – jk)

Why not DOM then?
So introducing a slew browser-centric CSS selectors doesn’t appeal to me if DOM manipulation can achieve the same thing more simply. Derek Anderson has created a nice demo of this approach. I think the Google AJAXSLT approach might offer a clean way of doing what Derek did, but what if you want all the extra bells and whistles? DOM just seems too slow – and Google’s AJAXSLT seems too slow (it also being an interface to browser DOM). You can see a test of a really simple table with like 100 rows being rewritten into two tables ala our DataGrid here.

So what then?
I don’t believe that it’s using Java to produce JavaScript! Currently, at FineTooth, we’re writing dynamic Smarty templates with inline functionality and using Ajax to completely repopulate container divs with the template-driven XHTML – basically bypassing all DOM interaction for setup, but using and relying on DOM for interactivity. This approach works for us now, but if we stopped using PHP or Smarty, we’d have to reimplement things. I’m going to check out Protowidget, but I’m already not a fan of its syntax – “leveraging custom attributes in order to do all of the wiring”. I guess I have this aversion to repurposing structural tags (anchors as buttons, spans as text) – but perhaps I’m just being semantically snotty. Server-side XSLT is an approach I’m looking into. I’d like to be able to produce these widgets somewhat server-agnostically, or at least have very similar implementations for ASP, Ruby on Rails, PHP, etc.. I’d rather not have the Rails way and an ASP way that looked totally foreign to one another. The hunt goes on then…

Add comment June 6th, 2006 Author: Lindsey Simon

Update: Reorganizable DataGrid built on Ajax / Smarty / Prototype / Scriptaculous

Welcome! Since many of you are arriving here from a recent mention on Ajaxian I wanted to let you know about a more bandwidth-optimized, working prototype of an XSLT version of this component – this is the direction I really want to see this grid move in!

Updated to version 0.3
README
LICENSE
Download: DataGrid-0.3.tar.gz (check out the QuickSearch filter feature by rightclicking on the “Owner” column!)
Note: Vendor column is intentionally non-draggable, non-reorderable, and non-resizable.

Changes
* Simpler template design for right-click column menus
* Patch to script.aculo.us allows mouse dragging outside of the Sortable container to update the list
(delay_and_ignorePosition_forSortables.diff, Functional Test of the patch)
* Improved speed on column reordering and resizing
* Removed a div inside the data table td’s which made an incredible speed boost to IE’s rendering
* Recoded some of the for loops to use do {} while structures
* General cleanup

Notes
Primarily, there have been many changes made to improve speed of the DOM operations, and we plan on doing more! Many thanks to this article which I found to be exhaustive and interesting: Optimizing JavaScript for Execution Speed. Also, I’ve been analyzing the code in the latest update to FogBugz by Fog Creek Software – they’ve included resizability and column reordering. Their mechanisms seem to use XML parsing a lot – anyhow, I can confirm very similar speed results in our DataGrid to theirs. They’ve chosen to make the column headers completely draggable which is intuitive if you don’t know that feature is there for sure. And you click the actual text of the column label to sort. I like that too. Anyhow, I’ve restyled the grid here too to be more pleasant on the eyes and I hope to get some feedback from other developers out there!

13 comments May 30th, 2006 Author: Lindsey Simon

A Better File Upload Progress Bar using Python, Ajax Prototype, & JSON

Update: Aug 22, 2006: Download the Code and a working Demo!
Also, read our documentation on the inner workings of the python CGI.

Demo:

The Python File Input CGI
About four months ago, Christopher Bottaro, another developer here at FineTooth, worked on a Python CGI script to handle file uploads and it is mighty ingenious. The CGI uses standard input as a stream, which PHP cannot do. The CGI writes the stream data as it arrives, in blocks, to a file buffer. This enables us to use another PHP server-side script to to poll the size of the file buffer while it’s being written to. With this, we can calculate the parameters necessary for a progress bar user interface (time remaining, speed of upload). We integrated the whole kit and kaboodle into our PHP Form class so that if a form indeed had a file input, the progress bar was all set up to popup automagically, and the form’s action was set to point to the CGI. Upon file upload completion, the CGI would perform a redirect to the ‘original’ action URI for data processing.

The Problem Case – File Inputs & JavaScript
As we’ve moved more and more towards using JavaScript to control our application’s form processing (as opposed to using the standard form action attribute as an URL) one problem situation has been caused by the file input. Since the file input is essentially an OS level object, JavaScript can’t access the information (thank goodness!) in order to send it programmatically to a handler. So what we’ve done in the past has been to use iframes for forms that needed file inputs. I’ve read that the Dojo Toolkit basically does this technique – by detecting if the form has a file input on it, and if so, it uses an iframe mechanism to submit the form. For awhile this was a good workaround. However, for aesthetic and programmatic design reasons, I wanted to come up with another solution.

Why not just use Iframes for those forms?
For one thing, iframes are just kind of annoying. If you want to access your JavaScript framework, you have to program in the scope of window.parent. Hopefully, the browser is caching your stylesheets and javascripts, but I’ve sometimes seen otherwise watching apache logs, so I don’t believe the iframe solution is always efficient. More importantly, though, using the classic form post technique and then rewriting output from the server to the iframe’s window is just not how I want to process forms in our application. I prefer to use Prototype’s Form.serialize and then Ajax.Request to deal with form transactions. Our application doesn’t have page refreshes overall, so introducing them into little iframes if and only when a form needed a file input just feels so cowardly. And, all that *really* needs to post is the file input data. So suddenly, the brain dinger started going off.

Embedding N-Iframes
Instead of using an iframe for the entirety of the form, I wanted to try embedding iframes individually for each file input that needed to be on the form. I could have an iframe that loaded up its own form object into its window.body with only the file input and some session information. Then, via the controlling form, we can use javascript to submit each embedded iframe’s form programmatically. Once the poller running in the main form detects that all of the files are safely on the server, we can submit the rest of the form data using Ajax.Request along with some logic to capture the information about the freshly uploaded files. It sounded funky, but lo-and-behold, it works and it scales.

A diagram might help make more sense of this approach:

Form
   *text input
   *text input
   *Iframe
        *form
            *file input
   *text input
   etc...

What’s key here is that instead of having a single CGI try to deal with multiple file inputs and all the other inputs on the form, our CGI is doing only what we originally wanted it for in the first place – the file upload.

Here’s the form processing logic:

* User chooses file(s) to upload in the form.
* User presses submit.
* JavaScript validation runs against the form.
* JavaScript looks inside the form for any embedded iframes.
* If it finds them, it submits the embedded form to the CGI.
* An interval runs again and again until all the files
   are reporting 100% upload
* The rest of the form data is at last sent using
   Ajax.Request

A few other niceties
Since our CGI is writing data in chunks, and since the juciest information about the file is in the first little bit of data, the CGI can do a little regular expression matching early on in the process to capture the file’s mimetype and name. Our poller can then read this information and we can show a purty little icon for the filetype and also reference the actual file name in the poller window – making it look pretty slick!

The Poller
The return data from the poller comes back to the browser as JSON and looks like this:

({"success":true,
"percent_done":3,
"kps":3,
"timeleft":"21 mins 8 secs",
"filename":"01 Mistakes.mp3",
"mimetype":"audio\/mpeg",
"mimetype_iconsrc":"music.png",
"current_size":126676,
"total_size":4044948})


So Why can’t we do multiple files all at once?
Originally I wanted a display that resembled Firefox’s Dowload Manager. One problem I haven’t found an adequate answer to is, why does it seem that I can’t be sending up multiple CGI requests from the embedded iframes at once? I can see that apache seems to be blocking until all but the last CGI are left in terms of sending responses back to the browser. Maybe it is the browser doing that; I did find one mention of that being the case back in old versions of Netscape, but I can’t seem to find any other validation of this limitation anywhere. So until that issue is resolved, files will be sent and polled sequentially in this version.

21 comments March 31st, 2006 Author: Lindsey Simon

SXSW Interactive 2006

I figured this would be a good place to record some thoughts that have come up at SXSW as well as to record some notes about the panels I’ve attended this year. So here goes:

Saturday: Daniel Gilbert Presentation: How to Do Precisely the Right Thing at All Possible Times

I accidentally just deleted my entire review of this panel by typing Ctrl-W a la emacs cut. In the end, it must not have been karmically all that useful!

Saturday: Jim Coudal / Jason Fried Opening Remarks
37signals spokesman and his partner in marketing, Jim Coudal, made strong cases for the value of curiosity and the waste of time that is dogma. Curiosity empowers individuals to know how to learn better and more quickly, as opposed to being just purely information libraries. By ditching the dogma (technical specs, documentation, etc.. ) developers can focus on developing itself – trying new ideas, challenging ways of doing things, etc. It was a very uplifting conversation, but I heard a great deal of mumbling, that “Yeah, that works for your little company, but I don’t think it would work for mine”. Mr. Fried had indeed said that their conclusions would not work for everyone, but rather to just be open to the possibilities that there is “no right way”. Start small, fail in obsucrity, iterate, and succeed!

Sunday: What’s Hot in Web Applications
Fix a pain-point for people, and wait patiently for the cash to start rolling in – that was the theme of these Silicon Valley webapp folks. They stressed the importance of solving a real life problem that you are actually having as being the key motivator behind successful startups. If you’re only trying to solve other people’s problems you will both tire of that and come to resent it. The sites demo’d were:

YackPack – an audio-based email webapp. Mostly struck me as being a pain in the butt and slow. I don’t want to use the web to listen to people talk myself, but it was certainly interesting to see how it could be used that way. Maybe when the speed isn’t such of an issue this technology will be cooler.

Meebo – web based IM client integrator. This was really cool – and made some great cases for how web-based apps are really making a web-based OS more possible.

Zimbra – competitor to the Exchange behemoth. Zimbra’s mail client was totally impressive. Really cool integration tools (Blackberry, Calendar, etc.. ) and made me start to wonder if mutt/vim isn’t the ultimate email tool…

Sunday: Making Web 2.0 Accessible
This was an unfortunately useless panel – unfortunate, because it left so much hanging. Short on usable advice and remiss on any truly hard-to-find information, the best way to sum up the theme would be to “get back to basics, and make sure there are ALT tags in your images”. Sure, that’s always good advice. But what the folks in the crowd are hungry for are options and useful examples and advice. There was only one code example in the whole presentation, and yet the conversation centered mostly on code.

If, as Shawn () from the W3C suggested, WCAG 2.0 is was designed from the bottom up for testability as opposed to readability, why was there no demo of a test we could all go replicate on our sites? Maybe the “testability” she meant involved getting users to test and watch them only, and not programmatic testing. Sure, Bank of America can create a team dedicated to doing Accessibility testing (only out of fear of lawsuit, mind you), but what can the developers in the crowd do? It’s easy to shoot down AJAX frameworks and applications(BaseCamp) for NOT being Accessible, but where was the demo of a truly good accessible site? What are the best techniques? What tools are better than others and how should we use them?

No one had a decent suggestion of a framework that had been was more accessibility minded than others. Did anybody “fix” one of these frameworks and post it for download? Now that would have been useful! Until we have the resources to hire less enabled to test our applications or the threat of a lawsuit on our hands, we are left as helpless as we started before the panel. It is indeed noble to code towards Accessibility. Do not mistake my bummed out reaction to this panel as apathy toward the goal of having webapps be truly accessible.

Monday: CSS Problem Solving

This was a fairly fun and useful panel that dealt with a variety of CSS problems and how to solve them. Not a whole lot to say about it really. I didn’t see much going on there that blew my mind, but I don’t think that was the intention. It did seem like everyone wanted to promote their new BSS book however.

Monday: Microformats

Now this was a really great panel. Tantek Celik is a great moderator. He was well prepared and his slides are worth downloading. The panel didn’t ever feel rushed, but still crammed a ton of info and examples into it. Examples of using hCard and vCard translation from technorati, mashups of SXSW speakers and after-parties – mapped on Google maps, Flock recognizing all this extra content.. Man, this was cool! But, I think it’s a technology that’s waiting for the killer app to take advantage of it, and even Flock seemed not to have convinced me that it was it. I’m going to install tails right now though and see if I can make use of these things. My gadget collection is currently pretty weak – my cell phone has trouble holding a charge, nevermind vCards

Add comment March 13th, 2006 Author: Lindsey Simon