Generalizing the File Uploader - Work Queue component

[Colin Clark]

Hi again Jonathan,

On 5-Jan-10, at 1:57 PM, Jonathan Hung wrote:
> Yes, reporting progress isn't going to be perfect and I guess it's  
> the nature of the beast. I think when you're talking about a job  
> that takes possibly hours, an estimate that's a few minutes off  
> won't be terrible (not great, but tolerable). I think as long as  
> everything is communicated clearly so there's a clear level of  
> expectation, I don't think the user will mind.
>
> So I don't think there will an incremental progress indicator.  
> Instead it will likely be a "busy" bar / spinner and updating status  
> text.

That makes sense to me. Some indication is still better than nothing,  
especially if we aren't displaying a traditional progress bar with  
percent completed.

In that case, I'm imagining the code is going to be much simpler and  
we may not need to use much of Uploader. The commonality seems to be  
the general template and styling, along with some of the core DOM  
manipulation logic.

> From the client side, we'll need some way of checking the status of  
> a job. Not sure if that's available currently. Also, I need the  
> steps clarified in the Export process. I suspect there is more  
> processing after the page-level processing, but I don't know what  
> they are or how long it takes.

At the moment, it's not possible to do, but we can certainly add it.  
We had been planning to switch from the CherryPy-based server to our  
Kettle JavaScript environment, but so far we haven't had a pressing  
enough reason to do so.

I'll defer to Boyan to give us a sense of how much work it will  
require to add job status tracking to the current Python codebase, and  
to Michelle or Antranig for what it would take to do so in Kettle.  
Clearly we've got a lot less available infrastructure in Kettle at  
this point than in CherryPy, so this may be an argument for sticking  
with it for awhile.

> Queuing is important for batch operations in Decapod. It's one of  
> the use cases.

Totally, I agree. Great wireframe, by the way. I think it balances  
nicely the desire for concrete information about how your jobs are  
progressing with the inevitable fuzziness of this sort of processing.

Colin

---
Colin Clark
Technical Lead, Fluid Project
http://fluidproject.org