Batch Jobs

We explore representing batch job dependencies as wiki pages and generate interactive visualizations from these.

Overnight jobs were organized as job steps with dependencies. These were collected into "boxes" which simplified scheduling. Boxes further obeyed a naming convention that indicated organizational ownership for the jobs.

The database did not explicitly represent preconditions which made it difficult to determine when a job would run. This was managed by hand using Visio and was reported to be a full-time job.

I read the entire database into memory and constructed the two-way dependency graph. I converted this to the nodes and links structure used by the d3.js force layout example and wrote it out for each organizational unit.

This was found and displayed by the "force" plugin which normally appeared on its own separate page and could be called up at will.

The nodes have hover text that identify the box, sub box or job. Drag them to see more of the graph. Double-click them to open a page describing the box.

Rendering the arrow heads was surprisingly difficult. The example did not come with them. It seems that SVG's designers did not want to design arrow heads and left this to users to employ SVG's exotic "marker" mechanism to craft their own.

The force plugin modifies the objects representing the json dataset. Double-clicking the data object shows the in-memory version that has been annotated by x, y, px and py variables used in the force simulation. This unintended editing demonstrates how wiki's read-write world differs from that assumed by the d3.js examples.

See source for Force Plugin on GitHub.

See example of Force-Directed Graph showing character co-occurence in Les Misérables.