Drawing Automation

I (Ward) have added automatically maintained dataflow diagrams to the federation search. I think the approach I have used has broad application for anyone who maintains complicated software with many interacting parts.

Federation search is only complex because it has grown over many years where coding preference have evolved. I've based this on what we have learned about manipulating Graphs and especially the Hypertext Super Collaborator.

In this reimplementation I have focused on GitHub actions and the potential for GitHub pages serving the role of the super collaborator with the minimum of "attitude" that might be objectionable in a mature engineering organization. See Solo Super Collaborator

For example, this SVG is the cron base systematic indexing that runs every six hours: github

This is generated from annotations added to the scripts at the point where important interactions happen. I've attached links to every relation but these are only active if you view the SVG raw version: github

The annotations are of the form:

In the above example the system takes the perspective "Index". Other systems are "Query", "Status", and "Debug". I found it useful to be thoughtful categorizing perspectives and wrote a wiki page with examples about my choices. See Search Services

I've written two scripts that run in a GitHub action with each commit to the repo. GitHub understands SVG files and does an impressive job comparing old and new files so one can evaluate contributions in history and pull requests. I just finished adding links and color to these diagrams. You can see the SVG "diffs" here: github

Collapse the SVG text diff to focus on the image diffs which are offered in three versions, 2-up, Swipe, and Onion Skin. Try each. They look like this:

My focus has been on having a credible maintenance story understanding that most systems span multiple teams, multiple repos, and multiple generations of development. I have confirmed that diagrams that are shown to be incorrect or incomplete can be corrected by clicking to the mistake, editing in GitHub to make a correction, and then posting that as a pull request to the team ultimately responsibility for the code in question.