Async Polymorphic Fetch

Modern client-side javascript offers a function, fetch, which returns a Promise to the response of an http request. I (Ward) have launched as many as 450 simultaneous fetches and gotten good results. Amazing. Now we generalize fetch to operate as a federation citizen where the history of one page offers good advice as to where to find another. We call this searching "Polymorphism" because of the similarity to object-oriented polymorphic message dispatch.

> @nrn found my last bug. I forgot to share the site where I found the page. That can't possibly work. Now we call polyget which returns {site, page}.

The implementation has multiple moving parts.

We call `collaborators (journal, implicit)` to find and prioritize possible locations where a page may be found.

We call `probe (site, slug)` to engage all necessary machinery to asynchronously fetch a slug from a site.

It calls `wiki.site(site)` to retrieve or construct a suitable SiteAdapter based on discovered network protocols.

It calls `get(${slug}.json, (err, page) => page)` to asynchronously fetch a page using the node.js callback conventions which will automatically resolve the async function's implicit promise.

async function probe (site, slug) { return wiki.site(site) .get(`${slug}.json`, () => null) }

From this we compose an outer wrapper of a second async function and its implicit promise.

async function polyget (context) { let slug = asSlug(context.name) let sites = collaborators( context.page.journal, [context.site, location.host] ) for (let site of sites) { try { return { site, page: await probe(site,slug) } } catch (err) { // 404 } } return null }

In this case polyget has been customized to advance a visitor from page to page following a breadth-first traversal where the state at each step is held in context. wikipedia

The async behavior is valuable here because the visitor logic can launch a flotilla of sequential searches, each using the lazy logic of returning from the first success, before recursing to traverse the next deeper layer.

We have yet to exhaustively test all possible failures in all combinations. We have reason to believe one try-catch but not both are optional. See improvements. github

See Collaborative Link for operational motivation.