Async job roundtrip
Accept now, work in the background, notify later
Accept
Background work
Notify and reconcile
Client
Mobile app
Jobs API
Request edge
Queue
Durable work
Worker
Background
Provider
External API
Job store
Source of truth
Notifier
Webhook
POST /jobs
enqueue job
202 + job id
deliver
perform work
result or timeout
retry if timeout
persist final state
job.completed
signed webhook
GET /jobs/:id
read status
completed
200 final result
diagram.html#The link keeps the view, selection, route and playback.
About this diagram
Messages over time between a few participants, for one scenario: a request with a cache miss, an async job round trip.
Ask for one like it
Draw the request path for a dashboard load as a sequence, including the cache miss.
The JSON
{ "kind": "sequence", "title": "Async job roundtrip", "subtitle": "Accept now, work in the background, notify later", "phases": [ { "id": "accept", "label": "Accept", "edges": ["post", "enqueue", "accepted"] }, { "id": "work", "label": "Background work", "edges": ["deliver", "perform", "result", "retry", "persist"] }, { "id": "notify", "label": "Notify and reconcile", "edges": [ "completed", "webhook", "poll", "status", "state", "final" ] } ], "nodes": [ { "id": "client", "type": "client", "card": { "title": "Client", "subtitle": "Mobile app" } }, { "id": "api", "type": "service", "card": { "title": "Jobs API", "subtitle": "Request edge" } }, { "id": "queue", "type": "queue", "card": { "title": "Queue", "subtitle": "Durable work", "brand": "rabbitmq" } }, { "id": "worker", "type": "service", "card": { "title": "Worker", "subtitle": "Background" } }, { "id": "provider", "type": "external", "card": { "title": "Provider", "subtitle": "External API" } }, { "id": "store", "type": "database", "card": { "title": "Job store", "subtitle": "Source of truth", "brand": "postgresql" } }, { "id": "notifier", "type": "queue", "card": { "title": "Notifier", "subtitle": "Webhook" } } ], "edges": [ { "id": "post", "from": "client", "to": "api", "label": "POST /jobs", "tone": "main" }, { "id": "enqueue", "from": "api", "to": "queue", "label": "enqueue job", "kind": "async", "tone": "main" }, { "id": "accepted", "from": "api", "to": "client", "label": "202 + job id", "kind": "return" }, { "id": "deliver", "from": "queue", "to": "worker", "label": "deliver", "tone": "main" }, { "id": "perform", "from": "worker", "to": "provider", "label": "perform work" }, { "id": "result", "from": "provider", "to": "worker", "label": "result or timeout", "kind": "return" }, { "id": "retry", "from": "worker", "to": "queue", "label": "retry if timeout", "kind": "async", "tone": "error" }, { "id": "persist", "from": "worker", "to": "store", "label": "persist final state", "tone": "main" }, { "id": "completed", "from": "worker", "to": "notifier", "label": "job.completed", "kind": "async" }, { "id": "webhook", "from": "notifier", "to": "client", "label": "signed webhook", "kind": "async", "tone": "security" }, { "id": "poll", "from": "client", "to": "api", "label": "GET /jobs/:id" }, { "id": "status", "from": "api", "to": "store", "label": "read status" }, { "id": "state", "from": "store", "to": "api", "label": "completed", "kind": "return" }, { "id": "final", "from": "api", "to": "client", "label": "200 final result", "kind": "return" } ], "notes": [ { "title": "Latency contract", "items": [ "The client gets 202 before any work starts", "Retries stay inside the background phase" ] }, { "title": "Reconciliation", "items": [ "The webhook is signed", "Polling reads the job store, the source of truth" ] } ]}