stackmapstackmap
Install

DocsDiagram kinds

Dataflow

Data moving through stages: pipelines, ETL and ELT, change data capture, event streams, lineage.

Use it only when the edges are data moving, in the direction it moves: from the database to the capture job to the stream to the store. Stages band an ordered pipeline, from sources to consumers, in flow order.

An example

The skill’s own example, clickstream.dataflow.json, laid out by stackmap.

file:///…/.stackmap/clickstream/diagram.html
clickstream.dataflow.json103 lines
{  "kind": "dataflow",  "title": "Clickstream analytics",  "subtitle": "From SDK events to dashboards",  "direction": "DOWN",  "groups": [    { "id": "ingest", "label": "Ingest" },    { "id": "warehouse", "label": "Warehouse" }  ],  "nodes": [    {      "id": "sdk",      "type": "client",      "card": { "title": "Web SDK", "subtitle": "Browser events" }    },    {      "id": "collector",      "type": "gateway",      "group": "ingest",      "card": {        "title": "Collector",        "subtitle": "HTTP ingest",        "rows": [{ "label": "Peak", "value": "40k ev/s" }]      }    },    {      "id": "stream",      "type": "queue",      "group": "ingest",      "card": {        "title": "events",        "subtitle": "Kafka topic",        "rows": [{ "label": "Partitions", "value": "24" }]      }    },    {      "id": "enrich",      "type": "service",      "group": "ingest",      "card": { "title": "Enricher", "subtitle": "Stream processor" }    },    {      "id": "archive",      "type": "storage",      "card": { "title": "Raw archive", "subtitle": "Object storage" }    },    {      "id": "dwh",      "type": "database",      "group": "warehouse",      "card": {        "title": "Warehouse",        "subtitle": "Snowflake",        "brand": "snowflake"      }    },    {      "id": "models",      "type": "service",      "group": "warehouse",      "card": { "title": "dbt models", "subtitle": "Hourly build" }    },    {      "id": "bi",      "type": "client",      "card": {        "title": "Dashboards",        "subtitle": "Grafana",        "brand": "grafana"      }    }  ],  "edges": [    {      "id": "sdk-collector",      "from": "sdk",      "to": "collector",      "label": "batch POST"    },    {      "id": "collector-stream",      "from": "collector",      "to": "stream",      "kind": "async"    },    {      "id": "stream-enrich",      "from": "stream",      "to": "enrich",      "kind": "async"    },    {      "id": "stream-archive",      "from": "stream",      "to": "archive",      "kind": "async",      "label": "raw"    },    { "id": "enrich-dwh", "from": "enrich", "to": "dwh" },    { "id": "dwh-models", "from": "dwh", "to": "models" },    { "id": "models-bi", "from": "models", "to": "bi" }  ]}
Point at a node in the JSON, or at its card, to find the other.

Its parts

  • Stages

    Ordered bands: sources, ingest, process, store, consume.

  • Direction

    Edges follow the data, not who calls whom.

  • Async

    Streams and events render dashed.

  • Compact cards

    density compact for long chains.

Rules

From the authoring contract the skill gives your agent. stackmap validate enforces the hard ones and names the fix.

  • Edges follow the data: db → cdc → kafka → job → store; whoever reads a store is store → reader.
  • Stages (phases with nodes) band an ordered pipeline: sources, ingest, process, store, consume. Stage members can’t also be in a group.
  • "kind": "async" for streams and events: they render dashed.
  • "density": "compact" for long chains (more than about five stages) and overviews; keep full cards when rows and stats carry the answer.
  • Label an edge (24 characters at most) only where the relationship isn’t obvious from its ends: a topic, a protocol.

Ask for one

Show how data flows through our feature platform, from the product databases to the models that read it.

Your agent picks the kind that answers the question; naming it is the surest way to get it.

Examples

See dataflow examples in the gallery