Observability Showcase

Category: Engineering & DevOps · 🔒 Self-hosted only

This page is generated from the Air Pipe marketplace. Browse it live to install into your organization.

Self-hosted pack — deploy this config and three things appear immediately with zero extra work:

/metrics — Prometheus scrape endpoint, auto-populated with HTTP request metrics for every route
/documentation — Swagger UI, auto-generated from the assertions and metadata in the config
Distributed tracing — set AIRPIPE_OTEL_ENDPOINT and AirPipe emits OTLP spans to any compatible collector (Jaeger, Grafana Tempo, Honeycomb, Datadog, etc.). Every request becomes a trace you can inspect end-to-end: method, path, status code, latency, and which actions ran inside it.

No annotations beyond the config. No code. No separate instrumentation setup.

What's included

File	Purpose
`config.yml`	Sample API with `docs: true` and full OpenAPI metadata
`schema.sql`	Users table with indexes
`grafana-dashboard.json`	Pre-built Grafana dashboard — import and go

Auto-docs: how it works

Adding docs: true to the config root enables documentation for every interface in that config. The doc system reads:

Config field	OpenAPI output
`summary`	Operation title
`description`	Operation description
`tags`	Grouping in the Swagger UI
`notes`	Appended to description
`request_example`	Example request body in the UI
`response_example`	Example response in the UI
Assertions on `input: a\|body`	Request body schema (field types, required, constraints)
Assertions on `input: a\|headers`	Header parameters
Assertions on `input: a\|params`	Query parameters

The Swagger UI is served at /documentation. The raw OpenAPI spec is at /docs.json.

Prometheus metrics: how it works

Every self-hosted AirPipe instance exposes /metrics automatically. There are two layers:

Built-in HTTP metrics

Metric	Type	Description
`axum_http_requests_total`	Counter	Request count by method, path, status
`axum_http_requests_duration_seconds`	Histogram	Latency per request
`axum_http_requests_pending`	Gauge	In-flight requests

Route labels use the interface name from the config (e.g. api/users/create), not the raw URL path. This means no cardinality explosion from IDs in paths — every interface is exactly one metric series regardless of traffic volume.

Custom business metrics via `emit_metric`

Any action can be replaced with an emit_metric action to emit a custom Prometheus metric at a specific point in the pipeline:

- name: TrackSignup
  run_when_succeeded: [CreateUser]
  hide_data_on_success: true
  emit_metric:
    name: app_users_created_total  # Prometheus metric name
    type: counter                  # counter | gauge | histogram
    labels:
      source: api                  # custom labels (values support interpolation)

Three metric types are supported:

Type	Use for	Value field
`counter`	counts of events (signups, errors, purchases)	optional — defaults to increment by 1
`gauge`	current state (active users, queue depth)	required — set to this value
`histogram`	distributions (response sizes, scores)	required — record this observation

Every emit_metric action automatically attaches config, interface, and action labels alongside any custom ones you define. Label values support the full interpolation syntax — a|ActionName::field| to label a metric with a value from the pipeline.

This config emits four custom metrics:

Metric	Type	Where
`app_users_total`	Gauge	`api/health` — live user count sampled on each health check
`app_users_listed_total`	Counter	`api/users` — increments on every list request
`app_users_created_total`	Counter	`api/users/create` — increments on successful signup
`app_users_looked_up_total`	Counter	`api/users/get` — increments on successful lookup
`app_errors_simulated_total`	Counter	`api/simulate/error` — for testing alerting pipelines

Setup

1. Set the environment variable

export DATABASE_URL=postgresql://user:pass@host:5432/dbname

2. Create the table

Either run the schema directly:

psql $DATABASE_URL -f schema.sql

Or use the seed endpoint after starting AirPipe — it creates the table and loads sample users in one step:

curl -X POST http://localhost:44111/api/seed

3. Run AirPipe

airpipe --config config.yml --address 0.0.0.0 --port 44111

4. Open the Swagger UI

http://localhost:44111/documentation

You should see a fully documented API with request/response examples and field schemas derived from the config's assertions.

5. Configure Prometheus

Add a scrape job to prometheus.yml:

scrape_configs:
  - job_name: airpipe
    static_configs:
      - targets: ['localhost:44111']
    metrics_path: /metrics
    scrape_interval: 10s

6. Import the Grafana dashboard

Open Grafana → Dashboards → Import
Upload grafana-dashboard.json
Select your Prometheus datasource when prompted

Generating test traffic

BASE=http://localhost:44111

# Successful reads
for i in $(seq 1 20); do curl -s $BASE/api/users > /dev/null; done

# Create some users
for i in $(seq 1 5); do
  curl -s -X POST $BASE/api/users/create \
    -H "Content-Type: application/json" \
    -d "{\"email\": \"[email protected]\", \"username\": \"user$i\", \"password\": \"Secret$i!\"}" > /dev/null
done

# Trigger 500s to populate the error-rate panel
for i in $(seq 1 5); do
  curl -s -X POST $BASE/api/simulate/error > /dev/null
done

Grafana dashboard panels

Panel	What it shows
Request Rate	Requests/sec by route
Error Rate	4xx + 5xx rate by route
Latency Percentiles	p50 / p95 / p99 in ms
In-Flight Requests	Current concurrent requests
Total Requests	Cumulative counter
Request Volume by Route	Relative route popularity
Status Code Distribution	2xx vs 4xx vs 5xx
p95 Latency Table	Slowest routes ranked

OpenTelemetry tracing

AirPipe emits OTLP spans automatically over HTTP Binary (not gRPC). Each span includes http.method, url.path, http.response.status_code, http.route, and server.address.

Enable tracing by setting AIRPIPE_OTEL_ENDPOINT before starting AirPipe:

export AIRPIPE_OTEL_ENDPOINT=http://localhost:4318
airpipe --config config.yml --address 0.0.0.0 --port 44111

Additional env vars:

Variable	Purpose	Default
`AIRPIPE_OTEL_ENDPOINT`	OTLP HTTP collector URL	— (tracing disabled if unset)
`AIRPIPE_OTEL_HDR_<name>`	Extra request headers (e.g. `AIRPIPE_OTEL_HDR_AUTHORIZATION`)	—
`AIRPIPE__OTEL_SAMPLE_RATIO`	Sampling ratio 0.0–1.0	1.0
`AIRPIPE__OTEL_EXPORT_TIMEOUT_SECS`	Export timeout in seconds	30

To spin up a local collector (Jaeger UI at http://localhost:16686):

# Port 4318 = OTLP HTTP (required). Port 4317 = OTLP gRPC (not used by AirPipe).
docker run -p 16686:16686 -p 4318:4318 jaegertracing/all-in-one

Notes

/metrics is only available in self-hosted mode. Managed deployments expose metrics through the AirPipe platform.
/documentation is available in both modes. In managed mode it is served at /{org}/{env}/docs.
The description field on test assertions feeds directly into the Swagger UI field descriptions — write it as if it were API documentation.

Configuration

config.yml

name: ObservabilityShowcase
description: >
  A sample REST API that demonstrates Air Pipe's built-in observability.
  Deploy this config and you get Prometheus metrics at /metrics,
  a Swagger UI at /documentation, and custom business metrics emitted
  via emit_metric actions — no extra instrumentation required.

docs: true

# Self-hosted pack — uses a|env::| for configuration.
# Run: airpipe --config config.yml --address 0.0.0.0 --port 44111
#
# Optional OpenTelemetry tracing (OTLP HTTP):
#   AIRPIPE_OTEL_ENDPOINT        — collector URL, e.g. http://localhost:4318
#   AIRPIPE_OTEL_HDR_<name>      — extra headers (e.g. AIRPIPE_OTEL_HDR_AUTHORIZATION)
#   AIRPIPE__OTEL_SAMPLE_RATIO   — sampling ratio 0.0–1.0 (default: 1.0)
#   AIRPIPE__OTEL_EXPORT_TIMEOUT_SECS — export timeout in seconds (default: 30)

global:
  databases:
    main:
      driver: postgres
      conn_string: "a|env::DATABASE_URL|"

interfaces:

  # ── Seed ──────────────────────────────────────────────────────────────────

  # POST /api/seed
  # Creates the users table and indexes (idempotent) then loads sample users.
  # Safe to re-run — truncates before inserting.
  api/seed:
    output: http
    method: POST

    actions:
      - name: CreateUsersTable
        database: main
        hide_data_on_success: true
        query: |
          CREATE TABLE IF NOT EXISTS users (
            id            BIGSERIAL    PRIMARY KEY,
            email         TEXT         NOT NULL UNIQUE,
            username      TEXT         NOT NULL UNIQUE,
            password_hash TEXT         NOT NULL,
            created_at    TIMESTAMPTZ  NOT NULL DEFAULT NOW()
          );

      - name: CreateIndexes
        run_when_succeeded: [CreateUsersTable]
        database: main
        hide_data_on_success: true
        query: |
          CREATE INDEX IF NOT EXISTS idx_users_email      ON users (email);
          CREATE INDEX IF NOT EXISTS idx_users_created_at ON users (created_at DESC);

      - name: TruncateUsers
        run_when_succeeded: [CreateIndexes]
        database: main
        hide_data_on_success: true
        query: |
          TRUNCATE users RESTART IDENTITY;

      - name: InsertUsers
        run_when_succeeded: [TruncateUsers]
        database: main
        hide_data_on_success: true
        query: |
          INSERT INTO users (email, username, password_hash) VALUES
            ('[email protected]',   'alice',   'hashed_pw_1'),
            ('[email protected]',     'bob',     'hashed_pw_2'),
            ('[email protected]',   'carol',   'hashed_pw_3'),
            ('[email protected]',    'dave',    'hashed_pw_4'),
            ('[email protected]',     'eve',     'hashed_pw_5');

      - name: SeedSummary
        run_when_succeeded: [InsertUsers]
        database: main
        query: |
          SELECT COUNT(*) AS users_inserted FROM users;
        post_transforms:
          - extract_value: "[0]"

  # ── Health ────────────────────────────────────────────────────────────────

  api/health:
    output: http
    summary: Health check
    description: Liveness probe. Returns server time and connected database name.
    tags: [system]
    response_example:
      server_time: "2024-01-01T12:00:00Z"
      database: mydb

    actions:
      - name: Health
        database: main
        query: |
          SELECT NOW() AS server_time, current_database() AS database;
        post_transforms:
          - extract_value: "[0]"

      # Gauge: total live users sampled on every health check.
      # Visible in Grafana as app_users_total{config=..., interface=api/health}
      - name: SampleUserCount
        run_when_succeeded: [Health]
        database: main
        hide_data_on_success: true
        query: |
          SELECT COUNT(*) AS total FROM users;
        post_transforms:
          - extract_value: "[0]"

      - name: EmitUserGauge
        run_when_succeeded: [SampleUserCount]
        hide_data_on_success: true
        emit_metric:
          name: app_users_total
          type: gauge
          value: a|SampleUserCount::total|

  # ── Users ─────────────────────────────────────────────────────────────────

  api/users:
    output: http
    summary: List users
    description: Returns the 50 most recently created users.
    tags: [users]
    response_example:
      - id: 1
        email: [email protected]
        username: alice
        created_at: "2024-01-01T12:00:00Z"

    actions:
      - name: ListUsers
        database: main
        query: |
          SELECT id, email, username, created_at
          FROM users
          ORDER BY created_at DESC
          LIMIT 50;

      # Counter: track how many times the list endpoint is called.
      - name: TrackListRequest
        hide_data_on_success: true
        emit_metric:
          name: app_users_listed_total
          type: counter

  api/users/create:
    output: http
    method: POST
    summary: Create user
    description: Creates a new user. Password is bcrypt-hashed before storage.
    tags: [users]
    request_example:
      email: [email protected]
      username: jane
      password: "Secret1!"
    response_example:
      id: 6
      email: [email protected]
      username: jane
      created_at: "2024-01-01T12:00:00Z"

    actions:
      - name: ValidateBody
        input: a|body|
        hide_data_on_success: true
        assert:
          http_code_on_error: 400
          tests:
            - value: email
              is_not_null: true
              is_not_empty: true
              description: "User email address"
            - value: username
              is_not_null: true
              is_not_empty: true
              description: "Unique username"
            - value: password
              is_not_null: true
              is_not_empty: true
              is_greater_than: 7
              description: "Password (minimum 8 characters)"

      - name: HashPassword
        run_when_succeeded:
          actions: [ValidateBody]
          http_code_on_error: 400
        input: a|ValidateBody|
        hide_data_on_success: true
        hide_data_on_error: true
        post_transforms:
          - bcrypt:
              value: password
              cost: 12

      - name: CreateUser
        run_when_succeeded:
          actions: [HashPassword]
          http_code_on_error: 400
        database: main
        query: |
          INSERT INTO users (email, username, password_hash)
          VALUES ($1, $2, $3)
          RETURNING id, email, username, created_at;
        params:
          - a|ValidateBody::email|
          - a|ValidateBody::username|
          - a|HashPassword::password|
        post_transforms:
          - extract_value: "[0]"

      # Counter: increment on every successful signup.
      # Labels let you slice by any dimension you care about.
      - name: TrackSignup
        run_when_succeeded: [CreateUser]
        hide_data_on_success: true
        emit_metric:
          name: app_users_created_total
          type: counter
          labels:
            source: api

  api/users/get:
    output: http
    method: POST
    summary: Get user
    description: Fetch a single user by ID.
    tags: [users]
    request_example:
      id: 1
    response_example:
      id: 1
      email: [email protected]
      username: alice
      created_at: "2024-01-01T12:00:00Z"

    actions:
      - name: ValidateBody
        input: a|body|
        hide_data_on_success: true
        assert:
          http_code_on_error: 400
          tests:
            - value: id
              is_not_null: true
              description: "User ID"

      - name: GetUser
        run_when_succeeded:
          actions: [ValidateBody]
          http_code_on_error: 400
        database: main
        query: |
          SELECT id, email, username, created_at FROM users WHERE id = $1;
        params:
          - a|ValidateBody::id|
        assert:
          http_code_on_error: 404
          error_message: "User not found"
          tests:
            - value: count()
              is_equal_to: 1
        post_transforms:
          - extract_value: "[0]"

      # Counter: track lookups separately from list calls.
      - name: TrackLookup
        run_when_succeeded: [GetUser]
        hide_data_on_success: true
        emit_metric:
          name: app_users_looked_up_total
          type: counter

  # ── System ────────────────────────────────────────────────────────────────

  # This endpoint always returns 500. Use it to populate the error-rate
  # panels in Grafana and verify alerting works.
  api/simulate/error:
    output: http
    method: POST
    summary: Simulate error
    description: >
      Always returns HTTP 500. Use this to verify error-rate panels and alerts
      in Grafana without touching real data.
    tags: [system]
    notes: |
      This endpoint is intentionally broken. It is safe to call repeatedly.

    actions:
      - name: ForceError
        database: main
        query: SELECT 1 AS result;
        assert:
          http_code_on_error: 500
          tests:
            - value: count()
              is_equal_to: 9999

      # Counter: track simulated errors so you can verify your alerting pipeline
      # fires correctly without waiting for real errors to occur.
      - name: TrackSimulatedError
        hide_data_on_success: true
        emit_metric:
          name: app_errors_simulated_total
          type: counter
          labels:
            endpoint: simulate_error

schema.sql

CREATE TABLE IF NOT EXISTS users (
  id            BIGSERIAL    PRIMARY KEY,
  email         TEXT         NOT NULL UNIQUE,
  username      TEXT         NOT NULL UNIQUE,
  password_hash TEXT         NOT NULL,
  created_at    TIMESTAMPTZ  NOT NULL DEFAULT NOW()
);

CREATE INDEX IF NOT EXISTS idx_users_email      ON users (email);
CREATE INDEX IF NOT EXISTS idx_users_created_at ON users (created_at DESC);

grafana-dashboard.json

{
  "__inputs": [
    {
      "name": "DS_PROMETHEUS",
      "label": "Prometheus",
      "type": "datasource",
      "pluginId": "prometheus"
    }
  ],
  "title": "AirPipe Overview",
  "uid": "airpipe-overview",
  "schemaVersion": 38,
  "version": 1,
  "refresh": "10s",
  "time": { "from": "now-1h", "to": "now" },
  "panels": [
    {
      "id": 1,
      "title": "Request Rate (req/s)",
      "type": "timeseries",
      "gridPos": { "x": 0, "y": 0, "w": 12, "h": 8 },
      "datasource": { "type": "prometheus", "uid": "${DS_PROMETHEUS}" },
      "targets": [
        {
          "expr": "sum(rate(axum_http_requests_total[1m])) by (path)",
          "legendFormat": "{{path}}"
        }
      ],
      "fieldConfig": {
        "defaults": { "unit": "reqps", "custom": { "lineWidth": 2 } }
      }
    },
    {
      "id": 2,
      "title": "Error Rate (4xx + 5xx)",
      "type": "timeseries",
      "gridPos": { "x": 12, "y": 0, "w": 12, "h": 8 },
      "datasource": { "type": "prometheus", "uid": "${DS_PROMETHEUS}" },
      "targets": [
        {
          "expr": "sum(rate(axum_http_requests_total{status=~\"4..|5..\"}[1m])) by (path, status)",
          "legendFormat": "{{path}} {{status}}"
        }
      ],
      "fieldConfig": {
        "defaults": {
          "unit": "reqps",
          "color": { "mode": "fixed", "fixedColor": "red" },
          "custom": { "lineWidth": 2 }
        }
      }
    },
    {
      "id": 3,
      "title": "Latency Percentiles (ms)",
      "type": "timeseries",
      "gridPos": { "x": 0, "y": 8, "w": 16, "h": 8 },
      "datasource": { "type": "prometheus", "uid": "${DS_PROMETHEUS}" },
      "targets": [
        {
          "expr": "histogram_quantile(0.50, sum(rate(axum_http_requests_duration_seconds_bucket[1m])) by (le, path)) * 1000",
          "legendFormat": "p50 {{path}}"
        },
        {
          "expr": "histogram_quantile(0.95, sum(rate(axum_http_requests_duration_seconds_bucket[1m])) by (le, path)) * 1000",
          "legendFormat": "p95 {{path}}"
        },
        {
          "expr": "histogram_quantile(0.99, sum(rate(axum_http_requests_duration_seconds_bucket[1m])) by (le, path)) * 1000",
          "legendFormat": "p99 {{path}}"
        }
      ],
      "fieldConfig": {
        "defaults": { "unit": "ms", "custom": { "lineWidth": 2 } }
      }
    },
    {
      "id": 4,
      "title": "In-Flight Requests",
      "type": "stat",
      "gridPos": { "x": 16, "y": 8, "w": 4, "h": 4 },
      "datasource": { "type": "prometheus", "uid": "${DS_PROMETHEUS}" },
      "targets": [
        {
          "expr": "sum(axum_http_requests_pending)",
          "legendFormat": "in-flight"
        }
      ],
      "fieldConfig": {
        "defaults": { "unit": "short", "thresholds": {
          "steps": [
            { "color": "green", "value": 0 },
            { "color": "yellow", "value": 50 },
            { "color": "red", "value": 100 }
          ]
        }}
      }
    },
    {
      "id": 5,
      "title": "Total Requests",
      "type": "stat",
      "gridPos": { "x": 20, "y": 8, "w": 4, "h": 4 },
      "datasource": { "type": "prometheus", "uid": "${DS_PROMETHEUS}" },
      "targets": [
        {
          "expr": "sum(axum_http_requests_total)",
          "legendFormat": "total"
        }
      ],
      "fieldConfig": {
        "defaults": { "unit": "short" }
      }
    },
    {
      "id": 6,
      "title": "Request Volume by Route",
      "type": "bargauge",
      "gridPos": { "x": 16, "y": 12, "w": 8, "h": 8 },
      "datasource": { "type": "prometheus", "uid": "${DS_PROMETHEUS}" },
      "targets": [
        {
          "expr": "sum(increase(axum_http_requests_total[1h])) by (path)",
          "legendFormat": "{{path}}"
        }
      ],
      "fieldConfig": {
        "defaults": { "unit": "short" }
      },
      "options": { "orientation": "horizontal", "reduceOptions": { "calcs": ["lastNotNull"] } }
    },
    {
      "id": 7,
      "title": "Status Code Distribution",
      "type": "piechart",
      "gridPos": { "x": 0, "y": 16, "w": 8, "h": 8 },
      "datasource": { "type": "prometheus", "uid": "${DS_PROMETHEUS}" },
      "targets": [
        {
          "expr": "sum(increase(axum_http_requests_total[1h])) by (status)",
          "legendFormat": "HTTP {{status}}"
        }
      ]
    },
    {
      "id": 8,
      "title": "p95 Latency by Route (last 5m)",
      "type": "table",
      "gridPos": { "x": 8, "y": 16, "w": 16, "h": 8 },
      "datasource": { "type": "prometheus", "uid": "${DS_PROMETHEUS}" },
      "targets": [
        {
          "expr": "histogram_quantile(0.95, sum(rate(axum_http_requests_duration_seconds_bucket[5m])) by (le, path)) * 1000",
          "legendFormat": "{{path}}",
          "instant": true
        }
      ],
      "fieldConfig": {
        "defaults": { "unit": "ms" }
      },
      "options": { "sortBy": [{ "displayName": "Value", "desc": true }] }
    }
  ]
}

Observability Showcase

What's included​

Auto-docs: how it works​

Prometheus metrics: how it works​

Built-in HTTP metrics​

Custom business metrics via emit_metric​

Setup​

1. Set the environment variable​

2. Create the table​

3. Run AirPipe​

4. Open the Swagger UI​

5. Configure Prometheus​

6. Import the Grafana dashboard​

Generating test traffic​

Grafana dashboard panels​

OpenTelemetry tracing​

Notes​

Configuration​

config.yml​

schema.sql​

grafana-dashboard.json​