Resilient HTTP

Category: Backend & APIs

This page is generated from the Air Pipe marketplace. Browse it live to install into your organization.

Calling someone else's API and surviving it. Every endpoint here is self-contained — the examples call this same engine over HTTP (a|internal::port|) against fixture routes that answer 201, 402, 404 and 204 on purpose, so you can deploy the pack and curl it with no external service, no keys and no database.

It showcases:

expect_status — declare what counts as success for a call. Anything outside it fails the action, which is what makes retry and run_when_failed work against a real upstream without writing an assertion to translate a status into an outcome. Takes an exact code (404), a class (4xx), a list ([2xx, 404]), or any. Default: 2xx, plus 304 for a conditional request whose validator matched.
Non-2xx as data — a 404 meaning "no such customer" and a 402 meaning "declined" are answers, not outages. Declare them and branch on them.
idempotency — the same Idempotency-Key on every attempt of one execution, so a retry after a timeout cannot become a second charge. RFC 9110 §9.2.2 allows auto-retrying a non-idempotent request only when you have "some means to detect that the original request was never applied"; this is that means.
Retry classification — retry repeats transient failures (408, 429, 500, 502, 503, 504 and transport errors) and leaves permanent ones alone. Retry-After wins over the computed backoff, and full jitter stops a fleet of clients re-converging on a recovering server.
defaults — one declaration for a whole interface, when every call in it shares an expectation. An action that sets its own always wins.
Empty bodies — 204 No Content and 304 Not Modified carry no body by definition; the body decodes to null and the action succeeds, with the status and headers still readable.

Requires Air Pipe 1.44+ (expect_status, idempotency, defaults, retry classification).

Endpoints

Method	Path	Demonstrates
`GET`	`/http/lookup`	`expect_status: [2xx, 404]` — a 404 is data; branch on it
`POST`	`/http/charge`	a 402 decline read as data instead of flattened into a 500
`POST`	`/http/pay`	`idempotency` — a retried POST that cannot charge twice
`DELETE`	`/http/delete`	a `204` with no body succeeds
`GET`	`/http/health-sweep`	`defaults: { expect_status: any }` across a whole interface
—	`/http/stub/*`	fixtures answering 201 / 402 / 404 / 204

Try it

BASE=https://your-airpipe-host

# 1. A 404 that means "absent" — the action SUCCEEDS and the guard branches on it.
curl $BASE/http/lookup
# → "Absent":  { "found": false, "message": "no such customer" }
# → "Present": skipped: condition not met

# 2. A decline is an answer. The provider's 402 and its decline code survive.
curl -X POST $BASE/http/charge -H 'content-type: application/json' -d '{}'
# → "Declined": { "declined": true, "reason": "insufficient_funds" }

# 3. Safe retry. The key is generated once per execution and repeated on every
#    attempt; the fixture echoes back what it received.
curl -X POST $BASE/http/pay -H 'content-type: application/json' -d '{}'
# → "Result": { "charged": true, "idempotency_key": "01a013f9-6fa0-70d2-a3b9-6ce724c49255" }

# 4. An empty 204 is not a parse failure.
curl -X DELETE $BASE/http/delete
# → "Confirm": { "deleted": true, "status": 204 }

# 5. One unhealthy dependency does not fail the sweep.
curl $BASE/http/health-sweep
# → "Report": { "checks": [ { "name": "payments", "status": 402 },
#                           { "name": "catalogue", "status": 404 } ] }

The rule worth remembering

The outcome is declared, not inferred. expect_status decides whether an action failed; an assert validates the data and never decides the outcome. So this does not do what it looks like:

# WRONG — the action fails on the 402 before the assert ever runs
http: { url: https://api.example.com/charge, method: POST }
assert:
  tests:
    - value: status
      is_equal_to: 402

# RIGHT — say what you expect, then read it as data
http: { url: https://api.example.com/charge, method: POST, expect_status: [2xx, 402] }

One rule in one place: read an action and you can see what counts as failure, without tracing which assertions happen to mention status.

Adapting it

Point the URLs at your real provider and drop the /http/stub/* interfaces:

Stripe / Adyen / Square — they already honour Idempotency-Key, which is the header idempotency sends by default. Set idempotency.header for a provider that spells it differently.
Declines — expect_status: [2xx, 402] for cards, [2xx, 409] for "already exists" on a create, [2xx, 404] for a lookup that may miss.
Polling — a GET that answers 404 until a record exists wants expect_status: [2xx, 404] plus retry, so the retry is driven by your assert on the body rather than by the status.
Rate limits — 429 is already in the default retry set and Retry-After is honoured, so a provider that asks you to wait is obeyed without extra config.

Configuration

config.yml

name: ResilientHttp
description: >
  Calling someone else's API and surviving it: declare which statuses are success
  (`expect_status`), retry only what could plausibly succeed, send an idempotency
  key so a retried POST cannot charge twice, and treat a 404 or a 402 as data
  instead of an error. Every endpoint calls this same engine over HTTP, so the
  examples run against real status codes with no external service to set up.

docs: true

interfaces:

  # ── Fixtures ────────────────────────────────────────────────────────────────
  #
  # Four routes that answer with a chosen status, so the examples below exercise
  # real responses. `a|internal::port|` is the port this engine is listening on,
  # which is what keeps the pack self-contained.

  http/stub/ok:
    output: http
    method: POST
    summary: Answers 201 and echoes the headers it received
    tags: [resilient-http, fixture]
    actions:
      - name: Echo
        input: a|headers|
        # A custom body, so the fixture answers like a third-party API would —
        # its own payload, not an Air Pipe envelope.
        response_on_success:
          http_code: 201
          body: '{"idempotency_key": "a|headers::idempotency-key|", "seen": true}'

  http/stub/missing:
    output: http
    summary: Answers 404 (the resource is genuinely absent)
    tags: [resilient-http, fixture]
    actions:
      - name: Missing
        json_output: '{}'
        response_on_success:
          http_code: 404
          body: '{"error": "no such customer"}'

  http/stub/declined:
    output: http
    method: POST
    summary: Answers 402 (a payment decline, which is an ANSWER not an outage)
    tags: [resilient-http, fixture]
    actions:
      - name: Declined
        json_output: '{}'
        response_on_success:
          http_code: 402
          body: '{"decline_code": "insufficient_funds"}'

  http/stub/no-content:
    output: http
    method: DELETE
    summary: Answers 204 with no body at all
    tags: [resilient-http, fixture]
    actions:
      - name: Deleted
        json_output: '{}'
        response_on_success:
          http_code: 204

  # ── 1. A 404 that means "absent", not "broken" ──────────────────────────────
  #
  # The default is that a response outside 2xx (plus 304) FAILS the action — that
  # is what makes `run_when_failed` and `retry` work against a real upstream. But
  # plenty of non-2xx responses are the answer you asked for. Declare them and the
  # action succeeds with the payload intact, so the pipeline can branch on it.
  http/lookup:
    output: http
    summary: Treat 404 as "not found" data and branch on it
    description: >
      `expect_status: [2xx, 404]` says a 404 is an answer, not an outage. The
      action succeeds, and a `run_on_assertion` guard turns the status into a
      branch — no error handling required for an expected outcome.
    tags: [resilient-http, expect_status, branching]
    produces:
      example:
        found: false
        message: no such customer

    actions:
      - name: Fetch
        http:
          url: "http://127.0.0.1:a|internal::port|/http/stub/missing"
          method: GET
          expect_status: [2xx, 404]

      - name: Absent
        run_when_succeeded: [Fetch]
        run_on_assertion:
          tests:
            - action: Fetch
              value: status
              is_equal_to: 404
        json_output: |
          { "found": false, "message": "no such customer" }

      - name: Present
        run_when_succeeded: [Fetch]
        run_on_assertion:
          tests:
            - action: Fetch
              value: status
              data_type: Number
              is_less_than: 300
        json_output: |
          { "found": true }

  # ── 2. A 402 decline reaches the caller as a 402 ────────────────────────────
  #
  # A declined card is not a 500. `expect_status` keeps the response as data, and
  # `http_code_inherit_error` lets the interface answer with the upstream's own
  # status so the client sees 402 rather than a generic failure.
  http/charge:
    output: http
    method: POST
    summary: Pass an upstream decline through with its own status code
    description: >
      The payment provider answers 402 with a decline code. The action treats it
      as data, the pipeline reads the reason, and the interface returns 402 to the
      caller instead of flattening it into a 500.
    tags: [resilient-http, payments, expect_status]
    produces:
      example:
        declined: true
        reason: insufficient_funds

    actions:
      - name: Charge
        http:
          url: "http://127.0.0.1:a|internal::port|/http/stub/declined"
          method: POST
          headers:
            content-type: application/json
          body:
            amount: 2500
          expect_status: [2xx, 402]

      - name: Declined
        run_when_succeeded: [Charge]
        run_on_assertion:
          tests:
            - action: Charge
              value: status
              is_equal_to: 402
        json_output: |
          { "declined": true, "reason": a|Charge::body.decline_code->double_quote| }

  # ── 3. Retrying a POST without charging twice ───────────────────────────────
  #
  # RFC 9110 §9.2.2 permits automatically retrying a non-idempotent request only
  # when you have "some means to know that the request semantics are actually
  # idempotent... or some means to detect that the original request was never
  # applied". An idempotency key is that means: every attempt of ONE execution
  # sends the SAME key, so the provider collapses duplicates. A new request gets
  # a new key.
  #
  # `idempotency: { key: ... }` defaults to the `Idempotency-Key` header, which is
  # what Stripe, Adyen, Square and the IETF draft all use; set `header:` for a
  # provider that spells it differently.
  http/pay:
    output: http
    method: POST
    summary: Retry a payment safely with an idempotency key
    description: >
      Sends `Idempotency-Key` with the charge and keeps it identical across retry
      attempts, so a retry after a timeout cannot become a second charge. The stub
      echoes the key it received.
    tags: [resilient-http, payments, idempotency, retry]
    produces:
      example:
        charged: true
        idempotency_key: 018f...-7c3e

    actions:
      - name: Pay
        retry:
          attempts: 3
          delay: 200
          exponential_backoff: true
        http:
          url: "http://127.0.0.1:a|internal::port|/http/stub/ok"
          method: POST
          headers:
            content-type: application/json
          body:
            amount: 2500
          expect_status: [2xx]
          idempotency:
            key: a|uuid|

      - name: Result
        run_when_succeeded: [Pay]
        json_output: |
          { "charged": true, "idempotency_key": a|Pay::body.idempotency_key->double_quote| }

  # ── 4. A response with no body is not a failure ─────────────────────────────
  #
  # 204 No Content and 304 Not Modified carry no body by definition, and a 200
  # with content-length 0 is legal. The body decodes to null and the action
  # succeeds — the status and headers are still there to read.
  http/delete:
    output: http
    method: DELETE
    summary: A 204 with an empty body succeeds
    tags: [resilient-http, expect_status]
    produces:
      example:
        deleted: true
        status: 204

    actions:
      - name: Remove
        http:
          url: "http://127.0.0.1:a|internal::port|/http/stub/no-content"
          method: DELETE

      - name: Confirm
        run_when_succeeded: [Remove]
        json_output: |
          { "deleted": true, "status": a|Remove::status| }

  # ── 5. One declaration for a whole interface ────────────────────────────────
  #
  # When every call in an interface shares an expectation — a health-check sweep
  # that records whatever each service answers, a test runner hitting routes that
  # return 4xx on purpose — declare it once with `defaults:` instead of repeating
  # it on each action. An action that sets its own always wins.
  http/health-sweep:
    output: http
    summary: Record whatever each dependency answers, without failing the run
    description: >
      `defaults: { expect_status: any }` makes every action in this interface read
      the status as data, so one unhealthy dependency does not fail the sweep.
    tags: [resilient-http, defaults, monitoring]
    produces:
      example:
        checks:
          - { name: payments, status: 402 }
          - { name: catalogue, status: 404 }

    defaults:
      expect_status: any

    actions:
      - name: Payments
        http:
          url: "http://127.0.0.1:a|internal::port|/http/stub/declined"
          method: POST

      - name: Catalogue
        http:
          url: "http://127.0.0.1:a|internal::port|/http/stub/missing"
          method: GET

      - name: Report
        run_when_succeeded: [Payments, Catalogue]
        json_output: |
          {
            "checks": [
              { "name": "payments",  "status": a|Payments::status| },
              { "name": "catalogue", "status": a|Catalogue::status| }
            ]
          }

Resilient HTTP

Endpoints​

Try it​

The rule worth remembering​

Adapting it​

Configuration​

config.yml​