Talking to other services

HTTP.get and HTTP.post, with timeouts that matter more than usual, and no way to make two calls at once.

HTTP is what you should use. net/http is on the require allow-list and Melee::HTTP is a thin client over it, so you can reach for it directly — but the wrapper exists because two Spinel traps sit in the obvious way of calling it: setting a header with req[k] = v and the block form of Net::HTTP.start both lose their static types and misbehave in a compiled app. The wrapper also gives you timeouts, redirect following and rescuable error classes. Use it.

res = HTTP.get("https://api.example/things", query: { "page" => "2" }, timeout: 5)
halt 502 unless res.ok?
things = res.json

HTTP is Melee::HTTP; the two spellings are the same for a call. They are not the same in a rescue clause — see below.

The shape of it

HTTP.get(url, query: {}, headers: {}, timeout: 10, raise_on_error: false)
HTTP.post(url, body: "...", query: {}, headers: {}, timeout: 10, raise_on_error: false)

The response has status, headers, body, url (after redirects), ok? (2xx), json and header(name).

There is no put and no delete, and no request signing. That rules out talking to most object stores today; it is a known gap.

Timeouts

timeout covers the connection and the read, in seconds, and it is the single most important argument here. Two reasons:

A request that hangs holds one of your app’s four slots for up to 30 seconds and then gets killed, taking the visitor’s request with it.
A durable object’s call blocks the app’s whole worker while it runs, including calls from other objects.

timeout is rounded up to a whole number of seconds, so timeout: 0.5 is the same as timeout: 1 — there is no sub-second timeout. Set it low and deliberately:

HTTP.get(url, timeout: 3)          # a request is waiting on this
HTTP.get(url, timeout: 20)         # a timer is doing it; nobody is waiting

Always rescue

An unrescued failure becomes a 500 for the visitor.

begin
  res = HTTP.get(url, timeout: 5)
  parse(res.body)
rescue Melee::HTTP::TimeoutError
  log.warn "upstream slow", url: url
  halt 504
rescue Melee::HTTP::Error => e
  log.warn "upstream failed", url: url, error: e.message
  halt 502
end

Write Melee::HTTP::Error in full. A rescue clause that reaches its class through a constant alias — rescue HTTP::Error — does not match in a compiled app, while matching perfectly under melee dev. So the app looks correct in development and does not rescue at all in production. melee check rejects the short spelling and names the line, which is the one place this trap is caught for you.

The classes: Melee::HTTP::Error is the parent; Melee::HTTP::TimeoutError and Melee::HTTP::TooManyRedirects (after five) inherit from it. A 4xx or 5xx is not an error unless you ask for raise_on_error: true — otherwise check res.ok?.

TLS

HTTPS verifies certificates and there is no way to turn that off. That is deliberate.

Doing it once instead of every request

The pattern that makes a slow upstream stop mattering: fetch on a timer, read from storage.

class Feed < Durable
  def setup = timer(after: 0)

  def on_timer
    fetch
    timer after: 600
  end

  def fetch
    res = Melee::HTTP.get("https://example.com/feed.json", timeout: 20)
    storage.put("items", res.json)
    storage.put("fetched_at", Time.now.to_i)
  rescue Melee::HTTP::Error => e
    log.warn "feed failed", error: e.message      # keep the last good copy
  end

  def items = storage.get("items") || []

  def stale? = Time.now.to_i - storage.get("fetched_at").to_i > 3600
end

get "/" do
  render :index, items: Feed.get("main").items, stale: Feed.get("main").stale?
end

The visitor’s request never touches the network. A failed fetch leaves the previous copy in place, and stale? lets the page say so. Background jobs and scheduled work is the fuller treatment.

No concurrency

There is no way to make two outbound requests at the same time. No threads, no fibers you can schedule, no Async. Ten feeds means ten sequential fetches — which is fine on a timer and not fine in a request.

What to watch out for

Nothing stops an app calling a private address today. If you fetch a URL a visitor supplied, you are one step from a request against 169.254.169.254 or a service on the host’s network. Validate the host yourself, and prefer an allow-list. (Refusing private addresses by default is a known open task.)
A response body is a String in memory, inside a 128 MB cap. Do not fetch something enormous.
Redirects are followed up to five times, and res.url tells you where the body actually came from.

Keyboard shortcuts

melee