Sikau

AWS · Day 6 · about 40 min

SQS queues

Visibility timeouts, at-least-once delivery and dead-letter queues.

Don't make users wait

Sending emails or resizing photos inside a web request makes it slow and fragile. Put a message on an SQS queue instead and let workers process it. A received message isn't deleted — it's hidden for a visibility timeout. If the worker crashes, it reappears for another try; after too many tries it moves to a dead-letter queue for a human to inspect.

Receive, work, delete

function createQueue({ visibilityTimeout = 30, maxReceiveCount = Infinity, deadLetter = null, now = () => Date.now() / 1000 } = {}) {
  const messages = [];
  let nextId = 1;
  const handleOf = (m) => `${m.id}#${m.receiveCount}`;
  return {
    send(body) {
      const id = `m${nextId++}`;
      messages.push({ id, body, receiveCount: 0, visibleAt: 0 });
      return id;
    },
    receive(max = 1) {
      const out = [];
      for (const m of [...messages]) {
        if (out.length >= max) break;
        if (m.visibleAt > now()) continue; // someone else is working on it
        if (m.receiveCount >= maxReceiveCount) {
          messages.splice(messages.indexOf(m), 1); // failed too often: park it
          deadLetter?.send(m.body);
          continue;
        }
        m.receiveCount++;
        m.visibleAt = now() + visibilityTimeout; // hidden, not deleted
        out.push({ id: m.id, body: m.body, receiveCount: m.receiveCount, receiptHandle: handleOf(m) });
      }
      return out;
    },
    delete(receiptHandle) {
      const i = messages.findIndex((m) => handleOf(m) === receiptHandle);
      if (i < 0) return false;
      messages.splice(i, 1);
      return true;
    },
    size: () => messages.length,
  };
}

The rest of this lesson

  • Visibility timeout, step by stepexplain
  • Designing workersexplain
  • A worker's looporder
  • Situation to outcomematch
  • Build a queuewrite code
  • Badge quizquiz
  • What you learnedrecap

Continue this lesson free — run real code in your browser

Continue with Google

See all 8 lessons of AWS