Skip to content

Queue Block

The Queue block creates an Amazon SQS queue that buffers messages between workflow steps. It guarantees message delivery and enables reliable, decoupled architectures.

AWS Service: Amazon SQS

Configuration Options

Property Default Range Description
fifo false true, false Enable FIFO (First-In-First-Out) ordering
visibilityTimeout 30 0 - 43,200 seconds (12 hours) Time a message is hidden after being read
messageRetentionPeriod 345,600 (4 days) 60 - 1,209,600 seconds (14 days) How long messages are kept in the queue
delaySeconds 0 0 - 900 seconds (15 minutes) Delay before messages become visible

Configuration Details

FIFO vs Standard Queues

Feature Standard Queue FIFO Queue
Message order Best-effort ordering Strict ordering guaranteed
Delivery At-least-once Exactly-once processing
Throughput Unlimited 300 messages/second (3,000 with batching)
Use case High throughput, order not critical Order matters, no duplicates

When to use FIFO

Use FIFO queues when message order matters (e.g., financial transactions, sequential processing) or when you need exactly-once processing.

Visibility Timeout

When a consumer reads a message, it becomes invisible to other consumers for this duration.

Scenario Recommended Timeout
Quick processing (< 10s) 30 seconds
Standard processing 60 seconds
Long processing Match your function timeout + buffer

Important

Set visibility timeout longer than your function's execution time. If processing takes longer than visibility timeout, the message may be processed again.

Formula: visibilityTimeout = functionTimeout + 30 seconds buffer

Message Retention Period

How long unprocessed messages remain in the queue before deletion.

Duration Use Case
60 seconds (1 minute) Very short-lived messages
345,600 seconds (4 days) Default, suitable for most cases
1,209,600 seconds (14 days) Maximum retention, critical messages

Delay Seconds

Postpones the delivery of new messages to the queue.

Delay Use Case
0 seconds Immediate processing
60 seconds Rate limiting, throttling
900 seconds (15 min) Scheduled delayed processing

AWS Reference

Example Usage

Reliable Processing Pipeline

API Trigger --> Function (validate) --> Queue --> Function (process)

The queue ensures that even if the processing function fails, messages are retained and reprocessed.

Decoupled Microservices

Service A --> Queue --> Service B

Queue acts as a buffer between services, allowing them to scale independently.

Best Practices

  1. Match visibility timeout to processing time: Prevent duplicate processing
  2. Use FIFO for ordered workloads: When sequence matters
  3. Implement Dead Letter Queues: Handle failed messages (configure in AWS Console)
  4. Monitor queue depth: Set up CloudWatch alarms for backlog
  5. Use delay for rate limiting: Prevent overwhelming downstream services