Using Queues on Mission-Critical Workflows
In mission-critical systems, losing data is not an option. A direct function-to-function call can fail, leaving lost messages. Queues solve this by storing messages persistently until they are successfully processed. If a consumer function fails, the message returns to the queue and waits for another attempt.
Orixen uses Amazon SQS to provide this reliability. Messages can be retained for up to 14 days, giving you time to deploy fixes and reprocess failed messages without losing data.
Two Ways to Add a Queue
Orixen provides two approaches for adding queues to your workflow:
| Method | Best For |
|---|---|
| Queued Connection | Simple, point-to-point reliable messaging |
| Queue Block | Advanced scenarios like multiple consumers |
Queued Connection
The queued connection is the easiest way to add reliability between two blocks. Instead of a direct connection, messages flow through an SQS queue.
How It Works
- Producer sends message → Message is stored in the queue
- Consumer processes message → Message is removed from the queue
- If processing fails → Message returns to the queue for retry
This retry mechanism ensures transient failures (network issues, timeouts, temporary bugs) don't result in data loss.
Configuration
Queued connections can be configured in the deploy sidebar. Click on the connection to access queue settings.
For detailed configuration options, see the Queue Block documentation.
Queue Block
For advanced use cases, use the Queue block directly. This gives you full control over the queue and enables scenarios when you have multiple consumers processing from the same queue.
See Queue Block for configuration details.
Batch Processing
Queued messages can be processed in batches for greater efficiency. Instead of invoking your function once per message, multiple messages are delivered together, improving throughput for high-volume workloads.
To enable batch processing, configure the "batch size" property on the Function Block consuming data from queue.
Dead Letter Queues
Coming Soon
Dead Letter Queue support will be added in a future release.
A Dead Letter Queue (DLQ) captures messages that fail processing after multiple retry attempts. This prevents poison messages from blocking your queue while preserving them for investigation and manual reprocessing.