Distributed Task Queue System

Repository Shape

Private repositories are represented through architecture notes, impact, and design tradeoffs instead of source links.

Role

Backend system design, queue semantics, observability, and production hardening across the worker service, scheduler API, and deployment manifests.

Architecture

• PostgreSQL stores durable job state, retry history, idempotency keys, and tenant ownership.
• Redis Lua scripts acquire leases atomically so multiple workers can compete without double execution.
• Stateless Kubernetes workers heartbeat active leases and reclaim abandoned jobs through reconciliation.

Highlights

• 40K+ daily jobs with P99 latency below 200ms.
• Tenant-hash sharding prevents noisy-neighbor job bursts from starving other tenants.
• Dead-letter queue and exponential backoff make poison jobs inspectable without blocking healthy work.

Constraints