Email Scheduler
- TypeScript
- Node.js
- Express
- BullMQ
- Redis
- PostgreSQL
- Prisma
- Elasticsearch
Demo
Overview
A production-grade email scheduling system built on BullMQ delayed jobs — no cron package anywhere, no lost emails, and no duplicate sends, even across restarts or a hard kill -9.
Highlights
- No cron, anywhere: every email is a persistent BullMQ delayed job in Redis (
jobId = email id). - Restart-proof: Redis AOF + Postgres as the source of truth, with reconciliation when the worker starts.
- Exactly-once claim: an atomic
UPDATE … WHERE status = 'scheduled'claims a send; an unknown SMTP outcome is reported, never resent. - Distributed rate limiting: one atomic Redis Lua script gives each sender a minimum delay and hourly cap, safe across any number of workers.
- 1000+ emails at once: O(1) slot reservation per job — 1000 emails on a 200/hour sender become 5 hours of evenly spaced delayed jobs, no polling or retry storms.
- Full-text search: Elasticsearch prefix/whole-word search on address, subject, and body via its own indexing queue.
- Deployed on Kubernetes: Kustomize manifests on Civo k3s with TLS, network policies, and 2 replicas per app tier.
Architecture
Overview
| Process | Responsibility |
|---|---|
| Express API | Auth, campaign scheduling, listing, search, Slack OAuth. Never sends email itself. |
| BullMQ worker | A separate process with two workers — email-send (rate limit → claim → SMTP → record → Slack) and search-index (writes Elasticsearch). Reconciles Postgres and Redis on start. |
| React dashboard | Vite + Tailwind + TanStack Query, proxied to the API in dev so the session cookie stays same-origin. |
| PostgreSQL | Source of truth: users, senders, campaigns, emails, Slack connections (via Prisma). |
| Redis | AOF on, noeviction: BullMQ delayed-job queues and rate-limit counters. |
| Elasticsearch | Search index only; rebuildable from Postgres at any time. |
Correctness under restarts and concurrency
- Every scheduled email is a BullMQ delayed job, so a crashed worker just picks up where Redis left off — no cron polling loop to miss a tick.
- A send is claimed with
UPDATE … WHERE status = 'scheduled', so two workers racing on the same job can never both send it. - An SMTP outcome that can't be confirmed is recorded as unknown and reported, not silently retried, to avoid duplicate sends.
Rate limiting
- A single atomic Redis Lua script enforces a minimum delay and hourly cap per sender; overflow is rescheduled to the next open slot instead of dropped or retried immediately — correct under any number of concurrent workers.
Gallery
