The Homelab Side: Pull-Only, Alerts and a Weekly Digest
The public path (sites, forms, confirmation emails) runs at the edge, so it keeps working when my server at home doesn’t. The homelab does the slower, private work: keeping a copy of every lead, running email sequences, sending me a summary. And it does it without accepting a single incoming connection for leads.
Pull, don’t push
The obvious design is for the Worker to push each lead to the homelab. That means exposing the homelab to the internet, and losing leads whenever it’s down. Instead, the homelab pulls:
flowchart LR
W["Forms Worker"] -->|write| KV[("KV")]
H["leads-hub (homelab)"] -->|"list + get, read-only token"| KV
H --> DB[("SQLite")]
H -->|backups| BK["Backup copies"]
H -->|sequences| M["Email provider"]
Every 15 minutes, a small Python program (standard library only) lists the KV keys, fetches new or changed values with a read-only API token, and upserts them into SQLite. Outbound HTTPS only. If the homelab is off for a week, it catches up when it comes back; KV is the buffer.
Details that made it robust:
- Skip non-lead prefixes. The KV namespace also holds the suppression list, spam and pending report requests; the mirror skips those prefixes and counts them separately.
- Mirror deletions. When a lead expires in KV (its TTL ran out), the
mirror marks it gone (
kv_gone_at) and purges it after a short grace period. It also deletes anything past its kind’s retention even if KV still had it. The copy never outlives the policy. - A safety brake. If a pull would mark more than half of the known leads
as gone at once (an API error or an empty listing looks exactly like “everything
was deleted”), it stops and asks for
--force. It tripped once, for the right reason: I had just deleted a batch of my own test leads. - SQLite is where analysis happens. KV is an inbox; questions like “how many qualified leads from LinkedIn this quarter?” are a SQL query on the mirror.
Phone alerts with self-hosted ntfy
ntfy is a small open-source push notification server. It runs on the homelab, reachable from the internet only through a Cloudflare Tunnel (an outbound connection from the homelab, so no open port). It’s locked down:
- deny-all by default;
- a write-only token for the Worker, which can publish to one topic and read nothing;
- an admin user for my phone.
Alerts carry no personal data: kind, site, email domain. A paid booking gets high priority; an enquiry is normal.
One trap cost me an evening: my homelab’s own failure alerts never arrived.
Cloudflare’s bot protection blocked Python’s default urllib User-Agent
(error 1010) before the request reached ntfy. Every script now sends an
explicit User-Agent.
A weekly digest: numbers only
Once a week, one low-priority notification:
Leads weekly - 2026-W40
New leads: 3
www.example.com enquiry: 2
webaudit.example.com audit: 1
Confirmed subscriptions: 1
Unsubscribes: 0
Drips sent: 4 (welcome 4)
Do-not-send list: 5 (+0)
Spam filtered (last 30 days): 2
Pulls: 672 (0 failed)
Counts only, never an address. It answers “is anything happening?” without opening a dashboard, and a week of zeros is a signal too.
Operating notes
- Unbuffered logs in Docker. Python buffers stdout when it isn’t a
terminal, so
docker logsshowed nothing;PYTHONUNBUFFERED=1fixed it. - Create bind-mount folders first. If Docker creates a missing host folder, it’s owned by root and the app can’t write to it.
- Never bind-mount a git repo with runtime data inside it. A production
container once wrote live leads into a CSV inside the repository folder,
and
git pullandgit statusstarted treating customer data as code. Runtime data lives outside the repo, and is gitignored anyway. - Kill by port, not by pattern.
pkill -f somethingcan match the shell running it.
What the homelab is not
It’s not on the critical path. If it burns down, visitors notice nothing, leads keep arriving in KV, confirmation emails keep going out. I lose sequences and the digest until it’s back, and I restore the SQLite copy from a backup. That’s the property worth designing for: the cheap, fragile box at home must never be what loses a lead.