How it's built

Last updated 2026-10-12

If you're going to load someone else's list into your firewall, you should know how it's made. This page is for the IT and security people who ask that question. They're right to.

The short version: WatchmanHQ is a small system built to run on its own every day, fail safe, explain every decision it makes, and cost next to nothing. It runs on free tiers and plain, well-understood tools, with the same habits we'd expect of any enterprise security team.

From research to block list

Every day, without anyone touching it:

1. Read. A collector reads the public RSS feeds of the research teams we follow, and treats every article as untrusted.
2. Extract. It keeps only indicators the authors defanged, with the sentence each came from, and works out the role each one played.
3. Corroborate. Repeats from one publisher count once. Reports from different teams back each other up.
4. Validate. Block-list candidates are checked against passive public records. We never visit, scan or resolve them ourselves.
5. Decide. Fixed, tested rules place each one on Confident, Blocking or Hunting, and record the reasons.
6. Gate. A release gate re-checks the finished lists and refuses to publish if anything is out of place.
7. Build. This site checks every file against its SHA-256 checksum, builds plain static pages and runs its quality gates.
8. Publish and watch. The site goes live, a smoke test checks the live copy, and a watchdog checks it again every three hours.

The rules behind steps 2 to 6 are written out in full in Methodology.

The principles behind it

• Fail safe. A bad release never replaces a good one. If a day's run is refused, you keep the previous lists and the Watch page says so. Every list expires 72 hours after it's built, so a stalled feed can't quietly go stale inside your firewall.
• Rules, not guesses. No AI model decides what goes on a block list. The same evidence always produces the same decision, and every decision traces back to a published report. How we use AI covers where AI does help.
• Short and right beats long and noisy. Confident is short on purpose. A list you can trust with a block rule is worth more than a big one you have to babysit.
• Remove first, investigate after. If an entry might be blocking something legitimate, it comes off first. See Report a false positive.
• Less to go wrong. The site is written in Python using only its standard library: no third-party packages, no JavaScript, no database. Less code means less to patch and less to attack.

The site is a target too

A security site that gets tampered with does real harm, so this one leaves an attacker very little to work with.

• Nothing to steal or hijack. No JavaScript, no forms, no cookies, no logins and no database. Every page is a plain file.
• A strict Content Security Policy (default-src 'none') blocks scripts, frames and third-party requests outright. It's sent as a header and repeated inside every page, alongside HSTS, nosniff, frame-ancestors 'none' and a locked-down Permissions Policy.
• Untrusted text stays text. Everything quoted from third-party reports is escaped. Indicators are shown defanged (evil[.]com) and never as links.
• Checksums end to end. Every published file has a SHA-256 checksum in its release manifest, and the site refuses to build if a single file doesn't match. You can check them yourself: see Pull it automatically.
• A locked-down build pipeline. Every GitHub Action is pinned to an exact commit, runs with read-only permissions, keeps no credentials after checkout, and never passes outside input straight into a shell command. Tests enforce all four, so it can't quietly drift.
• No cloud passwords. The hosting is built from code with Terraform. GitHub signs in to Azure with short-lived identity tokens that only the main branch can get, and that identity can manage the website and nothing else. The daily deploy holds a token that can publish files and do nothing more.
• Least privilege for data. The site reads the feed with a read-only token, or from a public repository with no token at all.
• A collector that distrusts its sources. It only connects to public internet addresses and re-checks every redirect, so a malicious link can't steer it into a private network. It refuses oversized downloads and XML feeds with risky features, and escapes article text so it can't turn into a spreadsheet formula.

Nothing goes live without passing

One failure anywhere and nothing is published. The live site keeps its last good release.

Keeping watch on ourselves

A watchman who falls asleep is worse than none, because people think they're covered. So the system checks on itself:

• After every deploy, a smoke test confirms the new release is live, every file matches its checksum, the security headers are in place and private routes are closed.
• Every three hours, a watchdog checks the site is up, the data is fresh, the TLS certificate isn't close to expiring and plain HTTP still redirects to HTTPS.
• If anything stops running at all, a dead man's switch notices the missing check-in and sends an alert. Silence counts as a failure.
• The money is watched too. Budgets send an email at set amounts, Azure flags unusual spending, and if spending ever ran away a circuit breaker would switch paid features off on its own. Anyone granting access in the cloud account triggers an alert.
• Every run is on the record. Each release carries a log of every run, including the ones our checks held back, and every false positive taken off the lists. See the Track record.

The stack

Who built it

WatchmanHQ is designed, built and run by Vincent Morris, drawing on more than twelve years across IT operations, cloud, cybersecurity and DevSecOps. It takes the habits of enterprise security (least privilege, defense in depth, fail-safe releases, monitoring that notices silence) and points them at the people who could never afford a security team.

Want to talk shop about the pipeline, the methodology or sharing data? Email hello@watchmanhq.com.