URL Watch overview
Track a specific list of URLs for changes on a schedule and capture the diff between crawls. When to use URL Watch over the Stream, and how it works.
URL Watch monitors a specific set of URLs you choose, re-crawling each on a schedule and recording what changed since the previous crawl. Where Stream watches the whole web for a pattern, URL Watch watches pages you already know about.
Not sure whether you want Stream or URL Watch? See Stream vs URL Watch.
When to use it
- Track a competitor's pricing or product page for edits.
- Watch a changelog, status page, or policy page for updates.
- Monitor a handful of landing pages you don't own.
How it works
You add a URL and pick a frequency. Firehose crawls it on that cadence, compares the new content against the last snapshot, and stores a diff of what was added and removed. The most recent crawls and their diffs are visible per subscription.
Watched URL + frequency ──▶ scheduled crawl ──▶ diff vs last snapshot ──▶ crawl historyURL Watch is managed from the dashboard or over HTTP with the URL Watch API. It is not billed per match the way the API stream is — usage is metered as checks against your plan's monthly quota (see Quotas & limits).
In this section
Creating watches
Add a URL, choose a frequency, preview, and bulk-import.
Diffs & crawl history
Read what changed between crawls.
Quotas & limits
Monthly checks, watched-URL caps, and minimum frequency by plan.
URL Watch API
Create and manage watches and read diffs with a management key.