Stream overview
The core Firehose product — your rules match every crawled page and matches are delivered over a live SSE stream. When to use it, and how a match is made.
Stream is the heart of Firehose. You attach rules (queries) to a tap, and every freshly crawled page that matches a rule is delivered to you the moment it's crawled — live over a Server-Sent Events connection, or in the dashboard feed if you'd rather not write code.
When to use it
Use Stream when you want to monitor the whole web for something, rather than a fixed list of pages. For example:
- Every new page mentioning your brand or a competitor.
- News in a category and language, published in the last hour.
- New pages matching a URL or content pattern across many sites.
Stream monitors the whole web — it doesn't poll a page for you. To watch specific URLs you already know, use URL Watch. See Stream vs URL Watch for the full comparison.
How a match is produced
Rule: title:tesla AND page_category:"/News" AND recent:24h
│
▼
Every crawled page is evaluated against all active rules on the tap
│
▼
Match ──▶ update event on GET /v1/streamA page is delivered if it matches any rule on the tap. Each delivered match tells you which
rule(s) matched (query_ids) and carries the page's metadata plus the content diff — the change
that matched, with a little surrounding context, not the whole page.
In this section
Rules & query syntax
Manage the queries on a tap, and the full query language they're written in.
Streaming
Open the SSE connection, event types, parameters, and reconnection.
Match payload
Every field on a delivered document.