From an institution nobody had checked to a record you can act on

Nothing here is a crawler pointed at the internet. Every stage has an input it is allowed to trust, an output another stage consumes, and a measurement of its own. That is what makes a gap findable instead of invisible.

1. Two registries, never merged

An institution registry answers who publishes or funds the work. A source registry answers where and how that work is announced. They are kept separate because the relationship is many-to-many in both directions: one county may use a central procurement portal, a grants page, a public works bid board, and a third-party construction plan room, while one hosted platform may serve hundreds of unrelated buyers.

Every institution gets an internal identifier, and external identifiers are preserved with the system that issued them — an EIN, an IPEDS UnitID, a Census government identifier, an agency code. Names are never merged because they resemble one another: “Washington County” means nothing without a state.

2. A source card, not a bookmark

Discovery produces a completed source card: the official entry page, the actual collection endpoint, the host and platform tenant, the opportunity families that appear there, the access method and permitted use, the expected publication behaviour, a responsible maintainer, and a sample notice plus one attachment to test during onboarding.

Source state is tracked independently of opportunity state — discovered, verifying, active, degraded, blocked, retired, redirected. Without that separation, a website that went down silently becomes a false claim that nothing is available.

3. Collection, under a policy

Operators run against one verified location at a time and write to disk under a request policy. They do not write to the application database. Collected material living in a working directory is a real, useful state — and it is not the same state as being searchable.

4. A manifest, then an idempotent publish

A completed run becomes a manifest: a deterministic list of notices, each with a stable identity derived from its source and the issuer's own identifier rather than from the moment it was imported. Publishing the same manifest twice is a no-op, which is the property that lets a run be repeated safely after a partial failure — and the reason a link or a decision recorded against a notice still resolves after a republish.

# collect → stage a manifest → publish into an environment
agent-stage   data/sam-gov-api/2026-10-22-1622
agent-publish work/manifests/sam-2026-10-22-1622 --env dev
agent-publish work/manifests/sam-2026-10-22-1622 --env prod

5. Change detection, with the previous state kept

Reprocessing is where most of the value is. An amendment may move a deadline, lift a restriction, close a program, or replace an attachment without changing its filename. The system detects meaningful change, preserves what was there before, and can say why an alert was sent. A reader who passed on a notice yesterday may need it today.

6. Qualification, as a separate question

Three questions, deliberately not collapsed into one score:

• Is the work relevant? Retrieval helps here.
• Is the organization eligible and able to participate? Requirement checks and a missing-evidence review help here.
• Is pursuing it worthwhile? Capacity, strategy, competition, and expected effort decide this, and no retrieval system knows them.

Matching therefore produces a cited fit analysis and an explicit list of what is missing. Missing qualifications, pricing, certifications, or past performance stay missing until they are supplied and verified — generated language cannot establish a fact about your organization.

7. Measurement, per stage

Each denominator is reported separately, because each one is a different kind of failure: institutions reviewed in a defined directory; verified publication locations with a working collection path; records imported out of those a bounded source query returned; expected attachments acquired; and whether the last successful verification met the source's target interval.

Feedback is tied to the stage that failed. A geographic rejection improves eligibility handling. A missed notice from a broken source improves coverage. Separate reason codes let discovery, data quality, matching, and qualification each improve without masking another's errors.