CLI reference

← Maintainer handbook · Local setup · Configuration · Troubleshooting · Security policy

Use opportunities for schema upgrades, search inspection, authorized collection, reviewed offline additions, and public projection validation. Commands below run from the repository root; check side effects before operating on canonical state.

Show the command overview:

uv run opportunities --help

General form:

uv run opportunities [GLOBAL OPTIONS] COMMAND [COMMAND OPTIONS]

Select an optional settings file:

uv run opportunities --settings configs/settings.local.yml stats

The global --settings option must appear before the command name. In this reference, public exports means all three generated files: open-opportunities.csv, open-opportunities.json, and dataset-metadata.json.

Contents

• Command overview
• db-upgrade
• searches
• search-test
• scrape
• Quality report
• add-job
• add-jobs
• check-availability
• render
• export-public
• stats
• validate
• Exit codes
• Command side effects
• Common command sequences

Command overview

Command	Purpose
db-upgrade	Create or upgrade the configured database schema
searches	Inspect the effective search registry and available run health
search-test	Run one authorized search without persistence
scrape	Run authorized collection, persist independent outcomes, and optionally check collection quality
add-job	Add a known LinkedIn listing to canonical state without search provenance
add-jobs	Validate and add a bounded batch of reviewed listings in one transaction
check-availability	Audit a bounded batch of due listings and delete explicitly unavailable rows
render	Regenerate owned README, search-registry documentation, and public data projections
export-public	Regenerate sanitized CSV/JSON downloads and their metadata manifest
stats	Display aggregate canonical state
validate	Check schema, lifecycle invariants, and generated projections

Configuration sources and precedence are documented in Configuration.

db-upgrade

uv run opportunities db-upgrade

Creates or upgrades the configured database to the single Alembic head.

It:

• loads Alembic configuration and revisions;
• creates missing tables;
• applies pending migrations;
• performs no LinkedIn network access;
• does not render the README.

Run it before commands that require canonical database state.

Migration design, backup, and recovery belong to the database lifecycle guide.

searches

uv run opportunities searches

Displays the effective search registry, including:

• configured search slugs and whether each is enabled;
• location scope;
• effective page, result, and recheck limits;
• latest run status;
• found and accepted counts when migrated state is available.

It performs no network access and does not modify SQLite.

Use it after changing search YAML or global search-limit overrides.

Search schema and tuning are documented in the search registry guide.

search-test

uv run opportunities search-test <slug>

Runs one enabled search without persistence.

It:

• requires the LinkedIn authorization interlock;
• performs bounded LinkedIn guest-page requests;
• parses, normalizes, and classifies candidates;
• prints the result and diagnostics;
• does not write SQLite;
• does not update the README.

Use it for one authorized parser, query, or classification preview before persisted collection.

Important

The interlock is a safety gate, not permission. Source authorization requirements are defined in SECURITY.md.

scrape

Run every enabled search:

uv run opportunities scrape

Run one selected search:

uv run opportunities scrape --search company-amazon

Persist state without refreshing generated projections:

uv run opportunities scrape --no-render

The canonical collection workflows also request a sanitized drift report:

uv run opportunities scrape --quality-report quality-reports/data-quality-report.json

The command:

1. verifies authorization and migration preconditions;
2. loads and synchronizes the complete search registry;
3. selects every enabled search or one requested slug, ordered by persistent oldest completion with stable registry ties;
4. fetches, parses, normalizes, and classifies candidates;
5. commits each search outcome independently;
6. updates provenance and explicit lifecycle evidence;
7. when --quality-report is supplied, compares aggregate counts, field completeness, category/country mix, and scraper warnings with prior aggregate observations, labels full versus partial registry scope, writes a JSON report, and stores a bounded aggregate baseline only after every enabled search succeeds and no blocking finding occurs;
8. renders the owned README, search-registry documentation, and all public exports when collection is eligible for publication, unless --no-render is set.

A partial run preserves successful search transactions.

HTTP 429 stops subsequent requests, retains only fully completed searches, and returns partial exit code 4 if at least one search completed and the requested quality gate passes. This source-stop partial path requires --quality-report; without it the command returns 1 and withholds projections. A completed search with zero accepted jobs can qualify, but still faces all quality and publication checks.

Partial publication requires every failed search to be an HTTP 429 denial or a search skipped after the source stop. Any other search failure blocks publication, even alongside HTTP 429. If no search completes or the quality gate blocks or cannot complete, publication also stops. In each case, the command returns 1 and leaves projections unchanged. Protected automation also withholds snapshot publication and README handoffs. Failed and skipped searches never provide closure evidence. Partial proposals require manual review and merge; they are not complete collection success.

Quality warnings are reported but do not block the scrape. Blocking drift (such as every enabled search unexpectedly returning zero candidates or an extreme acceptance-rate change) returns exit code 1; the report is still written before the command exits, and projections are not refreshed. The gate runs after independent search transactions commit, so blocking drift withholds projection rendering but does not roll back those canonical transactions.

The report contains aggregate metrics only, not listing content. CI retains it as a separate 30-day artifact.

A failed search:

• records bounded diagnostics;
• does not apply absence evidence;
• does not increment unavailability confirmations;
• does not close jobs.

Use --no-render where canonical state should change without refreshing any generated projection, including ignored public exports, such as a website-only VPS.

The collection lifecycle is documented in Architecture and Database lifecycle.

Quality report

--quality-report <path> opts into the quality gate, including with --no-render. It is mandatory for HTTP 429 partial publication. The versioned JSON report identifies collection_outcome (full, partial, or blocked), publication_eligible after collection-quality checks, source_stopped, and failed/skipped search metrics with sanitized codes and HTTP statuses. Integrity, projection, and snapshot gates still run afterward in automation. The report also contains status (passed, warning, or failed), warning/blocking counts, stable finding codes, aggregate metrics, run scope, and baseline readiness. It never includes job IDs, listing text, raw warnings, source responses, or database paths. Failed-search metrics are unavailable, not fabricated zeros; collection totals cover successful searches only.

Comparisons require at least three usable observations from the latest five stored full-registry snapshots, all within the preceding 30 days. New or changed collection settings warm up a new baseline; search names, notes, and verification dates do not reset it. baseline.status: warming_up means historical comparisons are not ready, even if the report has no findings. Invalid or future-dated snapshots generate a warning and are ignored; expired snapshots are ignored. Only successful full-registry runs without blocking findings advance history. Warning-only runs can advance it, so review warnings rather than treating them as permanent protection against sustained drift.

Check	Warning	Blocking
Candidate volume	≥80% drop against the matching registry median, with baseline ≥10 candidates	Every enabled search returns zero against that baseline
One search becomes empty	Zero candidates after a positive matching median	No
Acceptance rate	Absolute change ≥40 percentage points	Absolute change ≥75 percentage points
Scraper warnings	At least 3 warnings and ≥25% of the larger of candidate, parsed-detail, and warning counts	No; parser/transport failures retain their existing search-failure behavior
Field missingness	Increase ≥40 percentage points, or newly reaching ≥99% missing	No
Category disappearance	No open rows for a category with historical median ≥3	No
Country distribution	Total-variation distance ≥0.5, or fewer than 10 country observations remain against a sampled baseline	No
Failed searches	HTTP 429 failures and searches skipped after that stop	No selected search succeeds, or any other search failure occurs (including alongside HTTP 429)

Acceptance uses parsed details, including bounded known-job rechecks, rather than search-card counts. Both the current sample and at least three matching historical samples must contain 10 classified records. Field comparisons require 10 open jobs for optional stored fields or 5 parsed details for parser fields in both the current and at least three historical samples. Country comparisons require at least 10 country observations per historical sample; multi-country listings count once per recognized country. Parser fields are description, industries, locations, posting time, and start date; stored optional fields are industries and start date. Missing optional metadata alone is not a cold-start error.

Per-search comparisons require matching effective settings. Registry volume and parser comparisons also require every enabled search to succeed. Open-dataset profiles always cover all open rows, including on a selected-search run, but require a matching registry baseline. Counts summed across searches are observations, not deduplicated jobs. Profile changes may reflect legitimate availability updates or manually reviewed additions and are warnings only.

A baseline-read, analysis, report-write, or baseline-persistence error fails closed with exit code 1. The gate does not undo committed search transactions or prevent a separate explicit render; do not publish a blocked local collection. Inspect the retained report and compare its scope, sample sizes, and configuration before changing code. Reproduce suspected parsing changes with sanitized offline fixtures, then run the normal validation gates. Do not delete canonical state or disable source authorization to resolve an alert.

add-job

Maintainers can add a known LinkedIn listing without running collection. Independently verify the public source facts first: this offline command validates supplied evidence but cannot authenticate it. The example below is synthetic:

uv run opportunities add-job \
  --url https://www.linkedin.com/jobs/view/1234567890 \
  --company "Example Technology" \
  --title "Software Engineering Intern 2027" \
  --location "London, UK" \
  --category software-engineering \
  --employment-type internship

Required options are --url, --company, --title, --location, --category, and --employment-type. Optional metadata can be supplied with --industries, --start-date, and --posted-at; --posted-at requires an ISO-8601 timestamp with an explicit timezone, must not be in the future, and is normalized to UTC. A yearless title without eligible posting evidence is rejected. Use --no-render to update only SQLite.

The command:

• extracts the numeric identity from the canonical LinkedIn /jobs/view/<id> URL;
• validates and normalizes the row through DiscoveredJob, then applies the same deterministic classifier as collection (title, category, employment type, cycle/posting date, and European location) before writing;
• inserts or updates an open row through the repository; rejects a closed row, which must be reopened by valid discovery or a successful availability audit;
• creates no search, search run, or job_searches provenance;
• performs no network access and does not require the LinkedIn authorization interlock;
• refreshes the README, search-registry documentation, and all public exports by default.

A manual insertion does not change the README's last successful collection timestamp. That timestamp remains derived from successful collection runs. The full availability audit includes manual rows, and a later ordinary scrape can attach real search provenance to the existing job.

add-jobs

For several independently reviewed listings, place 1–10 jobs in a UTF-8 JSON array and run one batch:

[
  {
    "url": "https://www.linkedin.com/jobs/view/1234567890",
    "company": "Example Technology",
    "title": "Software Engineering Intern 2027",
    "location": "London, UK",
    "category": "software-engineering",
    "employment_type": "internship"
  },
  {
    "url": "https://www.linkedin.com/jobs/view/2345678901",
    "company": "Example Research",
    "title": "Machine Learning New Grad 2027",
    "location": "Paris, France",
    "category": "machine-learning",
    "employment_type": "new-grad",
    "posted_at": "2026-09-24T10:00:00+02:00"
  }
]

uv run opportunities add-jobs --input manual-jobs.json

Each object uses the same required and optional fields as add-job; JSON field names use underscores for employment_type, start_date, and posted_at. The input must be no larger than 64 KiB. Unknown fields, duplicate LinkedIn IDs, invalid or closed jobs, and any failed classification reject the whole batch before a canonical transaction commits. The repository applies all accepted rows in one transaction. By default the command then refreshes the normal projections; --no-render changes only SQLite. It makes no LinkedIn request and creates no synthetic search provenance.

check-availability

uv run opportunities check-availability

Check canonical state without refreshing generated projections:

uv run opportunities check-availability --no-render

The command requires the LinkedIn authorization interlock and checks at most 50 due job rows by default, including closed rows. Each row becomes due five days after its last non-denial attempt; never-checked rows are eligible immediately. The availability settings control the interval and cap. Production automation recommends OPPORTUNITIES_AVAILABILITY_MAX_JOBS=100 for each scheduled run; the CLI default remains 50. The summary reports checked, available, deleted, reopened, inconclusive, blocked, and deferred rows; blocked and unprocessed rows never advance their audit timestamps. It reports why collection was skipped after a source denial. Due rows deferred by the cap remain eligible; those rows stay unchanged and do not cause a partial-success exit.

The audit requests the public listing first; a successful public page without a closure alert is then followed by guest detail validation. It then applies one transaction:

• successful public-page and detail-page validation keeps the row open or reopens it;
• HTTP 404 or 410 from either request permanently deletes the job and cascading search provenance;
• a scoped public-page “No longer accepting applications” alert also permanently deletes the job;
• authentication failures, rate limits, server errors, malformed responses, and transport failures preserve the row as inconclusive.

An HTTP 301 from an approved numeric public listing is inconclusive: the destination is ignored, the listing's lifecycle is unchanged, and unrelated checks continue. These redirects do not block otherwise eligible HTTP 429 partial publication, but do not count as confirmed checks. Completed inconclusive attempts advance only audit scheduling. Denied and unprocessed listings keep their priority.

Exit	Audit result	Default projection handling
0	All selected checks confirmed, or no jobs due	Refresh
2	Inconclusive checks without a source denial	Refresh confirmed results
4	HTTP 429-only stop with at least one confirmed check; approved public-listing 301 responses are the only permitted inconclusive checks	Refresh completed results
1	Other source denial, mixed failures alongside HTTP 429, or a rate limit before any confirmed check	Preserve existing files

--no-render preserves projections in every case. Confirmed database changes remain committed even when publication is blocked. Other errors, such as missing migrations or failed rendering, retain their normal exit codes.

No source denial permits further requests or retries. Nightly scraping is skipped after an audit rate limit, even when the partial results qualify for publication. Eligible partials still need integrity, projection, and verified-snapshot checks and manual review. Audit-only runs create no collection report or quality baseline. Review authorization before another run; see the partial-update policy.

For scheduled and manual workflow sequencing, see Automation.

render

uv run opportunities render

Regenerates every owned projection from canonical SQLite state and the configured search registry.

The README projection includes:

• total open-job count;
• latest successful collection time;
• the public website link;
• a hidden SHA-256 review seal covering every website-visible open row and the exact latest successful collection timestamp;
• at most five internships and five New Grad opportunities ordered by immutable first-seen time, then descending ID text. First-seen time can use approximate source posting age and does not necessarily describe discovery order.

The generated registry-layout counts in search-registry.md are refreshed from configs/searches/ at the same time. The CSV and JSON downloads are generated from all open rows using the fixed public-field allowlist. Their metadata manifest records the schema version, UTC generation time, counts, and SHA-256 hashes.

The command:

• performs no network access;
• does not modify SQLite;
• preflights and writes the owned regions in English and all five supported translations through atomic per-file replacement when docs/translations/ exists beside the configured README.md; otherwise it updates only the configured file;
• refreshes the owned generated registry-layout block in search-registry.md;
• atomically replaces each of the three public export files.

Important

A fresh local database contains no listings. Do not render and commit the preview from empty development state.

Do not edit generated counts, timestamps, opportunity rows, registry-layout counts, or public exports manually.

export-public

uv run opportunities export-public

Generates open-opportunities.csv, open-opportunities.json, and dataset-metadata.json in the configured public-export directory. It reads open SQLite rows, selects only approved public fields, neutralizes spreadsheet formulas in CSV text, and atomically replaces each file. The manifest includes a fresh UTC generation time, schema version, counts, and hashes of the exact download bytes.

This command performs no network access, does not modify SQLite or documentation, and is used by deployment-only automation after restoring reviewed canonical state. Listing rows omit lifecycle timestamps, status, provenance, runs, closure evidence, and diagnostics. See the download contract for the separate row and manifest fields.

stats

uv run opportunities stats

Displays aggregate canonical state, including:

• total, open, and closed jobs;
• configured searches;
• successful and failed search runs;
• latest successful collection.

It performs no network access and does not modify state.

Use it after migration, collection, or restoration to check the configured pipeline database. In versioned production mode, stats does not inspect the website's selected data/current release; verify the served release and downloads using the deployment checks.

validate

uv run opportunities validate

Checks:

• required database tables;
• the Alembic revision;
• SQLite structural integrity and foreign-key consistency;
• monotonic lifecycle timestamps;
• exact README projection equality with canonical state, including the full-state review seal in English and each supported translation; missing translation files or invalid generated markers fail validation;
• generated search-registry layout counts against the configured YAML files;
• exact public CSV and JSON equality with the approved fields from open SQLite rows;
• v1 schema compliance for downloads and metadata, including manifest fields, UTC generation time, counts, and hashes.

Validation performs no network access and never repairs state automatically.

Run it only when the configured database contains the representative canonical state expected by the committed generated documentation.

For diagnosis, use Troubleshooting.

Exit codes

Code	Meaning
0	Command completed successfully
1	No search completed, a search failure other than HTTP 429 or source-skipped work occurred, a source denial blocked availability, validation found an inconsistency, or the quality gate blocked or could not complete
2	Availability audit with inconclusive checks, or rejected command/configuration input; never accepted as scrape success
3	Required database tables are missing or the schema is not at migration head
4	Eligible HTTP 429 partial collection or availability audit, not full success; required publication checks and manual review remain

After a partial scrape with exit code 4:

• successful search transactions remain committed;
• failed and source-skipped searches retain diagnostics and keep scheduling priority;
• failed searches do not mutate lifecycle state;
• quality, integrity, projection, and snapshot validation must pass before publication;
• manual review and merge are required before a partial dataset can be deployed.

An availability audit returning 4 likewise preserves only completed confirmed work and requires validation and manual review. Its denied and unprocessed rows keep their audit priority, and no follow-on scrape runs in that workflow execution.

GitHub Actions handling of these codes is documented in Automation.

Command side effects

Command	LinkedIn network	Writes SQLite	Writes projections
db-upgrade	No	Schema only	No
searches	No	No	No
search-test	Yes, after authorization gate	No	No
scrape	Yes, after authorization gate	Yes; optional bounded quality baseline	README + registry docs + public exports after eligible full/partial success and required quality checks; blocked source outcomes never render
scrape --no-render	Yes, after authorization gate	Yes; optional bounded quality baseline	No
add-job	No	Yes	README + registry docs + public exports
add-job --no-render	No	Yes	No
add-jobs	No	Yes, one transaction	README + registry docs + public exports
add-jobs --no-render	No	Yes, one transaction	No
check-availability	Yes, after authorization gate	Yes	README + registry docs + public exports after a nonblocked audit or eligible HTTP 429 partial audit
check-availability --no-render	Yes, after authorization gate	Yes	No
render	No	No	README + registry docs + public exports
export-public	No	No	Public exports only
stats	No	No	No
validate	No	No	No

Common command sequences

Initialize a local database

uv run opportunities db-upgrade
uv run opportunities searches
uv run opportunities stats

Use a new, disposable development path. A fresh database intentionally contains no listings; production recovery must use verified existing state.

Inspect registry and canonical state

uv run opportunities searches
uv run opportunities stats

Test one authorized search without persistence

uv run opportunities search-test <slug>

Persist one authorized search

uv run opportunities scrape --search <slug>
uv run opportunities validate

Validation assumes that the configured SQLite database and generated README represent the same canonical state.

Collect without refreshing projections

uv run opportunities scrape --no-render
uv run opportunities stats

Regenerate every projection from representative state

uv run opportunities render
uv run opportunities validate

Regenerate only public downloads

uv run opportunities export-public

Verify migrations during development

uv run opportunities db-upgrade
uv run python scripts/database/check_migrations.py

Valid slugs and query configuration are documented in the search registry guide. Complete engineering checks are documented in the testing guide.