A3.1 Dependencies and builds

The engine and the review application are installed separately and never import each other. The engine's pyproject.toml declares NumPy and SciPy plus optional extras for its input adapters. Flask, SQLAlchemy, the PostGIS driver, report rendering and scheduling belong to the review application's requirements.txt. This page lists the images and the requirement files, and explains how releases of both projects are identified and pinned.

Images

The application's dockerfile builds three stages; the engine repository has its own Dockerfile.

Build Contents
Sibling engine Dockerfile Engine package, selected adapters and CLI. No backend code. Build from the engine repository root.
docs-diagram-stage Build-only Node environment with local Chromium. Uses the pinned Mermaid package plus puppeteer-core to pre-render content-addressed SVGs; only those SVGs cross into the app image.
app-stage Review app with requirements.txt, templates, static assets, migrations, review configurations and docs, including the engine's Markdown pages. No engine source or package. The migrator service runs it too.
scheduler-stage Deployment composition: app importer/jobs plus a separately installed engine executable. Communicates through files, with no cross-project Python imports.

A development environment can hold both projects:

python -m pip install -e "../engine[test]"
python -m pip install -r requirements.txt -r requirements-dev.txt

The application tests do not need the engine; only the producer-to-consumer contract test runs both. The tests run in the quality image, not in an application image (A4.4).

Application release identity and Git tag

The review application has its own release identity, independent of the engine:

Identifier Example Meaning
Application version 0.6.0 Canonical version declared in backend/version.py, shown in the navbar and beside the review application's collection in the documentation sidebar.
Application Git tag v0.6.0 Annotated tag attached to the exact application release commit. It exists in the application repository and must never be moved or reused.
Application commit SHA 0123abcd... Immutable identity of the application source. Deployments should record the SHA or an image digest derived from it.

The application and engine are separate repositories, so each may have a tag such as v0.6.0 without conflict. A tag belongs only to the repository in which it was created. An application release tag says which importer, database migrations, API and UI were deployed; it does not select an engine release.

Application release checklist:

  1. Update APP_VERSION in backend/version.py and add the release to documentation/Changelog.md.
  2. Run the application, JavaScript and producer-to-consumer contract tests.
  3. Commit the complete release, including migrations and documentation.
  4. Create an annotated v<application-version> tag on that commit and push it. Never tag an earlier commit while release changes are uncommitted, and never force-move a published tag.
  5. Deploy that exact commit or an image built from it, and record the commit SHA or immutable image digest.
git tag -a v0.6.0 -m "Review application 0.6.0"
git push origin v0.6.0
git rev-list -n 1 v0.6.0

Engine release identity and deployment pin

Do not use the words “version” and “tag” interchangeably. The integration uses four separate identifiers:

Identifier Example Meaning
Engine package version 0.1.0 Release version declared by the engine and written to bundle manifests as engine_version.
Git tag v0.1.0 Human-readable Git/GitHub name attached to the release commit. Published release tags must never be moved or reused.
Commit SHA 0123abcd... Immutable identity of the exact engine source used for a build. This is what engine.ref pins.
Result schema version 1 Version of the file contract consumed by the app. It changes independently of the engine package version.

The committed file engine.ref pins the engine: one line with the full commit SHA. CI checks out that commit from the public engine repository. Docker builds install the local checkout in ENGINE_DIR (default ../engine), so before every build, run:

./scripts/verify_engine_checkout.sh

The verifier rejects a checkout that is not exactly at the pinned commit or has uncommitted files. Docker receives the verified directory as a named build context.

Upgrade procedure

  1. Fetch the engine repository and inspect the intended annotated release tag.
  2. Resolve it with git -C ../engine rev-list -n 1 v0.1.0.
  3. Check out that commit and run the engine suite plus the producer-to-consumer contract test.
  4. Put the full resolved SHA—not a branch name or tag—in engine.ref and commit it. CI then tests the app against that engine.
  5. Run ./scripts/verify_engine_checkout.sh.
  6. Rebuild the app/documentation and scheduler images, then verify that the engine version in the documentation sidebar and the installed osm-public-transport-matcher package report the expected package version.

The package version may change without a schema change when algorithms or documentation change. A breaking result-format change requires a schema-version change and coordinated consumer support.

Python requirements

File Contents Installed in
requirements.txt Everything the review application runs: the web app, the importer, the PDF rendering and the scheduler The app-stage image, and therefore the migrator and the scheduler
requirements-dev.txt The test tools and the quality analysers. Analysers and the property/schema test tools have exact versions, so a gate gives the same answer everywhere The quality image (Dockerfile.quality) and native development environments, together with requirements.txt

Neither file lists the engine's dependencies: the scheduler image installs the engine package with the extras of its own pyproject.toml. The quality runner stops before any check when an installed package does not satisfy either file, so a stale image cannot pass. A4.4 describes the CI jobs.

Frontend and documentation-build JavaScript dependencies are pinned in package.json and package-lock.json. The Mermaid package is shared by the browser documentation view and the local PDF pre-renderer so both paths interpret the same diagram syntax. Chromium exists only in the discarded docs-diagram-stage; production PDF generation consumes the resulting SVG cache and has no Node, browser or outbound-network requirement.

The Swiss/acquisition engine extras include duckdb>=1.4,<2 for native GTFS processing. The generic GTFS extra only adds pandas. DuckDB is absent from the independent web image; PostgreSQL COPY uses the app's psycopg dependency. After changing engine or scheduler dependencies, rebuild and recreate the scheduler service so its installed osm-public-transport-matcher executable matches the checkout. See E5.2 Pipeline performance for memory and thread settings and benchmark commands.

Code

File Role
dockerfile The docs-diagram-stage, app-stage and scheduler-stage images
docker-compose.yml The services, built from the dockerfile stages with the engine checkout (ENGINE_DIR) as a named build context
requirements.txt, requirements-dev.txt The Python dependencies of the review application, and its test and quality tools
package.json, package-lock.json The pinned frontend and documentation-build JavaScript dependencies
backend/version.py APP_VERSION, the engine source links built from engine.ref, and engine_dir(), which finds the engine checkout
engine.ref The full commit SHA of the pinned engine
scripts/verify_engine_checkout.sh Rejects an engine checkout that is not exactly at the pinned commit or has uncommitted files

Related pages

Data update in progress
Elapsed: -- ETA: -- Phase: idle