7. System Architecture and Operations

This chapter describes the container layout, migration ownership, scheduler ownership, and shared runtime state used by the application.

Compose Services

docker-compose.yml defines these primary services:

• app: Flask web API/UI. Depends on successful migrator completion. Exposes port 5001.
• scheduler: APScheduler worker running preprocessing, matching, payload precompute, and import orchestration in the background. Depends on successful migrator completion.
• db: PostgreSQL + PostGIS container.
• migrator: Fast, one-shot migration service (flask db upgrade). Starts, applies migrations, and exits.
• test: Merged test image for full-suite pytest runs.

Runtime Ownership

The services have deliberately separate responsibilities.

The app container does not run the data pipeline. It only reads imported data and shared pipeline status.

Migration Ownership

Migrations are owned solely by the dedicated one-shot container:

• service: migrator
• command: python [matching_and_import_db/database/init.py](https://github.com/openTdataCH/stop_sync_osm_atlas/blob/main/matching_and_import_db/database/init.py) && flask db upgrade

By isolating migrations, the stack avoids race conditions where multiple containers try to change the schema at the same time.

app and scheduler start only after the migration container succeeds via depends_on: condition: service_completed_successfully.

entrypoint.sh performs a final application-level connection check to ensure PostgreSQL is accepting connections before the web service boots.

Scheduler and Shared State

Pipeline execution state is independent from rate limiting.

• rate limiting uses RATELIMIT_STORAGE_URI
• shared runtime state uses the file backend selected by Docker Compose

The shared runtime-state backend stores:

• run status
• current phase and human-readable message
• next scheduled run timestamp
• the distributed run lock
• async export progress and completed-file metadata

This is what lets the scheduler, app, and browser agree on whether a pipeline run is active and whether the maintenance overlay should be shown.

The app and scheduler both mount ./data at /app/data, so the default
STATE_DIR=data/runtime resolves to the same host directory. Pipeline status is
stored in pipeline_status.json; the active lock uses pipeline_lock.json;
async-export state uses tasks_progress.json and tasks_completed.json.
File writes are atomic, and lock operations use fcntl.flock().

Operational Notes

• Production ([docker-compose.yml](https://github.com/openTdataCH/stop_sync_osm_atlas/blob/main/docker-compose.yml)) uses pre-built images from GitHub Container Registry and avoids source-code bind mounts.
• Development (docker-compose.override.yml) restores image build: targets and mounts the local directory (.:/app) for live editing.

Subpages

• 7.1 Dependency Management & Build Strategy: Python dependency split, Dockerfile stages, and build matrix
• 7.2 Security and Rate Limits: Flask-Talisman, CSP, and rate limiting configuration
• 7.3 Background Scheduler: APScheduler service, job runner phases, lock handling, and refresh modes
• 7.4 Atlas-Cached Import Optimization: Table-reuse behavior for unchanged ATLAS/GTFS inputs
• 7.5 Frontend Asset Management: Vendored frontend libraries, Font Awesome optimization, and sync pipeline