CI (GitHub Actions) and Tests

This repository uses GitHub Actions for continuous integration (CI). The current workflow runs documentation link checks, JavaScript tests, Python linting, and Python tests on every push and pull request.

Overview

Workflow Location

.github/workflows/tests.yml

When CI Runs

The workflow triggers on:

• push to main, master, or develop
• pull_request targeting main, master, or develop

Test Structure

stop_sync_osm_atlas/
├── tests/                       # Test suites directory
│   ├── js/                      # JavaScript tests (Jest)
│   │   ├── setup.js             # Global mocks (Leaflet, app globals)
│   │   └── *.test.js            # Test files
│   ├── matching_pipeline/       # Matching pipeline tests (pytest)
│   │   ├── test_matching.py     # Predicate and unit tests
│   │   ├── test_small_pipeline.py # End-to-end integration
│   │   ├── test_data_integration.py # Route data logic
│   │   └── test_validators.py   # shared validation helpers
│   ├── test_docs_links.py       # Docs route/link checks
│   ├── test_filter_semantics.py # Filter algebra/semantics checks
│   ├── test_async_export_payloads.py # Async API endpoint tests
│   ├── test_global_stats_service.py # Global stats logic
│   ├── test_search_security.py   # Search API security checks
│   └── conftest.py              # Shared fixtures
└── .github/workflows/
    └── tests.yml                # CI workflow definition

Coverage

Both test suites generate coverage reports:

Coverage artifacts are available for download from the GitHub Actions run page.

Run Tests Locally

To keep local runs close to CI, use the same commands manually. The test service in docker-compose.yml is configured with all necessary dependencies for both the web app and the matching pipeline.

Run individual test suites:

# Matching pipeline tests
docker compose run --rm --no-deps --entrypoint '' \
    -e DATABASE_URI=sqlite:// \
    test python -m pytest tests/matching_pipeline/ -v

# Documentation link tests
docker compose run --rm --no-deps --entrypoint '' \
    -e DATABASE_URI=sqlite:// \
    test python -m pytest tests/test_docs_links.py -v

# Backend / API tests
docker compose run --rm --no-deps --entrypoint '' \
    -e DATABASE_URI=sqlite:// \
    test python -m pytest tests/test_async_export_payloads.py tests/test_filter_semantics.py -v

# JavaScript tests (host)
npm ci
npm test -- --coverage --ci

Optional JavaScript run in a throwaway container:

docker run --rm -v "$PWD":/work -w /work node:20 \
  sh -lc "npm ci && npm test -- --coverage --ci"

Run with Coverage

# JavaScript
npm run test:coverage

# Python (matches CI)
docker compose run --rm --no-deps --entrypoint '' \
    -e DATABASE_URI=sqlite:// \
    -e FLASK_ENV=testing \
    test python -m pytest tests/ --cov=matching_and_import_db --cov=backend --cov-report=html -v

Subpages

Notes on Dependency Management

This repo maintains two separate dependency ecosystems:

CI uses npm ci for JavaScript (reproducible installs from package-lock.json) and pip install for Python from combined requirement files.

For runtime containers, Python dependencies are split into requirements-base.txt, requirements-web.txt, and requirements-scheduler.txt to keep images lightweight. The test image combines all of them plus requirements-test.txt.