Security and Rate Limits

This document outlines the application's current API rate limits, the current security middleware that is active in the Flask app.

1. Current Rate Limits

The backend API is shielded by Flask-Limiter to prevent abuse and ensure fair usage across our infrastructure. Limits are enforced based on the client's IP address (get_remote_address).

Storage Backend

Flask-Limiter is configured with RATELIMIT_STORAGE_URI, defaulting to memory:// when no shared backend is provided. In local development this is fine. In a multi-process deployment, a shared backend such as Redis is preferable so all workers see the same counters.

Pipeline scheduling status, run locking, and async export progress no longer reuse RATELIMIT_STORAGE_URI. Those concerns now use a separate runtime-state backend configured with STATE_BACKEND, STATE_REDIS_URL, and STATE_DIR.

Global Defaults

Specific Endpoints

Certain expensive or easily abusable endpoints have stricter throttles:

Endpoint Limit Location
/api/search 60 / minute backend/blueprints/search.py
/api/top_matches 60 / minute backend/blueprints/search.py
/api/stop_by_id 60 / minute backend/blueprints/search.py
/api/random_stop 60 / minute backend/blueprints/search.py

2. Shared Rate-Limit Storage

The codebase is structured so a shared storage backend can be plugged in without changing endpoint decorators. The default memory:// backend remains process-local, so deployments that use multiple workers should point RATELIMIT_STORAGE_URI at Redis or another supported shared store.

This is intentionally separate from the scheduler/app coordination store. A deployment can use Redis for rate limits and either Redis or the file backend for pipeline state.

Why In-Memory Storage Is Only a Development Default

With in-memory storage, each worker process maintains a separate bucket of request counts.

flowchart TD Client((Client IP\nLimit: 60/min)) subgraph Gunicorn W1[Worker 1\nMemory Count: 25] W2[Worker 2\nMemory Count: 10] W3[Worker 3\nMemory Count: 45] end Client -->|Requests| W1 Client -->|Requests| W2 Client -->|Requests| W3

In the scenario above, the client has actually made 80 requests (25 + 10 + 45), violating the 60/min limit. However, because no single worker has hit 60 internally, none of the workers will block the client.

  1. When a request arrives, the Gunicorn worker handling it connects to Redis.
  2. The worker increments a counter specific to the [Endpoint] + [Client IP] key.
  3. If the returned counter exceeds the allowed limit, the worker immediately rejects the request with a HTTP 429 Too Many Requests.
sequenceDiagram participant Client participant Worker as Gunicorn Worker participant Redis Client->>Worker: HTTP Request (e.g., /api/search) Note over Worker,Redis: Worker checks central store Worker->>Redis: INCR count for "search_api + IP" Redis-->>Worker: Returns new count (e.g., 61) alt Count Exceeds Limit (e.g., > 60) Worker-->>Client: HTTP 429 Too Many Requests else Count Within Limit Worker->>Worker: Process normal request logic Worker-->>Client: HTTP 200 OK (Response Data) end
flowchart TD Client((Client IP)) subgraph Gunicorn W1[Worker 1] W2[Worker 2] W3[Worker 3] end Redis[(Redis\nGlobal Counts)] Client --> W1 Client --> W2 Client --> W3 W1 <-->|Read / Increment| Redis W2 <-->|Read / Increment| Redis W3 <-->|Read / Increment| Redis

Because Redis handles atomic increments and expirations efficiently, it provides a unified source of truth.

3. HTTPS Enforcement and Browser Security Headers

This application is primarily a public-facing informational dashboard with no user accounts, authentication, or sensitive personal data. Because of this, we do not enforce strict browser security headers (such as a Content Security Policy via flask-talisman).

However, we do enforce HTTPS. Instead of relying on external middleware, the application uses a lightweight custom before_request hook in backend/app.py. When the FORCE_HTTPS=true environment variable is set, this hook checks the X-Forwarded-Proto header (or the request's native security state) and issues a 301 redirect to https:// if the request came in over HTTP. This ensures traffic remains encrypted while keeping the dependency tree small.

4. Security Risks and Attack Vectors

The search endpoint in backend/blueprints/search.py applies explicit input and query guardrails before hitting expensive text scans:

# snippet from [search.py](https://github.com/openTdataCH/stop_sync_osm_atlas/blob/main/backend/blueprints/search.py)
query_str = _normalize_search_query(request.args.get('q', ''))
if len(query_str) < 3:
    return jsonify({"osm": [], "atlas": []})
if len(query_str) > 50:
    return jsonify({"error": "..."}), 400

escaped_query = _escape_like_literal(query_str)
search_pattern = f"%{escaped_query}%"

# PostgreSQL-only guardrail
db.session.execute(text("SET LOCAL statement_timeout = 1500"))

# each query is capped
matched_query.limit(200)
unmatched_query.limit(200)

What is enforced:

  1. Query length bounds: short probes (< 3) are ignored and long payloads (> 50) are rejected with HTTP 400.
  2. Wildcard escaping: user input is escaped before ILIKE, so % and _ are treated as literals instead of attacker-controlled wildcard operators.
  3. Query timeout circuit breaker: PostgreSQL search statements use a local statement_timeout to prevent long-running scans from monopolizing workers.
  4. Result-size caps: both matched and unmatched search branches are hard-limited (200 each), reducing memory and serialization pressure.
Previous
7.1 Dependency Management & Build Strategy
Next
7.3 Background Scheduler
Data update in progress
Elapsed: -- ETA: -- Phase: idle