Matching Process

The matching pipeline correlates ATLAS boarding platforms with OSM public transport nodes. It is ATLAS-driven and sequential: predicates run in a fixed priority order, and every successful commit() immediately locks the matched ATLAS and OSM representatives so later logic only sees what is still free.

This page is organized in the same order the code runs:

1. Load raw data into state managers
2. Pre-group duplicates and obvious OSM pair/trio groups
3. Run the 10 predicate executions in order
4. Record matches through ctx.commit()
5. Return PipelineResult / MatchingOutput

End-to-End Flow

Conceptual Abstractions

The runtime flow above shows execution order. The diagram below shows the conceptual layers the code is built around: predicates decide, the context orchestrates, the state layer answers queries and tracks locks, and domain models flow upward as the shared data abstraction.

What Counts as a Match

A match is represented by a MatchRecord, which links one AtlasNode to one OsmNode plus metadata about how the link was made.

Most links are 1 ATLAS → 1 OSM, but the pipeline intentionally allows these exceptions:

For osm_trio, the pipeline still creates only the two primary side-node MatchRecords. The middle stop_position is not a MatchRecord; it becomes an effectively_matched database row later during import when both trio sides were matched.

Phase 1: Build State Before Matching

The matcher does not work directly on raw DataFrame rows or raw XML tags. It first builds two state managers.

AtlasState

AtlasState.from_dataframe() does three important things before the first predicate runs:

1. Converts ATLAS rows into immutable AtlasNode models on demand.
2. Detects duplicate groups where entries share the same number + designation.
3. Loads data/processed/atlas_routes_gtfs.csv into get_routes(sloid) so route matching can stay file-I/O free.

ATLAS duplicate grouping

ATLAS duplicates are grouped before matching, with no distance heuristic.

1. duplicate_sloid_map groups rows sharing the same (number, designation).
2. The first sorted SLOID becomes the representative.
3. The remaining SLOIDs become hidden siblings.
4. get_unmatched_records() returns one AtlasEntity wrapper for the representative instead of exposing all siblings individually.

When that representative is committed, siblings get their own MatchRecords with match_type='duplicate_propagation'.

OsmState

OsmState.from_xml_file() parses OSM nodes and route relations into several indexes over the same underlying node set.

OsmNode.is_station returns True for public_transport=station or railway=station, but explicitly not for public_transport=stop_position and not for aerialway=station.

OSM grouping before predicates

OsmState.build_groups() runs before matching so obvious OSM siblings behave like one logical entity during predicate execution.

Grouping now runs in two layers, in this order:

1. Trios first (osm_trio) for strict 3-node UIC cases.
2. Pairs second for all remaining reciprocal sibling pairs, with a perfect-count branch first.

The map uses one consistent visual convention for grouped OSM structures:

• Solid blue lines are ATLAS-OSM matches.
• Dashed green lines are OSM-OSM structural links inside a pair or trio.
• Trio middles are shown on the map once imported as effectively_matched, but they are still not pipeline MatchRecords.

Visual examples

The screenshots below are represented as simplified SVG schematics so the pair/trio semantics remain readable in the documentation PDF as well.

OSM trio matched to 2 ATLAS stops

The trio's two side nodes are the only true pipeline matches. The middle stop_position remains unmatched during predicate execution, then becomes effectively_matched during import so it can be rendered on the map with dashed green OSM-OSM links to both sides.

OSM pair matched to 1 ATLAS stop

For a pair, the representative node matches first and ctx.commit() propagates that result to the sibling as osm_group_propagation. On the map this reads as two blue ATLAS-OSM links plus one dashed green OSM-OSM pair link.

Important details:

1. Grouping uses reciprocal nearest-neighbour checks, not just raw proximity.
2. Trios are only registered when both side nodes are within 15 m of the middle stop_position; otherwise those nodes stay available for the later pair grouping paths.
3. For perfect-count anchors (left_count == right_count == atlas_effective_count), grouping first tries reciprocal conflict-free pairing within 15 m and bypasses ratio checks.
4. atlas_effective_count is computed per UIC by counting only ATLAS rows whose nearest same-UIC OSM node is within 30 m.
5. The strict/relaxed fallback path still uses the original (unfiltered) atlas_count.
6. If the perfect-count branch does not apply or cannot produce a full 1:1 pairing, it tries a 1.5 ratio threshold and accepts those pairs only when grouped + ungrouped OSM entities exactly match the ATLAS count for that anchor.
7. If the strict complete-count check fails, it retries with 2.0 and accepts reciprocal pairs it finds as an incomplete group set.
8. In ratio-based paths, the ratio test compares the nearest candidate (which must be within 12 m) against the true second-nearest candidate, even if that second candidate is beyond 12 m. If no second candidate exists at all, the ratio test does not reject the pair.
9. The same perfect-count-first then strict-vs-incomplete policy applies to UIC, name, and tram pairing paths.

After grouping, lookup methods such as get_by_uic(), get_by_name(), and batch_query_radius() return OsmEntity wrappers for representatives while hiding siblings.

Phase 2: Run the Predicate Pipeline

The runner executes predicates in this order:

Each predicate sees only the current unmatched view of the state. Once a representative is committed, later predicates skip it automatically.

Predicate contract

All predicates implement the same interface:

class BasePredicate(ABC):
    @abstractmethod
    def run(self, ctx: MatchingContext) -> None:
        ...

They do not return match records. They query state and call ctx.commit() when they decide to match.

Phase 3: Record Matches Through ctx.commit()

MatchingContext.commit() is the only mutation gateway in the pipeline.

For osm_trio groups, ctx.commit() intentionally skips OSM sibling propagation so the trio middle node does not become an osm_group_propagation match. During the database import phase, if both of its side nodes are successfully matched, the importer promotes the middle node directly to effectively_matched for fast querying and map rendering.

That immediate locking is why the pipeline is safe even inside a single predicate loop: later rows consult the updated state, not just the original snapshot.

This matters especially for the predicates that batch KDTree lookups up front:

1. LocalRefDistancePredicate
2. NearestDistancePredicate
3. RouteMatchPredicate

They all re-check used_ids against the live state before committing so they do not reuse a stale candidate computed earlier in the same run.

Phase 4: Global Matching Rules

Station filtering

Spatial constants

Distance storage

All predicates store the actual haversine distance they used, including exact UIC matches.

Outputs

run_pipeline() returns a PipelineResult, and run_matching() wraps it into MatchingOutput.

unmatched_atlas contains all unmatched nodes, including hidden duplicate siblings of unmatched representatives. unmatched_osm contains all unmatched OSM nodes, including group siblings and stations if they were never used.

No-nearby-OSM detection

After matching, the importer separately flags unmatched ATLAS rows that have no OSM node at all within 50 m. That does not create additional matches; it feeds problem detection.

Domain Models

The pipeline uses three core domain models plus two entity wrappers.

Key fields on AtlasNode:

Key fields on OsmNode:

Performance Notes

Spatial matching uses a lazy scipy.spatial.cKDTree over 3D unit-sphere coordinates. The tree is cached and only rebuilt when the station-inclusion mode changes; matched nodes are filtered at query time rather than by rebuilding the tree after every commit.

The batched KDTree path is used by the local-ref, nearest-distance, and route predicates. GroupProximityPredicate does not use the KDTree; it builds a full pairwise distance matrix with NumPy broadcasting over each grouped candidate set.

Results Summary

Code References