Place-of-interest status modelling
An XGBoost model that scores whether places in Overture Maps are still operating, from weekly search, listing, engagement and visit signals, with training and validation reports built into the Airflow pipelines.
Employer: Sparkgeo (client project)
When: 2025–2026
Data: Weekly place signals from industry contributors (search interest, directory listings, digital engagement, physical visits), Overture Maps Places, third-party place status labels
Methods: Entity matching, signal time series, multitemporal features, XGBoost classification, feature importance, accuracy assessment, scheduled retraining with temporally matched labels
Stack: Python, PySpark, XGBoost, Apache Iceberg, Apache Airflow, Quarto, Plotly
Background
Places data goes stale: businesses close, move, rebrand or shut for the season, and a map can’t keep up through periodic manual updates. The Overture Maps Foundation, which releases roughly 70 million places globally each month, needed a confidence score for whether each place is still operating. Sparkgeo built the pipeline that produces it from signals contributed by industry partners.
Approach
- Signals: contributors supply weekly signals in four categories: search interest, directory listings, digital engagement and physical visits. None is conclusive on its own; together they give an operational picture of a place.
- Privacy: Sparkgeo acts as a processing intermediary. Raw contributor data is never shared with Overture or other parties; only anonymized derivatives leave the pipeline.
- Matching: incoming signals are matched to Overture Places through Overture’s Global Entity Reference System (GERS) IDs, which give each real-world place one identifier across contributors.
- History matters: a single quiet week says little. A lavender farm goes dark every winter and a museum closes for renovation, so the model looks at each place’s signal history, which lets it separate seasonal or temporary inactivity from permanent closure.
- Model: an XGBoost model trained on the signal history scores each place from 0 to 1 for the likelihood that it is operating.
- Pipeline: weekly ingestion, matching, training and inference run on Apache Spark over Apache Iceberg tables, orchestrated by Apache Airflow.
My contribution
- Model assessment: assessed accuracy and feature importance for the models built on each contributor’s signals. The results guided the final feature set for each model.
- Pipeline reports: built training and validation reports that run as steps in the main training and inference Airflow DAGs, so every run produces an assessment of the signals data, model performance and inference results. Metrics and statistics are computed with PySpark from the Iceberg tables, and the reports are rendered with Quarto, with interactive Plotly charts.
- Ongoing training design: the model was originally trained on a static dataset. I designed a strategy for training and validating it on an ongoing basis:
- When new contributor signals arrive, fresh ground truth labels are fetched automatically from a third-party place status source.
- A place’s status label is only valid for the day it is fetched, so signals are paired with the closest-dated label available.
- Training and validation never use signals dated after a place’s label. In a running pipeline, more signals can arrive for a place after its label was fetched, and using them would leak information the label can’t account for.
- The model retrains on a regular cadence, and a newly trained model replaces the current one only if it meets a performance threshold.
- Multitemporal feature engineering: supervised a junior team member deriving multitemporal features from the signals data. I met daily with the team member, evaluated their results, and set directions for their future work.
Outcome
- The model’s scores feed the Operating Status attribute in Overture Places, which flags whether the signals point to a place being open, closed or temporarily inactive.
- Signal volumes grew from fewer than 1 million monthly observations in May 2025 to more than 50 million by March 2026. As of the March 2026 release, 9.4 million of the roughly 15.5 million US places (61%) were covered by at least one signal source.
- Expansion into Western Europe was underway as of May 2026.
- Because the reports run inside the DAGs, data quality and model performance are checked on every run rather than in occasional one-off reviews.