Impervious surface mapping
Mapping impervious surfaces in Metro Vancouver from multi-temporal Sentinel-2 with XGBoost (macro-F1 above 0.85), with a conservative change layer for later years and a comparison against ESA and Esri global land cover.
Employer: Sparkgeo (internal project)
When: 2021
Data: Sentinel-2 (2017–2020), Metro Vancouver Land Cover Classification 2014, ESA WorldCover 2020, Esri Land Cover 2020
Methods: Monthly median compositing, land cover classification (XGBoost, Random Forest), training label correction, post-classification change detection using class probabilities, comparison with global land cover products
Stack: Python, Xarray, Rasterio, GeoPandas, XGBoost, scikit-learn, STAC, Microsoft Planetary Computer, QGIS
Background
Impervious surfaces are those that water can’t soak into: rooftops, roads, parking lots and sidewalks, but also degraded ground such as compacted soil and burn scars. Rain that falls on them runs off instead of soaking in, so the more of a watershed they cover, the more severe flooding can be. This internal Sparkgeo project investigated mapping impervious surfaces and tracking their development over time using freely available Sentinel-2 imagery. Metro Vancouver was used as the study area. Because burn scars act as impervious surfaces, the work partially led to a later project on wildfires and flood damage.
My contribution
I owned the project end to end: design, data sourcing, preprocessing, training data, model training, validation and reporting. A colleague helped with preparing the training data.
- Input features: for each year from 2017 to 2020, I computed the median value of each Sentinel-2 band for each month from May to September, at 10 m. The per-band monthly medians were the model inputs. Taking medians reduces the effect of clouds, and keeping each month separate captures seasonal changes in vegetation that help tell land cover types apart.
- Reference data: training and validation labels came from Metro Vancouver’s 2014 land cover classification, made at 5 m from a combination of RapidEye satellite imagery and lidar. I merged spectrally similar vegetation classes, since vegetation type mattered little for this purpose, then aggregated the labels to the Sentinel-2 10 m grid, giving each pixel the class covering most of its area.
- Correcting the labels: the model was trained on the 2017 composite, the closest in time to the reference data, but three years apart is enough for land to change. Training areas were sampled across classes by stratified random sampling, and each was checked by eye in QGIS against the 2017 imagery. Where the reference was clearly out of date, such as vegetation since converted to development, we digitized polygons with the correct class, and these overrode the reference labels during training.
- Model: in initial tests XGBoost outperformed Random Forest, so I refined only XGBoost. Pixels in each class were split at random, 70/20/10, into training, validation and test sets.
- Change over time: I applied the 2017 model to the 2018, 2019 and 2020 composites and compared each year’s map with 2017, pixel by pixel. Classes were mostly stable, but there were false changes, especially between spectrally similar classes such as other built-up surfaces and pavement.
- Conservative change rule: land cover changes slowly, and the 2017 map was the most reliable because the model was trained on it, so I made the change layer conservative. When a pixel’s class changed from A in 2017 to B in a later year, I used the model’s class probabilities to compare how confidently each year made its call: in 2017, the probability of A minus the probability of B; in the later year, the probability of B minus the probability of A. The change was kept only if the later year’s margin exceeded the 2017 margin by at least 0.1; otherwise the pixel stayed class A.
- Final classes: for the final maps I aggregated the classes into four: impervious surface, vegetation, soil and water.
- Comparison with global products: two free global land cover maps at the same 10 m resolution, both also made from Sentinel-2, had just been released for 2020: ESA WorldCover and Esri Land Cover. A custom model is only worth building if it does better than maps that already exist, so I compared ours with them. I aggregated both to the same four classes, mapped where each disagreed with our 2020 classification and which classes they disagreed on, and judged which was right in those areas against the Metro Vancouver reference and the 2020 Sentinel-2 composite.
Outcome
- Accuracy: macro-F1 (F1 averaged across classes) on the test set was above 0.85. The built-up classes (buildings, paved and other built-up surfaces), the main interest of the project, performed particularly well. Note this accuracy assessment was performed before the final 4-class aggregation.
- Change detection: on visual inspection, the conservative change rule removed most false changes while keeping real ones, however the improvement was not quantified.
- Comparison with global products: by visual inspection, our map agreed best with the Metro Vancouver reference and with the imagery. ESA WorldCover also did well, though ours tended to be more accurate. Esri Land Cover was considerably worse: it over-generalized, folding many small areas of vegetation into the impervious class. That our model did best over the area it was trained on isn’t surprising, since global products have to work everywhere. We expected it to do substantially worse elsewhere, and planned more training over new areas to improve that, but the project ended first.
- Limitations:
- The reference data is itself a modelled map, not ground truth, and was three years older than the imagery; manual checking mitigated but didn’t remove that.
- The training, validation and test split was made by pixel rather than by area, so neighbouring pixels from the same areas likely appear in more than one set, which probably makes the test score optimistic.
- Neither the change layer nor the comparison with other land cover products was assessed quantitatively, and the model wasn’t tested outside Metro Vancouver.