Scientific validation report
How accurate Carbon-Eye's maps are, measured on 2026-10-15 at 7 Kenyan test areas - from a 0.3 ha plot to a whole county - on land the models never trained on.
In short
- The model beats a do-nothing baseline (predicting the median everywhere) at every site, removing 30%-74% of its error.
- The 90% ranges held the true reference value 90% of the time on average (sites 87%-95%), against a 90% target.
- Dense vegetation is under-estimated: errors grow with carbon density, as with every satellite biomass map. The tables below show by how much.
- A benchmark of tuned settings and other algorithms led to new Random Forest and Gradient Tree Boosting settings (adopted 2026-10-16, after the figures on this page were measured); gradient-boosting libraries not available in Earth Engine did 6% better than the previous ensemble.
- Since 15 November 2026, land the ESA map leaves blank (below its detection limit) counts as zero biomass in training and testing, which removed an over-estimate in drylands (e.g. Machakos: +12.8% to −0.9% against ESA's own county total). The site figures on this page were measured before that change.
- Since 2026-11-21, new runs correct the forest under-estimate when the models compress dense areas: on held-out test blocks at 6 areas the 50-100 t C/ha band went from −10.6 to −6.6 t C/ha on average, with overall error almost unchanged. Details.
- Against independent maps (2026-11-21, 6 areas), Carbon-Eye agrees on where the carbon is but reads lower than NASA/ORNL's 2010 map - conservative totals. Details.
- Accuracy here means agreement with the ESA CCI reference map, not with field plots. An independent lidar check is reported separately.
How it was measured
- Reference. ESA CCI Above-Ground Biomass v6 (2020), converted to carbon with the IPCC fraction 0.47. Imagery window 2024-10-01 to 2026-10-01.
- Spatial blocks. Each area is cut into squares and whole squares are held out: held-out spatial blocks (~0.71 km squares, ~101 blocks; test areas never trained on). Test points are never next to training points, which is stricter than a random split.
- Three-way split. Models are fitted on one part, the ensemble is weighted on a second, and every number here comes from the third - the test blocks.
- Baseline. Predicting the median of the held-out reference points everywhere. “Error removed” is 1 - RMSE(model) / RMSE(baseline).
- 90% ranges. Calibrated on held-out points, then checked on the test blocks: the share of test points whose reference value fell inside its range.
Results by site
| Site | Mean (t C/ha) | R² RF / GTB / SVM | Ensemble RMSE | Baseline RMSE | Error removed | 90% coverage | Test points |
|---|---|---|---|---|---|---|---|
| Kitui smallholder farm (277 ha) | 9.0 | 0.53 / 0.54 / 0.50 | 7.9 | 12.0 | 34% | 91% | 530 |
| 0.25 ha plot, Kitui | 4.8 | 0.57 / 0.56 / 0.55 | 7.5 | 13.1 | 42% | 87% | 461 |
| Mau farmland (~970 ha) | 6.0 | 0.50 / 0.47 / 0.40 | 6.6 | 9.4 | 30% | 88% | 625 |
| Kakamega rainforest (~2,500 ha) | 96.8 | 0.89 / 0.89 / 0.91 | 14.5 | 55.9 | 74% | 92% | 269 |
| Nairobi CBD (built-up) | - | -9.27 / -12.95 / -11.99 | - | - | n/a | - | 4 |
| Kirinyaga county | 23.9 | 0.90 / 0.90 / 0.89 | 8.0 | 28.1 | 71% | 89% | 1095 |
| Makueni county | 10.0 | 0.86 / 0.88 / 0.78 | 4.7 | 14.8 | 68% | 95% | 952 |
| Kitui farm, GEDI lidar training on | 9.9 | 0.04 / 0.59 / 0.56 | 8.0 | 12.0 | 34% | 95% | 530 |
R² rewards variety: small uniform farms score lower R² even when their error in t C/ha is small. The built-up control (Nairobi CBD) has almost no reference biomass, so there is nothing to score - it is included to show the map correctly reports no carbon there. The lidar-training row uses the same farm with the opt-in GEDI setting.

Errors by carbon density
Average error (predicted minus reference, t C/ha) on the test points, by the reference carbon density. Negative means the map is too low. Bands with fewer than 5 points are not scored (count shown).
| Site | 0-10 t C/ha | 10-25 t C/ha | 25-50 t C/ha | 50-100 t C/ha | 100+ t C/ha |
|---|---|---|---|---|---|
| Kitui smallholder farm (277 ha) | +2.8 (n=319) | -2.5 (n=141) | -14.1 (n=65) | -22.5 (n=5) | - |
| 0.25 ha plot, Kitui | +3.0 (n=271) | -1.4 (n=135) | -12.0 (n=50) | -26.8 (n=5) | - |
| Mau farmland (~970 ha) | +0.8 (n=484) | -4.5 (n=119) | -18.6 (n=17) | -47.7 (n=5) | - |
| Kakamega rainforest (~2,500 ha) | +10.7 (n=12) | +5.6 (n=33) | +3.3 (n=35) | -0.6 (n=27) | -2.1 (n=162) |
| Kirinyaga county | +0.8 (n=677) | +3.7 (n=181) | -0.6 (n=144) | -8.1 (n=56) | -17.3 (n=37) |
| Makueni county | +0.8 (n=673) | +1.2 (n=154) | +0.1 (n=106) | -14.1 (n=19) | - |
- Kitui smallholder farm (277 ha): in the 25-50 t C/ha band the map is 14.1 t C/ha too low on average (65 points).
- 0.25 ha plot, Kitui: in the 25-50 t C/ha band the map is 12.0 t C/ha too low on average (50 points).
- Mau farmland (~970 ha): in the 25-50 t C/ha band the map is 18.6 t C/ha too low on average (17 points).
- Kirinyaga county: in the 100+ t C/ha band the map is 17.3 t C/ha too low on average (37 points).
- Makueni county: in the 50-100 t C/ha band the map is 14.1 t C/ha too low on average (19 points).
- This is signal saturation and the pull of any model towards the average: satellites stop seeing more biomass in dense canopy. Totals for dense woodland and forest are therefore likely under-estimates; sparse farmland tends to read slightly high. Every run now shows this breakdown in its Validation tab.
Forest correction (2026-11-21)
Measured 2026-11-20 on held-out spatial test blocks. The correction: actual = a + b x ensemble, fitted on the calibration points (never trained or tested on); applied only when b > 1 and there are at least 100 calibration points. Bias is predicted minus reference (t C/ha); negative means the map is too low. Before → after.
| Area | Applied | 25-50 t C/ha | 50-100 t C/ha | RMSE (t C/ha) | Test points |
|---|---|---|---|---|---|
| Kiambu county | Yes | −6.0 → −4.3 (n=69) | −4.7 → +0.1 (n=87) | 6.00 → 6.01 | 959 |
| Nyeri county | Yes | −2.2 → +1.1 (n=180) | −13.0 → −7.1 (n=97) | 6.83 → 6.50 | 1119 |
| Kakamega county | Yes | −7.0 → −6.5 (n=49) | −2.6 → +0.2 (n=37) | 4.88 → 4.91 | 808 |
| Machakos county | Yes | −6.9 → −3.6 (n=55) | - | 3.31 → 3.33 | 1386 |
| Elgeyo-Marakwet county | Yes | −9.1 → −8.0 (n=156) | −16.3 → −14.6 (n=80) | 9.64 → 9.54 | 1132 |
| Kitui smallholder farm (277 ha) | No (slope 0.97) | −16.6 → −16.6 (n=13) | - | 7.05 → 7.05 | 177 |
| Kabarak University campus (94 ha) | Yes | −16.6 → −14.4 (n=20) | −28.7 → −24.4 (n=22) | 7.15 → 6.49 | 633 |
| Kirinyaga county | Yes | −2.2 → −0.9 (n=140) | −7.0 → −3.0 (n=75) | 7.08 → 7.19 | 1091 |
- Across the 6 areas with enough 50-100 t C/ha test points (398 points), the average bias in that band went from −10.6 to −6.6 t C/ha.
- Overall RMSE changed by −0.7 to +0.1 t C/ha where the correction applied - it moves the dense end without making the rest worse.
- It narrows the gap rather than closing it everywhere: Elgeyo-Marakwet and the Kabarak campus are still well under the reference in dense forest. Field plots and lidar are the way to close the rest.
- A farm whose models don't compress (slope at or below 1) is left as it is, as are runs made before 2026-11-21 and runs weighted on GEDI lidar. Each run's Validation tab shows whether the correction applied and its own before/after.
Comparison with independent maps (2026-11-21)
The same areas, mapped by groups that had nothing to do with Carbon-Eye: NASA/ORNL's 2010 global biomass map and NASA GEDI L4B spaceborne lidar (2019-2023), plus the ESA CCI map Carbon-Eye learns from. Means are compared on the same land; the correlation is over 1 km cells and shows whether the maps agree on where the carbon is. Runs: current method (forest correction, spatial test), Balanced preset; the Kitui farm with the gate's settings.
| Area | Carbon-Eye | NASA/ORNL 2010 | GEDI L4B lidar | ESA CCI (reference) |
|---|---|---|---|---|
| Kiambu county | 17.5 t C/ha | 24.4 (−28.0%, r 0.90) | 15.6 (−16.0%, r 0.46, 57% covered) | 19.6 (−8.4%, r 0.95) |
| Kirinyaga county | 22.5 t C/ha | 27.5 (−18.1%, r 0.94) | 19.5 (+2.9%, r 0.80, 32% covered) | 22.9 (−1.9%, r 0.97) |
| Kakamega county | 14.8 t C/ha | 33.7 (−56.1%, r 0.76) | 15.7 (−21.1%, r 0.74, 20% covered) | 15.2 (−2.4%, r 0.94) |
| Machakos county | 4.2 t C/ha | 10.8 (−61.2%, r 0.75) | 5.7 (−27.3%, r 0.43, 89% covered) | 5.7 (−21.6%, r 0.85) |
| Kabarak University campus (94 ha) | 4.8 t C/ha | 19.2 (−74.8%) | 2.2 (+114.8%, 89% covered) | 9.2 (−47.2%) |
| Kitui smallholder farm (277 ha) | 13.0 t C/ha | 30.3 (−57.0%, r 0.87) | 25.6 (−45.3%, 33% covered) | 12.1 (+7.5%, r 0.89) |
t C/ha; in brackets, Carbon-Eye minus that map on the same land, and the 1 km correlation where there were enough cells. GEDI samples tracks, not every hectare - its coverage is shown when under 90%.
- Carbon-Eye reads lower than NASA/ORNL at every area (−74.8% to −18.1%), while agreeing on where the carbon is (r 0.75-0.94). NASA/ORNL describes 2010 - before a decade of tree-cover loss - with a different method.
- Against GEDI lidar, where it covers at least half the area, the difference is −27.3% to +114.8%.
- Most of the gap to NASA/ORNL is already in the ESA reference the models learn from (Carbon-Eye minus ESA: −47.2% to +7.5%). Totals are therefore conservative compared with NASA/ORNL; field plots, not another satellite map, settle which is right for a given site.
- Chloris, Planet and Sylvera maps are commercial and not openly available to compare against.

Independent lidar check
NASA GEDI spaceborne lidar measures canopy structure directly and was never used for training (unless the opt-in setting is on). Mean difference, model minus lidar, over footprints in the area:
| Site | Footprints | Model - lidar (t C/ha) | Correlation |
|---|---|---|---|
| Kitui smallholder farm (277 ha) | 731 | -8.8 | 0.46 |
| 0.25 ha plot, Kitui | 402 | -6.6 | 0.35 |
| Mau farmland (~970 ha) | 1134 | -6.2 | 0.33 |
| Kakamega rainforest (~2,500 ha) | 16 | - | - |
| Nairobi CBD (built-up) | 15 | - | - |
| Kirinyaga county | 1450 | +8.6 | 0.62 |
| Makueni county | 1482 | +4.6 | 0.19 |
| Kitui farm, GEDI lidar training on | 188 | -6.8 | 0.13 |
The ESA reference - and any model trained only on it - reads lower than lidar on farms and higher in some counties. That gap is the reference map's, which is why the opt-in “Calibrate with GEDI lidar” setting exists.
Model benchmark
Nested spatial cross-validation at 5 sites (Kakamega rainforest: 978 points, 119 blocks; Kirinyaga county: 3835 points, 127 blocks; Kitui farm: 1688 points, 122 blocks; Makueni county: 3259 points, 131 blocks; Mau farmland: 2530 points, 119 blocks). Three outer folds of whole blocks; inside each, two inner block folds choose settings from a small grid, so the outer fold that scores them never influenced the choice. All models see the same points and folds.
| Model | Pooled RMSE | Pooled R² | Error removed vs median | Bias |
|---|---|---|---|---|
| Median baseline | 25.97 | 0.308 | 0.0% | -4.77 |
| Ensemble (app defaults) | 8.57 | 0.925 | 67.0% | -0.60 |
| RF default | 9.11 | 0.915 | 64.9% | -0.01 |
| RF tuned | 8.43 | 0.927 | 67.5% | +0.10 |
| GTB default | 8.69 | 0.922 | 66.5% | -0.93 |
| GTB tuned | 8.48 | 0.926 | 67.3% | -0.75 |
| SVM default | 9.05 | 0.916 | 65.1% | -0.82 |
| SVM tuned | 8.95 | 0.918 | 65.6% | -0.88 |
| Ensemble (tuned) | 8.30 | 0.929 | 68.0% | -0.50 |
| Extra Trees | 8.19 | 0.931 | 68.5% | +0.36 |
| XGBoost | 8.09 | 0.933 | 68.8% | +0.15 |
| LightGBM | 8.23 | 0.930 | 68.3% | +0.11 |
- Settings adopted. The rule, fixed before looking: change a default only if better at 4 of 5 sites, by at least 2% on average, with no site more than 2% worse. Each candidate was then re-checked as a fixed setting on every outer fold: RF {min_leaf:15,vps:6} → {min_leaf:5,vps:10}: -8.4% RMSE, better at 5/5 sites (adopted). GTB {shrinkage:0.05,max_nodes:6} → {shrinkage:0.1,max_nodes:12}: -4.4% RMSE, better at 5/5 sites (adopted). SVM tuning failed the rule (better at 4/5, worst site 2.5%), so it is unchanged.
- Other algorithms. Extra Trees, XGBoost and LightGBM beat the previous ensemble at every site (by 5.8%, 6.7% and 5.9% lower RMSE on average). They don't run inside Google Earth Engine, where the maps are made; the new settings close about 63% of that gap. Bringing tree models trained outside Earth Engine into the map is a possible next step.
Limitations
- Accuracy is measured against a modelled reference map (ESA CCI), not field plots. Its own errors are in the labels.
- High-biomass bands have few test points at farm sites; their numbers are indicative.
- Seven test areas in Kenya are not every landscape: drylands, mangroves and plantations are not separately tested yet.
- Field-plot validation (e.g. national forest inventory plots) would be the next independent check.
Reproduce it
Every run records its dataset versions, settings, code revision and an area fingerprint, and gives the same numbers for the same inputs. Saved areas can be recomputed from scratch by a second person, who sees whether the result fingerprint matches. Area fingerprints of the sites above:
- Kitui smallholder farm (277 ha): sha256:bc58f5445cabdc3e
- 0.25 ha plot, Kitui: sha256:9766a077160bb648
- Mau farmland (~970 ha): sha256:e98f97d0cf2a0342
- Kakamega rainforest (~2,500 ha): sha256:15cdb2a3baa7e35d
- Nairobi CBD (built-up): sha256:b2f894137ae5372d
- Kirinyaga county: sha256:2ffd15bc93f1827a
- Makueni county: sha256:05e734606d50e4e6
- Kitui farm, GEDI lidar training on: sha256:bc58f5445cabdc3e