Satellite Evidence in the Courtroom

Teaching AI to think like a lawyer

The most technically novel element of the project addressed a challenge that courts have not yet had to grapple with directly: what happens when the evidence has been processed by an artificial intelligence? Machine learning and deep learning are powerful tools for analysing satellite imagery, but they introduce the possibility of misclassification – and a misclassification in a legal context could mean inadmissible evidence, or worse, a wrongful conviction.

The team's solution borrowed a concept from legal practice. ‘Hot-tubbing’ is a process used in some court systems in which two expert witnesses are required to set out, jointly, what they agree on and what they disagree on – a structured way of handling scientific evidence that is genuinely disputed. Rapach and her colleagues applied the same logic to their AI model, running it in two configurations: one steered to be cautious about false positives, as a prosecution model might need to be, and one steered to be cautious about false negatives, as a defence model might require. By comparing the outputs of the two, it becomes possible to identify which classifications both models agree on – and those are the classifications that carry the greatest evidential weight. Areas of disagreement are flagged explicitly, allowing legal practitioners to focus further scrutiny where it is needed.

"Errors in AI-analysis evidence can lead to different predictions of the same evidence," says Rapach. “The hot-tubbing approach, inspired by established legal practice, directly addresses this by steering the model to its extremes and establishes consensus in these extremes. This not only makes it easier for the court to identify where experts could agree and disagree, but also strengthens the reliability, and ultimately legal weight, of agreed conclusions.”