UNDP's chief digital officer explained how he measures AI programs, and his answer split along a line robotics safety engineers know well: bounded behavior can be verified, open-ended behavior cannot.

Robert Opp, UNDP's Chief Digital Officer, gave an answer this week that any robotics safety engineer would recognize instantly, though he wasn't talking about robots.

He was guest lecturing at the Yale School of the Environment, the session before mine, when a student asked how UNDP actually knows whether its AI programs are working. His answer split cleanly in two.

Direct-service AI, like the platform sending crop alerts to millions of farmers in Telangana, India, or the one letting villagers in Indonesia submit budget priorities by voice, is measurable with tools UNDP already has, things like log frames, focus groups, and cost per person reached. AI used for strategy and policy, the kind that shapes what a ministry decides to do next, is a different problem. The KPIs exist on paper. The framework connecting a baseline to an outcome does not. By his own account, the field is still grappling with it.

That split has a name in the Physical AI world, where I've recently been spending a significant amount of my time. A robot's motor controller, deciding how much torque to apply in the next ten milliseconds, is deterministic. Its behavior is bounded and repeatable, which is why it can be certified against a real safety standard. The reasoning layer above it, deciding what the robot should do next in a messy, open-ended situation, runs on the same probabilistic models behind every large language model today. Nobody has found a way to certify that layer yet, for the identical reason UNDP can't yet measure whether a policy recommendation was good advice. Once a system's behavior stops being bounded enough to trace back to a cause, the tools built to check it stop working.

UNDP hit this wall in monitoring and evaluation. Robotics hit it in functional safety. Two fields with nothing else in common arrived at the same fault line, from opposite directions, in the same year.

The fix robotics is converging on is architectural. Keep the deterministic layer strictly separate from the probabilistic one, certify what can be certified, and never let the reasoning layer make a call the deterministic layer can't override.

Global development doesn't have that luxury yet. A ministry acting on AI-shaped policy advice doesn't get a hardware interlock. Somebody is going to have to build the institutional version of one.

Thoughts?