Reinforcement learning has been beating rule-based HVAC control in simulation for the best part of a decade. Almost none of it has reached a real building.
The reason is not performance. It is that a learned policy is a function nobody can read. Ask a facilities manager to hand temperature control of a residential block to a neural network and the first question is not how much energy it saves. It is what happens on the coldest night of the year when the model sees a state it was never trained on.
Nobody had a good answer. This paper offers one.
Certify after training, not during
Two agents were trained in EnergyPlus through the Sinergym interface, one using Proximal Policy Optimization and one using Soft Actor-Critic, both minimising energy while holding thermal comfort.
The contribution is what happens next. Post-training safety certification using Lipschitz-based forward invariance analysis, to guarantee constraint satisfaction. In plain terms: after the agent has finished learning, they prove mathematically that it cannot leave the safe region of operation.
The proof happens after the learning. You do not have to trust the training, only the certificate.
Note that the two agents won different prizes. PPO cut comfort violations by 67%. SAC saved 27.6% of energy. That is a genuine trade-off rather than a single winner, and it is the trade-off a building owner has to make explicitly rather than discovering after handover.
Why this is the more important HVAC paper of the fortnight
There was a second HVAC paper published two days later, ADAPT, which I have written up separately. It posts good numbers on energy and comfort and solves a real deployment problem.
This one solves a procurement problem, and procurement is where building AI actually dies. A 27.6% saving does not get you past a risk committee. A guarantee might.
What to ask a vendor
- What are the constraints, stated formally? Not what the system tries to do. What it is provably prevented from doing.
- Is the guarantee on the trained policy, or on the training process? Only the first survives contact with a state the model never saw.
- What happens when a sensor fails? Certification assumes the observations are real, and a stuck thermistor is outside that assumption.
- Who signs off? Every credible result in this literature keeps a person accountable. The certificate reduces the risk, it does not transfer the liability.