Why Neural Policies Fail on Hardware Without Control Bounds

When high-torque actuators saturate or thermally derate, black-box neural policies destabilize. Here is how to wrap learned models in deterministic MPC and PID governors.

At recent industry robotics exhibitions, including high-profile demonstrations at the Beijing World Robot Conference, multiple untethered humanoid and bipedal prototypes suffered sudden locomotion collapses and uncontrolled motor runaways. The videos followed a predictable pattern: a robot walked smoothly across a flat stage, encountered a slight lateral foot slip or a step disturbance, and within three execution cycles began violently thrashing its hip and ankle actuators until emergency e-stops cut power or the machine crashed onto the floor.

In post-incident debriefs, the failure mode is almost always misdiagnosed as an external sensor failure, a network drop, or a missing edge case in the simulation training dataset. The root cause is much more fundamental. When a machine learning policy trained purely via reinforcement learning (RL) or imitation learning is deployed directly onto physical hardware, it issues raw current or position setpoints without an explicit mathematical model of actuator state bounds. When physical motors hit torque saturation, bus voltage sag, or thermal derating, the effective plant transfer function shifts instantly. An unconstrained neural network cannot reason about loop gain collapse. Instead of backing off, the network interprets the resulting tracking lag as an uncorrected error and commands even higher control effort, driving the closed-loop system into destructive limit cycles.

For robotics engineering leads, mechatronics researchers, and control system architects, deploying unconstrained neural policies directly to hardware is a liability. It breaks prototypes, wastes months of fabrication budget, and prevents systems from passing real-world safety certification. The solution is not to generate another billion frames of synthetic simulation data. The solution is to isolate the neural policy to trajectory planning and wrap it inside deterministic control envelopes that guarantee stability margins, bounded input rates, and anti-windup protection.

Why Actuator Saturation Breaks Neural Inference

Modern physics engines like Isaac Gym or MuJoCo allow reinforcement learning agents to optimize locomotion policies across tens of thousands of parallel environments. These simulators use domain randomization to perturb friction coefficients, link masses, and communication latency. However, they almost universally rely on simplified actuator models, treating a permanent magnet synchronous motor (PMSM) or quasi-direct drive (QDD) brushless actuator as a linear torque source governed by a simple proportional-derivative equation:

$$\tau = \text{clip}\left(K_p (q_{cmd} - q) + K_d (\dot{q}{cmd} - \dot{q}), -\tau{max}, \tau_{max}\right)$$

This approximation hides the nonlinear dynamics that govern high-performance hardware under stress. On physical robots, the maximum instantaneous torque $\tau_{max}$ is not a static constant. It is a dynamic boundary governed by the inverter bus voltage, back-electromotive force (back-EMF), winding temperature, phase resistance, and instantaneous field-oriented control (FOC) current limits.

When a biped hits a lateral disturbance, the low-level motor controllers demand peak current to correct joint deflection. If the motor is operating near its base speed, the back-EMF voltage approaches the DC link voltage, severely reducing the available voltage margin to force current into the stator windings. The actual delivered torque drops sharply below $\tau_{max}$.

+-------------------------------------------------------------------------+
|                        UNCONSTRAINED NEURAL LOOP                        |
|                                                                         |
|  +------------+   q_des    +---------------+  tau_cmd  +-------------+  |
|  |   Neural   |----------->| Simplified PD |---------->|  Saturating |  |
|  |   Policy   |            |   Actuator    |           |    Motor    |  |
|  +------------+            +---------------+           +-------------+  |
|        ^                                                      |         |
|        |                                                      v         |
|        +----------------- Measured Joint States --------------+         |
|                                                                         |
|  Problem: Saturation causes effective loop gain to drop.               |
|  Neural policy sees persistent error -> outputs extreme q_des.          |
|  Phase margin erodes to zero -> High-frequency runaway.                 |
+-------------------------------------------------------------------------+

To a classical feedback controller with anti-windup, this gain reduction is anticipated and managed. To an unconstrained neural network operating at 50 Hz to 500 Hz, the failure of the joint to reach the commanded position $q_{cmd}$ looks like a rising error signal. Because the policy was trained in an environment where commanding a larger offset produces a larger restorative torque, it outputs an even more aggressive setpoint on the next inference step.

By driving the actuator deeper into hard saturation, the loop gain drops while the effective phase lag spikes by 90 degrees or more. The closed-loop poles cross into the right half of the s-plane, turning a minor tracking error into an uncontrollable, high-frequency torque chatter that destroys planetary gearheads and snaps cycloidal drive pins.

Thermal Drift and Effective Gain Collapse

Actuator performance degrades continuously over duty cycle. During ten minutes of active locomotion testing, motor winding temperatures regularly rise from 25 degrees Celsius to over 85 degrees Celsius. As copper stator windings heat up, phase resistance $R_s$ increases according to the temperature coefficient of copper:

$$R_s(T) = R_{s,0} \left(1 + \alpha_{Cu} (T - T_0)\right)$$

Where $\alpha_{Cu} \approx 0.00393 / ^\circ\text{C}$. A 60-degree temperature rise increases stator resistance by roughly 24 percent. At the same time, high-energy neodymium rotor magnets experience reversible thermal demagnetization, reducing the torque constant $K_t$ by 0.10 to 0.12 percent per degree Celsius.

Combined, these thermal effects mean that for the same commanded FOC phase current, the motor produces significantly less mechanical torque while generating substantially more $I^2 R$ heat, triggering software thermal derating profiles inside the motor drives to prevent permanent demagnetization.

Operational Parameter	Cold Bench State (25 C)	Sustained Run State (85 C)	Impact on Closed-Loop Dynamics
Phase Resistance ($R_s$)	$0.180\ \Omega$	$0.223\ \Omega$	Higher voltage drop, lower peak torque bandwidth
Torque Constant ($K_t$)	$0.095\ \text{Nm/A}$	$0.088\ \text{Nm/A}$	7.4% reduction in direct torque generation
Continuous Torque Limit	$14.2\ \text{Nm}$	$9.8\ \text{Nm}$ (derated)	Premature entry into hard torque saturation
Open-Loop Crossover Freq	$42\ \text{Hz}$	$28\ \text{Hz}$	Phase margin erosion under aggressive neural gains
Peak Sensitivity ($M_s$)	$1.28$	$2.15$	Severe susceptibility to mechanical resonance

When an end-to-end neural network runs without an explicit plant model, it assumes the hardware admittance is time-invariant. It expects that issuing torque vector $\boldsymbol{\tau}_k$ will produce angular acceleration $\boldsymbol{\ddot{q}}_k = \mathbf{M}^{-1}(\mathbf{q})(\boldsymbol{\tau}_k - \mathbf{C}(\mathbf{q}, \mathbf{\dot{q}})\mathbf{\dot{q}} - \mathbf{g}(\mathbf{q}))$.

When thermal derating cuts available torque by 30 percent, the true plant acceleration falls short of the policy expectation. If the neural policy operates with high internal gain, the mismatch pushes the system out of the narrow distribution of states seen during training, triggering out-of-distribution hallucinations where the network commands non-physical, high-frequency joint oscillations.

The Architecture of Bounded Deterministic Control

Preventing hardware runaway requires decoupling high-level behavioral reasoning from low-level joint authority. Learning-based policies excel at semantic task planning, footstep placement heuristics, and high-level trajectory shaping. They are completely unsuited for direct torque allocation or raw high-bandwidth current loop regulation.

A reliable mechatronics architecture uses a three-tier execution hierarchy:

+-------------------------------------------------------------------------+
|                   THREE-TIER DETERMINISTIC ENVELOPE                     |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | LAYER 1: Learned Task Policy (10 - 50 Hz)                         |  |
|  | Outputs reference trajectories: p_ref(t), v_ref(t), F_contact     |  |
|  +-------------------------------------------------------------------+  |
|                                  |                                      |
|                                  v Reference Trajectories               |
|  +-------------------------------------------------------------------+  |
|  | LAYER 2: Deterministic MPC / QP Governor (100 - 500 Hz)           |  |
|  | Constrained optimization: Enforces tau_min <= tau <= tau_max,     |  |
|  | Zero-Moment Point (ZMP), friction cones, and slip boundaries      |  |
|  +-------------------------------------------------------------------+  |
|                                  |                                      |
|                                  v Bounded Torque / Accel Commands      |
|  +-------------------------------------------------------------------+  |
|  | LAYER 3: Low-Level Joint Regulators (1 - 20 kHz)                  |  |
|  | Linear PID / FOC with anti-windup, notch filters, and            |  |
|  | verified gain/phase margins against identified plant dynamics     |  |
|  +-------------------------------------------------------------------+  |
|                                  |                                      |
|                                  v Phase Voltages (PWM)                 |
|  +-------------------------------------------------------------------+  |
|  | Physical Plant Actuators (Brushless PMSM + Harmonic Drive)        |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+

1. The High-Level Neural Planner (10 to 50 Hz)

The neural policy does not command joint torques or raw PWM duty cycles. Instead, it generates Cartesian reference trajectories for the center of mass, desired foot swing profiles, or nominal end-effector wrenches. Operating at a moderate frequency, this layer can absorb sensor noise and visual inference delays without threatening the closed-loop stability of the physical joints.

2. The Model Predictive Control Governor (100 to 500 Hz)

The reference trajectories from the neural network are passed to a deterministic Model Predictive Control (MPC) or Quadratic Program (QP) governor. This layer solves a constrained optimization problem at every time step:

$$\min_{\mathbf{u}} \sum_{k=0}^{N} |\mathbf{x}k - \mathbf{x}{ref,k}|_{\mathbf{Q}}^2 + |\mathbf{u}k|{\mathbf{R}}^2$$

$$\text{subject to: } \mathbf{x}{k+1} = \mathbf{A}\mathbf{x}k + \mathbf{B}\mathbf{u}k$$ $$\mathbf{u}{min}(T) \le \mathbf{u}k \le \mathbf{u}{max}(T)$$ $$\Delta \mathbf{u}{min} \le \mathbf{u}k - \mathbf{u}{k-1} \le \Delta \mathbf{u}{max}$$ $$\mathbf{C}\mathbf{x}_k + \mathbf{D}\mathbf{u}k \le \mathbf{b}{friction}$$

Notice that the actuator limits $\mathbf{u}{min}(T)$ and $\mathbf{u}{max}(T)$ are parameterised dynamically by real-time telemetry from the motor drives, including winding temperature and DC bus voltage. If an actuator begins to overheat, the MPC governor automatically shrinks the feasible torque space.

If the neural policy commands a footstep or torque profile that would violate contact friction cones or exceed the thermal budget of the joint, the optimizer projects the command onto the boundary of the safe, dynamically feasible set. The neural policy is completely physically incapable of commanding an unstable or saturating input.

3. The Low-Level Deterministic Feedback Loop (1 to 20 kHz)

The output of the MPC governor is passed to dedicated joint-level microcontrollers running deterministic Field-Oriented Control and tuned PID loops with rigorous anti-windup tracking. This layer runs on real-time microcontrollers or FPGA hardware, guaranteeing execution timing without operating system jitter.

Receipts: Hardware Stability Under Actuator Faults

To demonstrate the mechanical and electrical difference between raw neural policy execution and a deterministically governed envelope, consider a representative composite benchmark of an 80 kg bipedal robot platform subjected to a 30 percent torque saturation event during dynamic stair descent.

The following data represents an illustrative composite benchmark compiled from empirical hardware testing protocols across brushless quasi-direct drive joints subject to sudden DC bus voltage step-downs and thermal derating.

Verification Metric	Raw End-to-End Neural Policy	Neural Policy + Classical PID Outer Loop	Neural Policy Wrapped in MPC Governor	Target Spec (Safe Operation)
Peak Joint Torque Chatter	$148.6\ \text{Nm}$	$72.1\ \text{Nm}$	$38.4\ \text{Nm}$	$\le 45.0\ \text{Nm}$
Max Current Slew Rate ($di/dt$)	$412\ \text{A/s}$ (Violated)	$185\ \text{A/s}$	$64\ \text{A/s}$	$\le 80\ \text{A/s}$
Phase Margin at Crossover	$-14.2^\circ$ (Unstable)	$21.5^\circ$ (Marginal)	$52.8^\circ$ (Stable)	$\ge 45.0^\circ$
Gain Margin	$-2.8\ \text{dB}$	$3.4\ \text{dB}$	$8.6\ \text{dB}$	$\ge 6.0\ \text{dB}$
Peak Sensitivity ($M_s$)	$\infty$ (Limit Cycle)	$2.84$	$1.31$	$\le 1.40$
Recovery from 30% Saturation	Runaway / E-Stop Crash	Oscillatory (1.8 s ring)	Smooth Tracking (0.22 s)	$< 0.40\ \text{s}$ Settling
Hardware Damage Incurred	Sheared cycloidal pins	High bearing thermal wear	Zero mechanical stress	Zero structural damage

Under an unconstrained neural policy, entering torque saturation immediately causes the open-loop phase margin to flip negative ($-14.2^\circ$). The joint enters a violent limit cycle, drawing destructive current slew rates ($412\ \text{A/s}$) that trip the inverter overcurrent protection, dropping the robot.

When the same policy is filtered through an MPC governor with an identified plant model, the optimizer respects dynamic torque limits. The phase margin remains safely pinned at $52.8^\circ$, peak sensitivity stays below the critical $1.40$ threshold, and the system smoothly tracks through the disturbance without hardware wear.

Frequency-Domain Verification Before Hardware Deployment

Before flashing any control policy to a physical robot or bench test rig, the control loop must be validated using rigorous frequency-domain system identification. You cannot verify safety simply by running ten thousand random rollout seeds in a physics simulator.

+-------------------------------------------------------------------------+
|                    PRE-FLIGHT VALIDATION WORKFLOW                       |
|                                                                         |
|  1. Swept-Sine System ID                                                |
|     Inject multisine chirp (0.1 - 500 Hz) -> Measure actual plant G(s)  |
|     Identify compliance, gear resonance, and deadband                   |
|                                                                         |
|  2. Closed-Loop Margin Verification                                     |
|     Compute Loop Transfer L(s) = C(s) * G(s)                            |
|     Confirm Phase Margin >= 45 deg, Gain Margin >= 6 dB                 |
|     Confirm Peak Sensitivity Ms = max |1 / (1 + L(s))| <= 1.4           |
|                                                                         |
|  3. Dynamic Governor Constraint Setup                                   |
|     Set MPC input bounds to match identified motor thermal limits       |
|     Configure slew rate limiters to prevent inverter overcurrent        |
|                                                                         |
|  4. Hardware-in-the-Loop Saturation Injection                           |
|     Inject artificial torque derating (30%) in test rig                 |
|     Verify governor prevents windup and maintains positive margins      |
+-------------------------------------------------------------------------+

Step 1: Swept-Sine Plant Identification

Mount the actuator to a rigid dyno bench. Inject a multisine or swept-sine torque perturbation across the operational bandwidth (0.1 Hz to 500 Hz). Measure the actual rotor position and output shaft torque with high-resolution encoders. Fit a non-parametric frequency response function (FRF) to capture structural compliance, stator inductance delays, and gearhead backlash.

Step 2: Compute Sensitivity and Margins

Construct the open-loop return ratio $L(s) = C(s)G(s)$, where $G(s)$ is the identified physical plant and $C(s)$ represents the low-level controller transfer function. Calculate the classical sensitivity function $S(s)$ and complementary sensitivity function $T(s)$:

$$S(s) = \frac{1}{1 + L(s)}, \quad T(s) = \frac{L(s)}{1 + L(s)}$$

Verify that the peak sensitivity $M_s = \max_{\omega} |S(j\omega)|$ does not exceed $1.40$ (roughly $3\ \text{dB}$). An $M_s > 2.0$ guarantees that unexpected disturbances or thermal parameter shifts will cause significant overshoot and possible oscillation.

Step 3: Hard-Code Actuator State Boundaries in the Governor

Program the MPC governor with the true physical boundaries extracted during system identification. If the gearbox has a maximum shock torque limit of $60\ \text{Nm}$, set the hard optimization constraint at $50\ \text{Nm}$. If the current loop bandwidth drops above $60\ \text{Hz}$, enforce input rate limits (slew limits) in the QP solver to prevent high-frequency setpoints from exciting mechanical anti-resonances.

What this means for LabCD

Building control architectures that survive contact with physical hardware requires treating system identification and stability proofs as first-class engineering deliverables. LabCD (labcd.ai) provides the analytical tooling to bridge learned policies with deterministic plant bounds.

By importing swept-sine frequency responses, parameterizing thermal derating envelopes, and generating real-time MPC governors with verified gain and phase margins, engineering teams can replace empirical guesswork with reproducible, mathematically defended control systems. You design the controller against the true identified plant, verify the sensitivity peaks, and export deterministic C code that runs reliably on physical microcontrollers.

Practical Checklist for Deploying Learned Policies

When transitioning any neural locomotion or manipulation policy from simulation to physical hardware, use the following operational checklist:

Strip torque authority from the neural network: Never let a neural policy output raw current, voltage, or unconstrained motor torque directly. Restrict network outputs to kinematic references ($p_{ref}, v_{ref}$) or compliance parameters.
Parameterize dynamic thermal limits: Connect inverter temperature telemetry directly into your mid-level MPC or low-level current limiters. As $R_s$ rises, clamp maximum allowed acceleration in real time.
Audit the sensitivity peak ($M_s$): Measure the frequency response of each joint on a dynamometer. Verify that the closed-loop system maintains $M_s \le 1.4$, a phase margin $\ge 45^\circ$, and a gain margin $\ge 6\ \text{dB}$ across the entire thermal operating envelope.
Enforce hard rate limits ($du/dt$): Place explicit slew-rate limiters on commands entering the low-level motor drives to prevent current saturation and harmonic excitation during unexpected foot impacts.
Bench-test torque step-downs: Before untethered flight or walking tests, place the robot on a gantry. Artificially throttle the DC bus voltage by 25 percent while the policy is executing. If the robot enters a limit cycle or chatters, the policy lacks sufficient deterministic envelope protection.

Direct Summary: Why Neural Policies Fail on Hardware

Why do learned neural policies fail and cause runaway on real hardware? Neural policies trained in unconstrained simulation assume time-invariant plant dynamics and linear torque responses. When physical motors experience back-EMF voltage drop, winding heating, or physical torque saturation, the effective open-loop gain collapses and phase lag increases. Because the neural policy lacks an internal model of these hardware boundaries, it interprets tracking lag as error and commands even higher control effort, driving the closed-loop system into destructive limit-cycle oscillations. Wrapping the policy in a deterministic MPC governor and verified low-level PID loops enforces physical state constraints and guarantees stability margins under all operating conditions.