IDO
Insight

labcd · 2026-09-20 · 13 min

Benchmarking PLC AI Copilots on Deterministic Logic

AI tools generate IEC 61131-3 code that compiles cleanly but fails on scan cycles. Here is what breaks in Structured Text and how to verify it.

An industrial automation workbench with an open control box and PLC hardware connected to a programming laptop

If you prompt an AI copilot to write a Structured Text routine for a two-cylinder pick-and-place sequence, it will hand you twenty lines of clean code in three seconds. You paste it into CODESYS V3.5, TwinCAT 3, or Siemens TIA Portal. You hit compile. Zero errors. Zero warnings.

You download the routine to a bench test PLC, wire a push button to an input, toggle the start bit, and the pneumatic slide moves forward. Then it stalls halfway through the second cycle. Or the clamp opens while the arm is still descending. Or worse, the cylinder fires once on power-up before anyone presses the start button.

The compiler did not catch any of this because compilers check syntax, variable typing, and memory boundaries. They do not check scan cycle physics.

Generalist mechatronics engineers and robotics builders are increasingly turning to generative tools like PLC Copilot, Schneider Electric's EcoStruxure Copilot, PLCAutoPilot, and generic LLMs to draft IEC 61131-3 logic. If PLC programming is only twenty percent of your job, the appeal is obvious. You want to avoid spending four hours looking up the exact syntax for a time-on-delay function block or building boilerplate state machines.

Generative models treat Structured Text as if it were Python, C++, or standard procedural scripts. In a normal programming environment, code executes sequentially from top to bottom, finishes, and releases execution. A PLC does not work that way. It executes within a strict, continuous cyclical scan (typically 1 ms to 20 ms) where execution order, variable persistence, memory retention, and scan-to-scan state transitions govern physical motion. When generative AI writes Structured Text, this mismatch creates subtle, non-deterministic bugs that can break hardware.

The Real Difference: Scripting vs The Cyclical Scan Engine

To understand why AI-generated Structured Text breaks on real hardware, you have to look at the execution model that large language models assume by default.

An LLM predicts the most statistically probable next token based on training data. Most public code repositories contain Python, JavaScript, C, and Rust. In these languages, if you write an if block containing a function, that function runs when the condition is true and simply does not exist in execution memory when the condition is false. Once the script finishes, local stack variables disappear until the next discrete call.

A PLC task behaves differently. The runtime executes an infinite loop with three distinct phases:

  1. Input Scan: Read physical inputs from IO cards and fieldbus drops into the process image memory.
  2. Program Execution: Execute the user logic sequentially, updating internal variables and output image registers in memory.
  3. Output Scan: Write the updated output image to the physical output cards and field devices.

If your code takes 4 milliseconds to execute, the runtime waits out the remainder of the configured cyclic task time (say, 10 milliseconds), writes the outputs, and immediately restarts from line 1 of the program on the very next scan.

Every line of code you write is re-evaluated dozens or hundreds of times per second. Memory persistence is the rule rather than the exception. When an AI generates Structured Text, it consistently makes structural errors around this cyclical model. Let us walk through the four most common failure points.

Failure 1: Function Blocks Encapsulated Inside Conditionals

The single most frequent error generated by AI assistants involves placing standard IEC timer function blocks (TON, TOF, TP) or edge detection triggers (R_TRIG, F_TRIG) inside IF statements or CASE branches.

Consider this typical snippet generated by a copilot tasked with running a lubrication pump for five seconds after a spindle starts:

// AI-Generated Structured Text (Faulty)
IF SpindleRunning THEN
    LubeTimer(IN := TRUE, PT := T#5S);
    IF LubeTimer.Q THEN
        LubePump := FALSE;
    ELSE
        LubePump := TRUE;
    END_IF;
ELSE
    LubePump := FALSE;
END_IF;

This logic looks entirely reasonable to someone coming from a Python or C background. It compiles without a single error. On hardware, it will fail.

An IEC standard TON timer works by detecting a transition on its IN input from FALSE to TRUE. It then compares the internal runtime clock against its preset time (PT) on every scan where IN remains TRUE. Crucially, to reset the timer for the next cycle, the timer function block must be executed at least once with IN := FALSE.

In the AI-generated code above, when SpindleRunning drops to FALSE, execution jumps straight to the outer ELSE branch. The runtime skips the call to LubeTimer entirely. The function block instance never sees IN := FALSE. Its internal state remains frozen with IN = TRUE and elapsed time ET = T#5S.

The next time SpindleRunning turns TRUE, the timer instantly evaluates LubeTimer.Q as TRUE on scan cycle zero. The pump runs for exactly zero milliseconds. The spindle burns out its bearing.

To write this deterministically, the function block must be called unconditionally on every scan, allowing its internal state machine to process falling edges:

// Correct IEC 61131-3 Structured Text
LubeTimer(IN := SpindleRunning, PT := T#5S);

IF SpindleRunning AND NOT LubeTimer.Q THEN
    LubePump := TRUE;
ELSE
    LubePump := FALSE;
END_IF;

Generic LLMs fail this pattern almost half the time because their training sets contain thousands of forum posts and amateur scripts that make this exact error.

Failure 2: Edge Triggers Trapped Inside State Machines

A related issue occurs when AI tools use rising edge triggers (R_TRIG) inside state machines (CASE statements) to handle step transitions.

Suppose you prompt an AI to create a state machine where State 10 waits for an operator push button before advancing to State 20. The copilot often produces code like this:

// AI-Generated State Transition (Faulty)
CASE State OF
    10: // Wait for Start
        StartEdge(CLK := StartButton);
        IF StartEdge.Q THEN
            State := 20;
        END_IF;
        
    20: // Execute Motion
        CylinderExtend := TRUE;
        // ...
END_CASE;

Here is what happens on the bench. The machine powers up in State 10. The operator presses StartButton. The StartEdge block detects the rising edge, StartEdge.Q turns TRUE, and the routine transitions to State 20 on scan N.

On scan N+1, State is 20. The code inside State 10 is no longer executed. The internal CLK memory of StartEdge remains stored as TRUE.

Later, the machine finishes its sequence and loops back to State 10 while the operator happens to still be holding the button down, or after the operator has released and pressed it again while the machine was in State 30. When State 10 runs again, StartEdge cannot reliably calculate whether a fresh physical transition occurred because it missed the intermediate scans where StartButton went low. Depending on the PLC runtime implementation, it may trigger immediately on re-entry or miss the physical press entirely.

Edge triggers, like timers, should generally live outside the CASE structure, updated once per scan cycle on global or program-scoped variables.

Failure 3: Output Latching and Split Assignments

In procedural code, variable assignments are transient updates to a value in memory. In a PLC, writing to an output variable updates a register that will directly energize a 24V solenoid valve, a contactor, or a digital drive input at the end of the scan.

AI copilots struggle with the concept of single-point output assignment. When asked to control an actuator across multiple operational modes (Manual, Auto, Fault), models frequently generate code that writes to the same output variable from different conditional branches:

// Split Output Assignment Pattern (Hazardous)
IF ManualMode THEN
    IF JogForwardButton THEN
        MotorContactor := TRUE;
    ELSE
        MotorContactor := FALSE;
    END_IF;
END_IF;

IF AutoMode THEN
    IF AutoSequenceRunning THEN
        MotorContactor := TRUE;
    ELSE
        MotorContactor := FALSE;
    END_IF;
END_IF;

If the system is switched to ManualMode, AutoMode is FALSE. The first IF block executes and sets MotorContactor := TRUE because the operator is pressing the jog button.

Ten lines down, the runtime evaluates the second block. Because AutoMode is FALSE, it skips the inner logic. But if an AI wrote a fallback ELSE on the outer block (which copilots do regularly when attempting to write defensive code), MotorContactor is immediately overwritten with FALSE three microseconds later. The physical output card never turns on.

Even without the outer ELSE, split assignment makes debugging on a live machine an operational nightmare. The rule in standard automation is simple: assign an output exactly once per scan cycle, at the very end of your routine, evaluating a consolidated Boolean expression or state mask.

Failure 4: The Hallucinated Safety Interlock

This is the most dangerous pattern observed in AI-generated control code.

Mechatronics engineers often ask copilots to write interlocks: "Generate a safety interlock that stops the gantry if the light curtain is broken or the e-stop is pressed."

The copilot responds with Structured Text that checks the inputs and sets an enable flag to FALSE. 

// AI Safety Attempt (Never use in production)
IF NOT EmergencyStop OR NOT LightCurtain THEN
    GantryEnable := FALSE;
    SafetyTripped := TRUE;
END_IF;

For an engineer whose primary background is software or general robotics, this might look complete. In industrial automation, standard PLC logic is strictly non-safety-rated. Standard cyclic logic can freeze, memory can be corrupted by pointer overruns, and standard digital input cards do not detect cross-circuit wiring faults or welded contactors.

Safety logic must be implemented either in hardwired safety relays or within dedicated functional safety controllers (such as Siemens Safety Integrated, Beckhoff TwinSAFE, or Rockwell GuardLogix) using safety-certified function blocks (like SF_EmergencyStop or SF_TwoHandControl) running in isolated, fail-safe task loops.

When copilots generate standard Structured Text that purports to handle safety interlocking, they create a false sense of compliance. If you paste that code into a standard cyclic task, you violate ISO 13849-1 and IEC 62061 machinery safety standards.

Benchmarking Current Copilots

Recent academic and industrial benchmarking studies have started quantifying how well automation copilots handle deterministic logic.

Research published across IEEE and ScienceDirect evaluating prompt techniques and models for IEC 61131-3 code generation points to a clear trend. While large frontier models (like Claude 3.5 Sonnet and GPT-4o) achieve over 85% syntax accuracy on standard algorithmic Structured Text (math calculations, data conversions, sorting arrays), their zero-shot functional accuracy on state machines and scan-dependent logic drops significantly, often below 45%.

Specialized industrial tools attempt to bridge this gap through retrieval-augmented generation (RAG) and platform-specific grammar injection:

  • PLC Copilot (desktop tool at roughly $99/month) focuses heavily on vendor-specific dialect conversion and Structured Text drafting. It handles Studio 5000 and CODESYS syntax rules better than raw generic models, though it still requires the user to explicitly enforce scan cycle architecture in the prompt.
  • Schneider Electric EcoStruxure Copilot uses proprietary product libraries to ensure generated blocks match Schneider's native architectures (such as EcoStruxure Control Expert and Machine Expert). It reduces variable naming hallucinations, but users must still verify execution sequencing.
  • PLCAutoPilot and similar multi-platform web wrappers offer quick translation across Siemens, Rockwell, Mitsubishi, and CODESYS dialects. However, multi-platform translators frequently struggle with dialect-specific timer behaviors (such as how different platforms handle edge-case resets on pulse timers).

Across all these tools, academic benchmarks consistently show that prompt structure and architectural constraints matter far more than the underlying model size. If you prompt an AI with a raw functional narrative ("make a conveyor run when box is detected"), it will almost always write broken, procedural code. If you constrain the prompt using a strict state-machine template with explicit scan-cycle rules, deterministic accuracy jumps above 80%.

The Non-Specialist's 4-Step Verification Workflow

If you are an ops engineer, robotics builder, or hardware founder who uses AI copilots to speed up control box programming, do not copy-paste generated Structured Text directly onto an active controller.

Use this four-step verification workflow to catch deterministic errors at your desk before hardware commissioning.

+-------------------------------------------------------------+
| 1. Scan-Cycle Audit                                         |
|    - Extract all timers/triggers from conditional blocks   |
|    - Place them at the top-level scan loop                  |
+------------------------------+------------------------------+
                               |
                               v
+-------------------------------------------------------------+
| 2. Memory & Variable Audit                                  |
|    - Verify VAR vs VAR_TEMP vs VAR_STAT scope               |
|    - Ensure internal state flags persist across scans       |
+------------------------------+------------------------------+
                               |
                               v
+-------------------------------------------------------------+
| 3. Single-Point Output Consolidation                        |
|    - Remove assignments from scattered IF/CASE branches     |
|    - Map final output states in a single end-of-scan block  |
+------------------------------+------------------------------+
                               |
                               v
+-------------------------------------------------------------+
| 4. Virtual Simulation & IO Forcing                          |
|    - Run code in simulated PLC runtime                      |
|    - Force inputs manually to test edge transitions         |
+-------------------------------------------------------------+

Step 1: Unconditional Function Block Extraction

Open the generated code and search for every instance of TON, TOF, TP, R_TRIG, and F_TRIG.

If any of these function blocks are indented inside an IF, ELSIF, FOR, WHILE, or CASE branch, pull them out. Move them to the top of your program or execution block so they execute unconditionally on every single scan cycle. Feed their inputs with conditional Booleans rather than putting the entire block call behind a conditional gate.

Step 2: Check Variable Scope and Persistence

Look at the variable declaration block (VAR ... END_VAR).

AI copilots frequently confuse VAR_TEMP (temporary variables that reinitialize to default values on every single scan) with VAR (static/retained variables that maintain their value between scans).

If a variable is used to store a state machine index (State : INT;), an edge memory bit, or an accumulator value, it must never be declared as VAR_TEMP. If it is declared temporarily, your state machine will reset to State 0 on every single scan cycle, creating an apparent lockup where the code simply refuses to advance.

Step 3: Single-Point Output Mapping

Scan the entire routine for output tags (variables mapped to physical %Q* addresses or fieldbus drive words).

Count how many times each output variable appears on the left side of an assignment operator (:=). If any output variable appears more than once, refactor the logic.

Create internal intermediate state Booleans (such as AutoRunRequest and ManualRunRequest), combine them in a single network, and write to the physical output tag exactly once at the bottom of the routine:

// Deterministic Output Consolidation
PhysicalPumpOutput := SafetyChainHealthy AND (AutoPumpRequest OR ManualPumpRequest);

Step 4: Virtual PLC Simulation Before Wiring

Never test generated code directly on a live panel connected to pneumatic valves, linear stages, or servo drives.

Modern engineering suites include local soft-PLC simulation tools that cost nothing to run:

  • CODESYS provides built-in Simulation Mode (Online -> Simulation).
  • Beckhoff TwinCAT allows you to activate configuration on your local development PC target (Local Run Mode).
  • Siemens TIA Portal includes PLCSIM.

For higher-level systems validation, integrated control development platforms like LabCD (labcd.ai) provide tools to synthesize, simulate, and verify control box logic against electrical and physical constraints before physical build-out.

Load the generated routine into your simulation runtime. Open the watch table. Manually force the physical inputs in sequence: trigger the start button, toggle the cylinder extended sensor, force a cycle stop. Watch the state variables in real time. Pay special attention to what happens when you turn an input off after turning it on.

Practical Prompting Rules That Actually Work

If you want better Structured Text out of an AI copilot, stop asking open-ended questions. Treat the copilot as an inexperienced junior drafter who knows syntax rules but has never stood in front of a live panel.

When drafting control logic prompts, enforce these four constraints directly in your prompt text:

  1. Specify the exact architecture: "Use a strict CASE state machine pattern. Declare all states in an explicit ENUM."
  2. Forbid inline function calls: "Call all TON timers and R_TRIG triggers unconditionally at the top of the program. Do not call timers inside CASE branches or IF statements."
  3. Mandate single assignment: "Do not assign physical outputs inside state transitions. Set intermediate Boolean flags inside states and assign physical outputs in a single block at the bottom of the code."
  4. Require explicit variable declarations: "Provide the full VAR ... END_VAR block. Explicitly specify data types and initial values. Do not use temporary variables for state tracking."

When you give an LLM those boundary conditions, the quality of its Structured Text shifts immediately. It stops generating brittle procedural scripts and starts outputting solid, scan-aware industrial logic that you can paste, test, and commission without risking your hardware.

Sources

Industrial AutomationPLC ProgrammingMechatronicsStructured TextAI Copilots