Why industrial real-time communication is not just faster TCP

A software engineer looking at PROFINET for the first time often asks why it does not just use TCP. Networks are fast, a request over a local switch takes a fraction of a millisecond, and TCP gives you reliability for free. The answer is that the problem is not speed. It is predictability: knowing that a value arrives within a bound, every time, and knowing quickly and safely when it does not.

The central question of this article: why do some industrial protocols bypass ordinary request/response networking?

It is part of the series that starts with 14 industrial protocols, 5 communication patterns.

Latency, jitter and deadlines

Three terms do most of the work:

• Latency is how long a message takes from sender to receiver.
• Jitter is how much that time varies from one message to the next.
• A deadline is the latest time at which a message is still useful.

A motion controller that closes a loop every millisecond cares little whether the average latency is 50 or 200 microseconds. It cares that no update arrives later than its cycle allows. Average latency is a throughput metric; real-time systems are judged by the worst case.

“Real time” is therefore not one universal guarantee. PROFINET RT, PROFINET IRT, EtherNet/IP Class 1, CANopen with SYNC and IEC 61850 Sampled Values each offer different bounds under different configurations. Protocol mechanisms define the timing contract; they do not by themselves make a general-purpose host meet every deadline. Every one of them can be undermined by the end systems: an operating system that schedules the sending thread late, a network card that coalesces interrupts, a switch that queues behind a burst of other traffic.

Polling at fixed intervals versus cyclic relationships

Polling a device every 10 milliseconds over TCP and running a 10 millisecond cyclic connection look similar on a slide. They are not.

With polling, each cycle is an independent request. If the device is slow, the response is late; if a segment is lost, TCP retransmits after a timeout that is much longer than the cycle; head-of-line blocking delays everything behind it. The client must notice that responses are late, decide what to do, and reconnect if the connection is broken. Nothing in the protocol says that a value older than 30 milliseconds must not be used.

With a cyclic relationship, the deadline is part of the contract. Both sides agree on the cycle when the relationship is established; frames flow every cycle in both directions without requests; a sequence or cycle counter identifies each one; and a watchdog declares the relationship failed when no valid frame arrived for a configured number of cycles. A late frame is not retried: the next one is already due, with newer data.

PROFINET: startup services and cyclic RT frames

PROFINET keeps the two worlds apart.

• Startup and acyclic services use DCE/RPC over UDP: the IO-controller connects to the IO-device, establishes an application relationship (AR) with communication relationships for cyclic data and alarms, writes parameters, and signals the end of parameterization. The device answers ApplicationReady. Records can be read and written later through the same path.
• Cyclic real-time data uses Ethernet frames with their own EtherType, without IP or UDP, sent with VLAN priority. Each frame carries the process data and status defined by one configured I/O communication relationship (IOCR): submodule data with its provider status (IOPS), consumer status (IOCS) saying whether the receiving side can process the data sent the other way, and a cycle counter and data status for the frame. An application relationship can contain several IOCRs, so one frame does not necessarily carry all of a device’s subslots. Cycle times for RT class 1 typically range from 1 ms upwards, configured as a send clock and a reduction ratio.
• Alarms travel as acknowledged acyclic frames on Layer 2.

Bypassing IP shortens the path through the stack and lets switches prioritize frames, but the key benefit is that the cyclic path has nothing that waits: no handshake, no retransmission, no congestion control. IRT, the isochronous class, goes further with a time-scheduled reservation on the wire, and needs hardware support in switches and devices.

EtherNet/IP: explicit TCP services and implicit UDP I/O

EtherNet/IP follows the same split with IP all the way:

• Explicit messaging over TCP port 44818 carries CIP services: reading attributes and tags, configuration, diagnostics, and the Forward_Open that sets up I/O connections. Class 3 connected messaging is still request/response.
• Implicit messaging, Class 1, carries I/O data over UDP port 2222. The originator requests a packet interval (RPI) for each direction; the target produces at that rate, unicast or multicast, with a sequence count. A connection timeout closes the connection when packets stop. It is the RPI multiplied by a timeout multiplier that the originator chooses in the Forward_Open and the target accepts, so it differs per connection rather than being a fixed constant.

Because implicit I/O runs on UDP and IP, it can cross a routed network in principle, and multicast lets several consumers share one producer. In practice, timing still depends on the network being engineered for it: segmented, prioritized and quiet.

CAN: arbitration, priorities and event-driven access

CAN reaches predictability by a different route. There are no addresses: every frame has an identifier, which names its content and also sets its priority. When several nodes start sending at once, they arbitrate bit by bit. A 0 is dominant and overwrites a 1; a node that sends a 1 and reads a 0 back stops sending and becomes a receiver.

Arbitration is non-destructive: the winning frame continues as if nothing happened, and no bandwidth is lost to collisions. Combined with a known bit rate, up to 1 Mbit/s for classical CAN, this lets a designer calculate the worst-case response time of each message. The calculation rests on assumptions: the periods and frame lengths of all higher-priority messages, the bus load, and an error model, since error frames and retransmissions add delay. With those written down, the bound is a real engineering number; without them it is a guess. The ACK slot at the end of a frame tells the sender that at least one node received it correctly; it says nothing about whether the intended consumer did.

CANopen: SYNC and periodic or event-driven PDOs

CANopen uses CAN’s properties to build both cyclic and event-driven exchange:

• A SYNC producer sends a high-priority SYNC message at a fixed period. A transmit PDO’s transmission type decides how it relates to SYNC: type 0 is acyclic synchronous, sent after a SYNC only when an event occurred; types 1 to 240 are cyclic synchronous, sent after every nth SYNC. Receivers apply synchronous receive PDOs at the next SYNC. The network moves in step.
• Event-driven PDOs, types 254 and 255, are sent when the manufacturer-specific or device-profile event occurs, typically a change of data, limited by an inhibit time and backed by an event timer.
• Heartbeats let configured consumers supervise selected producers. A node produces heartbeats at the interval in object 0x1017 and monitors only the producers and timeouts listed in its consumer entries, object 0x1016; supervision is not automatically all-to-all and can be switched off.

Because PDO identifiers carry priority, the designer decides which process data wins under load, which is something no TCP-based protocol lets you express.

GOOSE retransmissions versus Sampled Values

IEC 61850 defines two Layer 2 mechanisms for substations that look alike and behave very differently.

• GOOSE carries state, such as a trip signal or a breaker position. It is sent immediately on a change and then repeated, quickly at first and then at a heartbeat interval, so that a lost frame is replaced within milliseconds. A subscriber invalidates the data if nothing arrives within the time-allowed-to-live in the last frame. Reliability comes from repetition, not acknowledgement. Repetition makes it very likely that subscribers receive the current state; it does not preserve intermediate states that changed again before a subscriber saw them.
• Sampled Values carry measurements, the instantaneous currents and voltages of a merging unit. The common 9-2LE profile for protection sends 80 samples per power cycle, 4,000 per second at 50 Hz and 4,800 at 60 Hz, one ASDU per frame; other profiles and configurations use other rates and group several ASDUs in one frame. A lost sample is never retransmitted; a gap in the sample counter is visible to the subscriber, and the protection function decides what a gap means. Samples from different merging units are aligned by a shared time reference, typically PTP or a pulse-per-second signal, which the stream flags as synchronized or not.

Watchdogs, missed cycles and stale outputs

Every cyclic mechanism ends in the same question: what happens when data stops?

• In PROFINET, the IO-device detects missing frames after a watchdog interval, a configured multiple of the cycle, and applies its configured substitute behaviour to the outputs. The controller marks the device’s inputs as invalid. The provider and consumer status bytes (IOPS and IOCS) say whether data is valid and was consumed; they are not evidence that an actuator physically moved.
• In EtherNet/IP, the connection times out after the RPI times the multiplier, and the target applies its configured behaviour for outputs: clearing them, going to fault values, or holding the last state, which some devices allow indefinitely.
• In CANopen, a missing heartbeat triggers the consumer’s configured reaction, and receive PDO deadline monitoring can flag data that did not arrive in time.
• In GOOSE, data whose time-allowed-to-live has passed is invalid.

The design rule is to tolerate occasional loss and act decisively on sustained loss. A single lost frame should not trip a machine, and the fault reaction for sustained loss is an engineering requirement: decide what each output must do when the relationship fails, configure it in the device, and verify it, rather than assuming the protocol makes it safe. Software that sits next to cyclic I/O needs the same discipline: every value it takes from the I/O image should carry its validity, and every output it writes needs a defined behaviour when the relationship goes away. Time, freshness and ordering picks this up for supervisory data.

Docker networks, routers and the cloud

Layer 2 protocols do not survive the abstractions that software engineers use every day:

• Routers do not forward PROFINET RT frames, GOOSE or Sampled Values; they live in one broadcast domain or VLAN. DCP discovery does not cross a router either.
• Container bridge networks use NAT and virtual switches. They carry TCP and UDP well enough for development, but a container on a default bridge does not see the plant’s Layer 2 traffic, and scheduling inside a busy host adds jitter.
• Cloud services are reachable over IP across routers and the internet. They are the right place for history, analytics and fleet management, and the wrong place for anything with a millisecond deadline.

Where raw sockets, virtual CAN and test networks come in

Working with these protocols in software needs the right access to the network:

• Raw Ethernet access: AF_PACKET sockets on Linux, BPF on macOS and the BSDs, Npcap on Windows. They usually require elevated privileges or capabilities such as CAP_NET_RAW, and a dedicated network interface so that test traffic does not leak into an office network.
• Virtual CAN (vcan and vxcan on Linux) lets CAN and CANopen software run and be tested without hardware, with the caveat that a virtual bus shows encoding and protocol logic, not bit timing, arbitration under electrical conditions or bus-off behaviour.
• Carefully controlled test networks: a Docker network or a separate VLAN with only the devices under test, a capture point for Wireshark, and no production equipment in reach.
• Operating-system scheduling: a general-purpose OS with a garbage-collected runtime can hold a millisecond cycle in many conditions, with occasional outliers that a watchdog of several cycles absorbs. It is not a substitute for a real-time OS or dedicated hardware for motion control or safety functions.

Practical lesson

Real-time communication is about bounded behaviour and freshness, not simply moving messages faster. In software, keep cyclic I/O in its own loop, away from request/response work and garbage-heavy processing; make every value from the I/O image carry its validity; define what outputs do when the relationship fails; and test on networks that reproduce Layer 2 behaviour rather than on a convenient container bridge. Designing a multi-protocol gateway shows where such a loop fits next to the rest of an application.