Archives

Introduction to Precision Time Protocol (IEEE 1588)

This entry is part 1 of 2 in the series Precision Time Protocol

This is the first part of a multi-part series that will cover the basic concepts, terminology, behaviours, and applications of PTP.

PTP Goals

PTP was originally defined in the IEEE 1588-2002 standard, officially entitled “Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems” and published in 2002. In 2008, IEEE 1588-2008 was released as a revised standard; also known as PTP Version 2 [PTPv2], it improves accuracy, precision and robustness but is not backward compatible with the original 2002 version.

IEEE 1588 was designed to fill a niche that neither NTP nor GPS could fulfill by solving two problems:

  1. Local systems needing clock accuracy in the sub-microsecond range
  2. Systems needing synchronisation in areas that cannot afford the cost of a GPS receiver at each node, or that cannot receive GPS signals (consider an underground or windowless datacenter)

The origins of IEEE 1588 development come from test and measurement (instrumentation and data acquisition), industrial automation or power generation, and received strong support from the automation industry leading to the adoption of the IEEE 1588-2002 standard by the Industrial Electro-technical Commission [IEC] as IEEE/IEC 61588-2004.

Specific application domains and targets had driven first release of IEEE 1588 work. Applications (e.g. data acquisition, military instrumentation, industrial robots and high-speed printers) and their targets, had sufficient shared objective to define common criteria:

  • Achievable sub-microsecond accuracy (if required).
  • Operation over control localised areas, such as local area network [LAN], with no intent to cover Internet or wide area networks, but supporting a PTP system.
  • Administration-free operation allowing “plug-and-play” deployment of IEEE 1588 clocks
  • Designed for minimal resource requirement in devices supporting PTP (clock filtering or averaging might not be necessary).

Currently one of the main industries using PTP is the High-Frequency Trading [HFT], and financial sector, to ensure highly accurate timing and synchronisation for their market feeds and trading servers.

When reading the spec

Just to note, that as an IEEE spec, 1588 can be far more difficult to read than IETF/RFC documents. A key thing to note in this particular spec is that IEEE 1588 also specifies “the node, system, and communications properties necessary to support PTP” and proposes models of devices, called “clocks” that support the protocol and aim to help and enhance the transfer of packet-based timing references. In IEEE 1588 terms, a PTP system is a set of IEEE 1588 clocks with specialised functions to support the protocol, PTP.

For instance, specifications of an IEEE 1588 clock include some material, such as a port, that is not part of the protocol itself. However, the messages sent and received by each port of an IEEE 1588 clock and the resulting behaviour of that clock is part of the protocol. Nonetheless, the standard does not define any performance objective for transport network device or clock.

To simplify – a PTP clock is a device that runs the PTP protocol, and has one or more network interfaces

Some PTP Essentials

This entry is part 2 of 2 in the series Precision Time Protocol

IEEE 1588 aims at creating a master-slave timing distribution model across PTP clocks. In IEEE 1588, a clock is a node that implements PTP. In that node, a “PTP port” is a physical, logical, or virtual interface supporting PTP and handling PTP communications. To define the state or function of the port, IEEE 1588 defines state machines as part of its Best Master Clock Algorithm [BMCA] process.

Best Master Clock Algorithm

The BMCA actually has two key functions in a clock. Based on the information received on its PTP Port(s), the BMCA of a clock will:

  • define the master-slave hierarchy of the PTP distribution
  • select the best master (the clock source that it will synchronise to)

NOTE: Be aware of the distinction between the port state and the master clock selection, though both are handled as part of BMCA.

The BMCA is designed in such a way that a local clock determines which of all the clocks (including itself) is the “best”, then it decides on the state of all the ports (master/slave/passive). Clocks do not negotiate who should be master or slave – instead each computes the state of it’s own ports. The algorithm, by design, avoids configurations with two masters, no masters, or ones that thrash.

 

PTP ports can be in 1 of 3 states (the above diagram only considers GM, Slave, and intermediate port states. The role and type of the intermediate clocks isn’t considered yet, simply assume that they run the BMCA);

  1. Master – providing timing information to downstream clocks
  2. Slave – learning timing from an upstream (master) clock
  3. Passive – preventing a loop in the PTP topology

The master port of a clock transmits its packet-based clock signal in the form of PTP messages to the slave port or another clock. Slave ports allow a clock to synchronise with the root of the hierarchy, the ultimate master clock e.g. the Grandmaster [GM] clock. A passive port prevents timing loops or conflicting master ports in the same hierarchy, when there are multiple valid transport network paths (consider an L3 network – no ports blocked due to STP). The state of the port, and the selected clock source (best master or master port) can change over time.

In a steady state network, the BMCA processed by the clocks establishes a hierarchy up to the root, i.e. a GM, and ultimately to slave clocks, the leaves of the hierarchy.

The BMCA will avoid creating timing loops through valid transport network paths e.g. ports that are not blocked by spanning tree. In the case of any changes e.g. device or link failure, the BMCA might have to re-establish the hierarchy and/or reselect the master.

How do we select the “best” clock?

The best master clock algorithm compares one clock to another by comparing datasets (my dataset, vs current best dataset, vs. the dataset just received) that represent those clocks. The table below shows the parameters used to compare any two PTP clocks.

GM Priority 1 Configurable
GM Class Like NTP Stratum
GM Accuracy Atomic Clock, GPS, NTP, Hand set, Internal Oscillator, etc.
GM  offsetScaledLogVariance Provided info from GM regarding PTP variance due to temp, etc.
GM Priority 2 Configurable
GM Identity Usually based on the MAC Address
Steps Removed Number of hops to GM. 0 for local clock
Identity of Senders Identity of parent clock. Same as self for single clock
Identity of Receiver Own Identity
Port number of the receiver Own port number

GM selection can be manipulated using Priority1 and Priority2 values. Typically a true Grand Master device (Meinberg, etc.) will have a low Priority1, with the Priority2 value used to select a preferred active Grand Master device.

We would expect devices with a stable clock (for example the -CL variant of Arista’s 7150s), close to the GMs in the hierarchy to have an intermediate Priority1 and 2 values e.g. 127 – this way they would only become the GM if both “true” GM devices fail.

Slave devices (servers, etc.) would have the highest Priority value configured e.g. 255 to ensure that they never become the GM for the timing domain.

PTP Domains

IEEE 1588 defines the idea of a PTP Domain. These domains can be used for different purposes, e.g. technical or administrative, at the convenience of the user.  The standard defines the protocol and rules but does not define usage. A PTP domain corresponds to one GM delivering its clock signal over the hierarchy established for that domain. Simply put, a PTP domain corresponds to a unique hierarchy of two or more clocks synchronised to the same GM.

The spec allows a clock device to support multiple domains, but a PTP port within that device can only be part of a single domain. Logically, because a PTP communication path is established between two PTP ports, a PTP path belongs to one PTP domain and can never span multiple PTP domains.

A transport network supports multiple PTP domains and a physical network interface can convey multiple PTP paths depending on vendor implementation. Remember, a PTP port is a logical description that can be translated into a physical or virtual interface in a clock implementation. The specification allows for a PTP port to be assigned to a physical port, a VLAN or a loopback interface.

A network or a clock device can support multiple domains on the condition that each domain remains separate. This separation can be physical or logical, as distinct communication networks, VLANs or clocks, and could lead to more complex and expensive design implementation, particularly for maintaining different timescales in single clocks.