NTP—How to Check It Is Working?

Network Time Protocol (NTP) synchronizes timekeeping among a set of distributed time servers and clients. This synchronization allows events to be correlated when system logs are created and other time-specific events occur. 

Clock Skew

Clock skew is a phenomenon in computers in which the same sourced clock signal arrives at different components at different times. The instantaneous difference between the readings of any two clocks is called their skew.

Causes of Clock Skew could include:

• No running NTP services 
• Not properly configured NTP services^[1]
• NTP attack^[2,7-9]
• Some of which result in shifting time on NTP clients
• Another threat consideration is a malicious insider, who could modify system time in attempts to hide events or manipulate time sensitive transactions.
• Network is congested or lossy^[3]

NTP—How Does It Work?

NTP uses the User Datagram Protocol (UDP) as its transport protocol. All NTP communication uses Coordinated Universal Time (UTC). An NTP network usually receives its time from an authoritative time source, such as a radio clock or an atomic clock attached to a time server. NTP distributes this time across the network. 

NTP is extremely efficient; no more than one packet per minute is necessary to synchronize two machines to within a millisecond of each other. 

Stratum

NTP uses the concept of a “stratum” to describe how many NTP “hops” away a machine is from an authoritative time source. A “stratum 1” time server typically has an authoritative time source (such as a radio or atomic clock, or a GPS time source) directly attached, a “stratum 2” time server receives its time via NTP from a “stratum 1” time server, and so on. 

Was Your NTP Service Properly Configured?

Your web server’s system time can keep on slipping far into the future or past if NTP is not properly configured. Having accurate system time is critical for:

• Application logic
• Scheduled jobs 
• Logging
• If the system time is off, log forensics and log correlation of security events across systems becomes a nightmare

and this is especially true for virtual machine based deployments.

How to Sync the Clock on VMs?

If you use Red Hat Enterprise Linux, here are some existing knowledge base documents on how to sync the clock on VMs, such as:

• Best practices for accurate timekeeping for Red Hat Enterprise Linux running on Red Hat Virtualization 
• How to troubleshoot NTP issues
• Understanding Red Hat Enterprise Linux System Clocks And Time Protocol Implementations 

How to check NTP is working?

You can use the below commands to check:

• ntpq — standard NTP query program
• ntpstat — show network time synchronisation status
• timedatectl — show or set info about ntp using systemd

ntpq

The ntpq utility program is used to monitor NTP daemon ntpd operations and determine performance.

$ ntpq

ntpq> pe
     remote      refid       st t when poll reach delay offset jitter
=====================================================================
-isipc6.cairn.ne .GPS1.        1 u  18  64  377  65.59 2 -5.891  0.044
+saicpc-isiepc2. pogo.udel.edu 2 u 241 128  370  10.477 -0.117  0.067
+uclpc.cairn.net pogo.udel.edu 2 u  37  64  177 212.111 -0.551  0.187
*pogo.udel.edu   .GPS1.        1 u  95 128  377   0.607  0.123  0.027

• *
• The tattletale symbol at the left margin displays the synchronization status of each peer. The currently selected peer is marked *, while additional peers designated acceptable for synchronization, but not currently selected, are marked +. 
• Peers marked * and + are included in the weighted average computation to set the local clock; the data produced by peers marked with other symbols are discarded. See ntpq for the meaning of these symbols.
• remote
• Correspond to the server and peer entries listed in the configuration file; however, the DNS names might not agree if the names listed are not the canonical DNS names. 
• refid
• Shows the current source of synchronization
• st
• Reveals the stratum
• t
• The type (u = unicast, m = multicast, l = local, - = don't know)
• when (in secs)
• Shows the time since the peer was last heard in seconds
• poll (in secs)
• The poll interval 
• reach
• Shows the status of the reachability register (see RFC-1305) in octal. 
• delay (in ms)
• Show the latest round-trip delay
• offset (in ms)
• Show the latest offset
• Offset generally refers to the difference in time between an external timing reference and time on a local machine. 
• The greater the offset, the more inaccurate the timing source is. Synchronized NTP servers will generally have a low offset. 
• jitter (in ms)
• Show the latest jitter (or estimated error) in milliseconds
• The jitter associated with a timing reference indicates the magnitude of variance, or dispersion, of the signal. Different timing references have different amounts of jitter. The more accurate a timing reference, the lower the jitter value. 
• Note that in NTP Version 4 what used to be the dispersion column has been replaced by the jitter column.

To avoid possible distractions due to name resolution problems, run the ntpq program using the -n switch.

• -n
• Output all host addresses in dotted-quad numeric format rather than converting to the canonical host names.

# ntpq -np

     remote           refid      st t when poll reach   delay   offset  jitter

==============================================================================

*xxx.254.169.yyy 192.168.0.151    2 u  141 1024  377    0.545    0.066   0.131

ntpstat

ntpstat  is  a  script  which  prints  a brief summary of the system clock's synchronization status when the ntpd or chronyd daemon is running.

# ntpstat

synchronised to NTP server (xxx.254.169.yyy) at stratum 3

   time correct to within 56 ms

   polling server every 1024 s

# echo $?

0

You can also use the exit status (return values) to verify its operations from a shell script or command line itself.  If exit status is

• 0 – Clock is synchronized
• 1 – Clock is not synchronized
• 2 – If clock state is indefinite or questionable, for example if ntpd is not contactable

How "time correct to within 56  ms" was calculated?

The ntp query outputs from ntpd and chronyd are different.  The below discussion is based on ntpd.  

    distance=$(echo "$delay $disp" | awk '{ printf "%.3f", $1 / 2.0 + $2 }')

    if [ -n "$distance" ]; then

        printf "   time correct to within %.0f ms" "$distance"

Therefore, distance = (delay / 2 + dispersion) 

delay = 0.649

dispersion = 55.480

and "time correct to within 56 ms" was printed

Raw Data

# ntpq -c rv

associd=0 status=0615 leap_none, sync_ntp, 1 event, clock_sync,

version="ntpd 4.2.6p5@1.2349-o Tue Jun 23 15:14:56 UTC 2020 (1)",

processor="x86_64", system="Linux/4.14.35-1902.10.8.el7uek.x86_64",

leap=00, stratum=3, precision=-24, rootdelay=0.649, rootdisp=55.480,

refid=xxx.254.169.yyy,

reftime=e2f66690.7a8a26cd  Sun, Aug 30 2020 17:55:28.478,

clock=e2f66814.1623cd2d  Sun, Aug 30 2020 18:01:56.086, peer=35146,

tc=10, mintc=3, offset=-0.116, frequency=13.462, sys_jitter=0.000,

clk_jitter=0.057, clk_wander=0.009

# ntpstat

synchronised to NTP server (xxx.254.169.yyy) at stratum 3

   time correct to within 56 ms

   polling server every 1024 s

timedatectl 

If you are using systemd based system, timedatectl may be used to query and change the system clock and its settings.

Run the following command to check the service status:

# timedatectl status

      Local time: Sun 2020-08-30 17:12:19 UTC

  Universal time: Sun 2020-08-30 17:12:19 UTC

        RTC time: Sun 2020-08-30 17:12:20

       Time zone: UTC (UTC, +0000)

     NTP enabled: yes

NTP synchronized: yes

 RTC in local TZ: no

      DST active: n/a

How can I see the Time Difference between Client and Server?

Normally ntpd maintains an estimate of the time offset. To inspect these offsets, you can use the following commands:^[5]

1. ntpq -np will display the offsets for each reachable server in milliseconds
2. ntpdc -c loopinfo will display the combined offset in seconds, as seen at the last poll. If supported, ntpdc -c kerninfo will display the current remaining correction, just as ntptime does.

The first can be used to check what ntpd thinks the offset and jitter is currently, relative to the preferred/current server, the second can tell you something about the estimated offset/error all the way to the stratum 1 source. 

References

1. Avoiding clock drift on VMs
2. System Clock Skewed? Read this Post, Especially if You Don’t Have Time
3. NTP Debugging Techniques
4. All you need to know about sosreport tool
5. Troubleshooting - ntp.org
6. Check the status of NTP
7. NTP server misuse and abuse
8. NTP Amplification
9. Attacking the Network Time Protocol
10. Implementing NTP
11. What does "ntpd -x" mean? Can I run NTP in slewmode?
12. Selecting Offsite NTP Servers
13. How to verify NTP is working Or not (Check Status of NTP)
14. Restrict NTP mode 6 queries to avoid NTP amplification attacks