Network Time Protocol (NTP) synchronizes timekeeping among a set of distributed time servers and clients. This synchronization allows events to be correlated when system logs are created and other time-specific events occur.
Clock Skew
Causes of Clock Skew could include:
- No running NTP services
- Not properly configured NTP services[1]
- NTP attack[2,7-9]
- Some of which result in shifting time on NTP clients
- Another threat consideration is a malicious insider, who could modify system time in attempts to hide events or manipulate time sensitive transactions.
- Network is congested or lossy[3]
NTP—How Does It Work?
NTP uses the User Datagram Protocol (UDP) as its transport protocol. All NTP communication uses
Coordinated Universal Time (UTC). An NTP network usually receives its time from an authoritative time
source, such as a radio clock or an atomic clock attached to a time server. NTP distributes this time across
the network.
NTP is extremely efficient; no more than one packet per minute is necessary to synchronize two
machines to within a millisecond of each other.
Stratum
NTP uses the concept of a “stratum” to describe how many NTP “hops” away a machine is from an authoritative
time source. A “stratum 1” time server typically has an authoritative time source (such as a radio or atomic
clock, or a GPS time source) directly attached, a “stratum 2” time server receives its time via NTP from a
“stratum 1” time server, and so on.
Was Your NTP Service Properly Configured?
- Application logic
- Scheduled jobs
- Logging
- If the system time is off, log forensics and log correlation of security events across systems becomes a nightmare
and this is especially true for virtual machine based deployments.
How to Sync the Clock on VMs?
If you use Red Hat Enterprise Linux, here are some existing knowledge base documents on how to sync the clock on VMs, such as:
- Best practices for accurate timekeeping for Red Hat Enterprise Linux running on Red Hat Virtualization
- How to troubleshoot NTP issues
- Understanding Red Hat Enterprise Linux System Clocks And Time Protocol Implementations
How to check NTP is working?
You can use the below commands to check:
- ntpq — standard NTP query program
- ntpstat — show network time synchronisation status
- timedatectl — show or set info about ntp using systemd
ntpq
The ntpq utility program is used to monitor NTP daemon ntpd operations and determine performance.
$ ntpq
$ ntpq
ntpq> pe remote refid st t when poll reach delay offset jitter ===================================================================== -isipc6.cairn.ne .GPS1. 1 u 18 64 377 65.59 2 -5.891 0.044 +saicpc-isiepc2. pogo.udel.edu 2 u 241 128 370 10.477 -0.117 0.067 +uclpc.cairn.net pogo.udel.edu 2 u 37 64 177 212.111 -0.551 0.187 *pogo.udel.edu .GPS1. 1 u 95 128 377 0.607 0.123 0.027
- *
- The tattletale symbol at the left margin displays the synchronization status of each peer. The currently selected peer is marked *, while additional peers designated acceptable for synchronization, but not currently selected, are marked +.
- Peers marked * and + are included in the weighted average computation to set the local clock; the data produced by peers marked with other symbols are discarded. See ntpq for the meaning of these symbols.
- remote
- Correspond to the server and peer entries listed in the configuration file; however, the DNS names might not agree if the names listed are not the canonical DNS names.
- refid
- Shows the current source of synchronization
- st
- Reveals the stratum
- t
- The type (u = unicast, m = multicast, l = local, - = don't know)
- when (in secs)
- Shows the time since the peer was last heard in seconds
- poll (in secs)
- The poll interval
- reach
- Shows the status of the reachability register (see RFC-1305) in octal.
- delay (in ms)
- Show the latest round-trip delay
- offset (in ms)
- Show the latest offset
- Offset generally refers to the difference in time between an external timing reference and time on a local machine.
- The greater the offset, the more inaccurate the timing source is. Synchronized NTP servers will generally have a low offset.
- jitter (in ms)
- Show the latest jitter (or estimated error) in milliseconds
- The jitter associated with a timing reference indicates the magnitude of variance, or dispersion, of the signal. Different timing references have different amounts of jitter. The more accurate a timing reference, the lower the jitter value.
- Note that in NTP Version 4 what used to be the dispersion column has been replaced by the jitter column.
To avoid possible distractions due to name resolution problems, run the ntpq program using the -n switch.
- -n
- Output all host addresses in dotted-quad numeric format rather than converting to the canonical host names.
# ntpq -np
remote refid st t when poll reach delay offset jitter
==============================================================================
*xxx.254.169.yyy 192.168.0.151 2 u 141 1024 377 0.545 0.066 0.131
ntpstat
ntpstat is a script which prints a brief summary of the system clock's synchronization status when the ntpd or chronyd daemon is running.
# ntpstat
synchronised to NTP server (xxx.254.169.yyy) at stratum 3
time correct to within 56 ms
polling server every 1024 s
# echo $?
0
- 0 – Clock is synchronized
- 1 – Clock is not synchronized
- 2 – If clock state is indefinite or questionable, for example if ntpd is not contactable
How "time correct to within 56 ms" was calculated?
distance=$(echo "$delay $disp" | awk '{ printf "%.3f", $1 / 2.0 + $2 }')
if [ -n "$distance" ]; then
printf " time correct to within %.0f ms" "$distance"
Therefore, distance = (delay / 2 + dispersion)
and "time correct to within 56 ms" was printeddelay = 0.649
dispersion = 55.480
Raw Data
# ntpq -c rv
associd=0 status=0615 leap_none, sync_ntp, 1 event, clock_sync,
version="ntpd 4.2.6p5@1.2349-o Tue Jun 23 15:14:56 UTC 2020 (1)",
processor="x86_64", system="Linux/4.14.35-1902.10.8.el7uek.x86_64",
leap=00, stratum=3, precision=-24, rootdelay=0.649, rootdisp=55.480,
refid=xxx.254.169.yyy,
reftime=e2f66690.7a8a26cd Sun, Aug 30 2020 17:55:28.478,
clock=e2f66814.1623cd2d Sun, Aug 30 2020 18:01:56.086, peer=35146,
tc=10, mintc=3, offset=-0.116, frequency=13.462, sys_jitter=0.000,
clk_jitter=0.057, clk_wander=0.009
# ntpstat
synchronised to NTP server (xxx.254.169.yyy) at stratum 3
time correct to within 56 ms
polling server every 1024 s
timedatectl
Run the following command to check the service status:
# timedatectl status
Local time: Sun 2020-08-30 17:12:19 UTC
Universal time: Sun 2020-08-30 17:12:19 UTC
RTC time: Sun 2020-08-30 17:12:20
Time zone: UTC (UTC, +0000)
NTP enabled: yes
NTP synchronized: yes
RTC in local TZ: no
DST active: n/a
How can I see the Time Difference between Client and Server?
Normally ntpd maintains an estimate of the time offset. To inspect these offsets, you can use the following commands:[5]
- ntpq -np will display the offsets for each reachable server in milliseconds
- ntpdc -c loopinfo will display the combined offset in seconds, as seen at the last poll. If supported, ntpdc -c kerninfo will display the current remaining correction, just as ntptime does.
References
- Avoiding clock drift on VMs
- System Clock Skewed? Read this Post, Especially if You Don’t Have Time
- NTP Debugging Techniques
- All you need to know about sosreport tool
- Troubleshooting - ntp.org
- Check the status of NTP
- NTP server misuse and abuse
- NTP Amplification
- Attacking the Network Time Protocol
- Implementing NTP
- What does "ntpd -x" mean? Can I run NTP in slewmode?
- Selecting Offsite NTP Servers
- How to verify NTP is working Or not (Check Status of NTP)
- Restrict NTP mode 6 queries to avoid NTP amplification attacks
