# Wrong time after restart: TLS errors and misleading log chronology

Diagnose wrong time after restart: TLS errors and misleading log chronology. Evidence to collect, a worked operational example and recovery checks.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/clock-skew-tls-telemetry/
Language: en

An incorrect node clock can cause TLS not-yet-valid errors and misleading log chronology. Compare current time with certificate validity first. For telemetry, distinguish the source Timestamp from the Collector's ObservedTimestamp: they describe different moments, not duplicate measurements of one instant.

## Identify the validating clock

Record UTC time from the node, container and a trusted control source within authorized access. Containers generally depend on host time, while applications may format time zones separately. An incorrect displayed time zone does not necessarily mean the system clock is wrong. Preserve raw timestamps and units. TLS validation depends on the checking time relative to notBefore/notAfter, not the operator interface's localized display.


[Official reference 1](https://www.rfc-editor.org/rfc/rfc5280.html).

## Separate validity and identity

RFC 5280 defines certificate validity from notBefore through notAfter, inclusive. Compare that interval with the client's actual clock, then separately inspect chain trust and service name. Do not automatically reissue a certificate after every not-yet-valid error; a restored or newly started node may have incorrect time. Conversely, correct time cannot repair a genuinely expired certificate. Retain the fingerprint so before/after checks use the same leaf.


[Official reference 2](https://opentelemetry.io/docs/specs/otel/logs/data-model/).

## Examine two logging times

OpenTelemetry Timestamp describes the event according to the source clock. ObservedTimestamp describes when the collection system observed it. Suppose a worker clock is wrong by 5 minutes while the Collector's clock is correct. Sorting by Timestamp changes apparent cross-node order. Sorting by ObservedTimestamp shows arrival order but cannot reconstruct causality through buffering and network delays. The 5-minute difference is illustrative, not an acceptable operating threshold.

| Field | Investigative meaning |
| --- | --- |
| System UTC | Basis for the client's time check |
| Certificate validity | Permitted certificate interval |
| Timestamp | Event source clock |
| ObservedTimestamp | Observation-point clock |

## Correct the time source

Inspect the node's intended synchronization mechanism and its condition after reboot or restore. Avoid arbitrary production clock changes without assessing effects on processes and observability. Repair display conversion separately when raw timestamps are correct. State which layer changed: host synchronization, application formatting or Collector processing. These are different operations and cannot support one unqualified claim that distributed time is repaired.

## Preserve an interpretable sequence

Repeat TLS access with the same client and compare both event timelines. Keep raw records, measured offset and correction time. If the clock jumped, mark the boundary across which historical duration calculations are unreliable. The artifact should make old events interpretable without silently rewriting them. Recovery means correct new validation and a clear account of earlier records, rather than merely producing an attractive dashboard ordering. Keep the distinction when correlating incidents from several nodes.

Related operational reading: [gpu owner network outage log](/journal/gpu-owner-network-outage-log/); [gpu monitoring scorecard](/journal/gpu-monitoring-scorecard/).