The delay of the original publisher and the receiver node is calculated by subtracting the current time of the receiver node from the time that is inside the payload of the message, like the following:

Publishing function:

let now = getTime()
let timestampNs = now.toUnix().int64 * 1_000_000_000 + times.nanosecond(now).int64
let msgId = uint64(msg)

var payload: seq[byte]
payload.add(toBytesLE(uint64(timestampNs)))
payload.add(toBytesLE(msgId))
payload.add(newSeq[byte](msg_size - 16))  # Fill the rest with padding

info "Publishing message", msgId = msgId, timestamp = timestampNs

doAssert((await gossipSub.publish("test", payload, useCustomConn = true)) > 0)

Message handler:

  proc messageHandler(topic: string, data: seq[byte]) {.async.} =
    if data.len < 16:
      warn "Message too short"
      return

    let
      timestampNs = uint64.fromBytesLE(data[0 ..< 8])
      msgId = uint64.fromBytesLE(data[8 ..< 16])
      sentMoment = nanoseconds(int64(timestampNs))
      sentNanosecs = nanoseconds(sentMoment - seconds(sentMoment.seconds))
      sentDate = initTime(sentMoment.seconds, sentNanosecs)
      recvTime = getTime()
      delay = recvTime - sentDate

    info "Received message", msgId = msgId, sentAt = timestampNs, delayMs = delay.inMilliseconds()

If we order the parsed data like:

df.sort_values(by=['msg_id', 'delayMs'])

We can observe that the first node that is reporting receiving the message is pod-109, which is not part of the mix net, as only pods from 0 to 9 are part of it.

pod-0 is the original publisher, as in the nim script we have enabled triggerSelf = true, of course the delay of receiving the message from itself is zero.

Test in same K8s worker:

All nodes were set up in ruby-k8s-w15 for this example.

We introduced 10 messages, with 10 nodes that will be connected to all other nodes, and we will have 4 nodes with mix (from 0 to 3). Yaml example:

- name: MESSAGES
  value: "10"
- name: NODES
  value: "10"
- name: MSGRATE
  value: "1000"
- name: MSGSIZE
  value: "100"
- name: PUBLISHERS
  value: "4"
- name: CONNECTTO
  value: "9"

Analyzing the logs, we observed that the first time a message appear in some cases are not nodes from mix. ie: