In order to analyze if all nodes received all messages, we are using an image that logs the received and sent relay message events. The images are based on waku v0.30 and waku v0.31:
soutullostatus/nwaku-jq-curl:v0.30.1-msg-log-final-short (git commit hash: v0.30.1-9-g96d0b4). Message sent/received logging over 0.30.1 with short peer_ids.
Timestamp: [2024-07-23T15:40:00, 2024-07-23T15:47:00]
soutullostatus/nwaku-jq-curl:v0.31.0-msg-log (git commit hash: v0.31.0-rc.1-11-g12db97). Message sent/received logging over 0.31.0 with short peer_ids
soutullostatus/nwaku-jq-curl:v0.31.0-msg-log-extra-2 (git commit hash: v0.31.0-rc.1-13-gc716d4)
[2024-07-26T06:51:00, 2024-07-26T06:59:00]
[mplex version failing]
Over these images, we are only adding jq and curl for some checks we need to do from Kubernetes. This is why we don’t directly use the images from harbor/wakuorg.
We are deploying 3 bootstrap nodes and 100 waku nodes. Bootstrap nodes have relay disabled, as they only serve as bootstraping for discv5.
nWaku parameters:
/usr/bin/wakunode --relay=true --max-connections=150 --rest=true --rest-admin=true --rest-private=true --rest-address=0.0.0.0 --discv5-discovery=true --discv5-enr-auto-update=True --log-level=TRACE --metrics-server=True --metrics-server-address=0.0.0.0 --discv5-bootstrap-node=$ENR1 --discv5-bootstrap-node=$ENR2 --discv5-bootstrap-node=$ENR3 --nat=extip:${IP} --cluster-id=2 --pubsub-topic="/waku/2/rs/2/0"
Bootstrap parameters:
/usr/bin/wakunode --relay=false --rest=true --rest-address=0.0.0.0 --max-connections=500 --discv5-discovery=true --discv5-enr-auto-update=True --log-level=INFO --metrics-server=True --metrics-server-address=0.0.0.0 --nat=extip:$IP --cluster-id=2 --pubsub-topic="/waku/2/rs/2/0"
We are injecting 60 messages at 1 message per second rate:
/app/traffic.py --messages=60 --msg-size-kbytes=1 --delay-seconds=1 --pubsub-topic="/waku/2/rs/2/0" --debug
Once the injection is done, we are waiting 20 seconds. This should be more than enough time for the peers to propagate the last message. After that, the nodes are shut down.
This is a small visual for this few minutes deploy.