Timestamps: [2024-07-23T15:40:00, 2024-07-23T15:47:00]

src.mesh_analysis.readers.victoria_reader - INFO - Reading <https://vmselect.riff.cc/select/logsql/query>
src.mesh_analysis.readers.victoria_reader - INFO - Fetched 5822 messages
src.mesh_analysis.readers.victoria_reader - INFO - Fetched 38400 messages
src.mesh_analysis.tracers.waku_tracer - INFO - Nº of Peers: 100
src.mesh_analysis.tracers.waku_tracer - INFO - Nº of unique messages: 59
src.mesh_analysis.tracers.waku_tracer - WARNING - Some peers missed messages: ['16U*4SY3zo', '16U*gPN1W8']
src.mesh_analysis.tracers.waku_tracer - WARNING - Missing messages: ['0x036e2858e47f5c829b023c6c7646b98a9be30aa721e88e1870244006300ae6f6', '0x17d5d207003d54591f1c260d405551f3ce9ff8e7df51a12c5eab156cb3902606', '0x18dfb75b9647aab71ff6d671947cd460404eb4905b5d043c0850c22b13a3c23e', '0x1a80d31fd13d2bc1b6484ca8f077d2de7305e0fa293277b24734bb0c8f508489', '0x200706ad0b666f828477c89a9af0a6b204f366070479e8f3ad6df84413aa39e3', '0x22b89c823d61726384e3b841407d9eeb6ed389a1c2d9aa4e9d7950a2e901b9f4', '0x2cfa734d1c785719785719799c939e07a415af11b1636b9b053dffacd2df3814', '0x2ff8cd8771de559ef0f436810f9a038b1cb0cd32df320e69a1df19d013a9ee60', '0x33a2367fbfe4e0b3b0804867cdc21fa365fb7f405b3345aa73eaf7ce216af82f', '0x47a34b3862aaf868e3de0dd10fe9cf55862be302f0dc25439a465381ec850bbd', '0x4b5fb953538f886dee25565f7075ee2f6657c2b668ef34402a9d3a18291bcdf8', '0x5478873a8709abe1251df09402b328fb3f42cdb08252d220bde74db88bb8e7d4', '0x5833e4ec2aa0fab4234d330e42fcee151e625fdb3661af5b1f612eb7dd4e0fdd', '0x70cd966ca155981ec8981bf9552c8c24afcfe8dba226c8aced20154950ff8fce', '0x724d29a788beefebc8c12e470b7c07c208cf457b1cbb5490b3162a38e36057c8', '0x7c6a9bc01a83c534a793c6b9d6f00dffa658e938273aeb8806df0841eef639c7', '0x82962d48da3e28c70fa071ed9471c8116f2d2892231ad4c0ad2094fbf1329aed', '0x8b8f167e7ac069eead6ca4421493e3acb07c5eb5b49298ad978856ee740dfe13', '0x8d93a4601694330e59167d59198bb9c52e20128e23fd5dc7c5cfcddc3c8a798b', '0x902c9730983a0c5af4ebc499a4e201e6940cb428e58abcc09bda88eff77ac888', '0x932dc374cffffbfa504a418a3da08ae7b218f8f0ce61e25bf583dce46b597b06', '0x95a67d2da92b3746c1caf76d481c9d0e512fedd4fd4b9d84e26f6bef732367ae', '0x9f565b3dc02803cd5674841bf1ad02ea231dc5bf848b7c985ed51ff81a05a146', '0xa25c665319b950e830a5e6ceddafdd915a5021431a1af60c856e9b00a38157e6', '0xa3592eada7a6304cd3973bb316b05b619b558686e89774efb4e9ed1a551fe600', '0xa44e716b4d9289c3366262b745995f5aed152b73a6b8faf75adb5e3dcaa3bda3', '0xa93f90ae2fba119aa1fafb7c6e1af3e8edeb75866b2008b2c5b0fe10970f26d0', '0xb219d4bb12ac9ff12978051f3272202c722ff6d974f40cdcb49e94346bdd2422', '0xbd383826e0b4c314f607f6c8da9f916a9bbe8d844c3317ed9b4967be0e17faf2', '0xc7e048f1cd9e5f11695e4a5b2bc897de0f7c9c28ec895fd056ddf3d70277558f', '0xc909d7959961cc2f4fedd18be66488c756ee67c82e89d72957c3ef968e24d97a', '0xe0d05fce2129f5e937bf21e10f3e1dd59fff6d55d5f9f73844b22579cacb99e0', '0xe0d34cb65a0f2c756fc7ee8565e73aaa53b9ee3e0ef42a5c4b04890e4b1ded8f', '0xe507ea1f0112e91c628fe8baaed87e38dcff82e8ea7d57915d0c481d9365bae4', '0xe74570f958b6f3139a7c9c5bc01399bf4cd8b1b609722865fc2f29a46d0d8d0a', '0xe82fa9d374215bc10a5040cab60f3cd129abe6e233dad4d9ec44c64f4d59c77b', '0xebfb1ddcdc0faaf4f778e64f7f8b21bf60ee2d7888551b1d251bdd97b3b2769a', '0xece583edc0041407cbd247e7072e14cb6e9a54845de6cfa5075568f320377afc', '0xfe2da19a30b82bd3d183f4e5a881810e56c4b9f6bddbf81d37e46173a9fa198a']

Node-49

gPN1W8

gPN1W8.txt

Two biggest time jumps in log:

2024-07-23 15:42:32.032000+00:00 to 2024-07-23 15:43:23.027000+00:00 -> 0:00:50.995000
2024-07-23 15:41:44.027000+00:00 to 2024-07-23 15:41:45.031000+00:00 -> 0:00:01.004000

Log line with the issue:

TRC 2024-07-23 15:42:32.032+00:00 waiting for data                           topics="libp2p pubsubpeer" tid=7 file=pubsubpeer.nim:196 conn=16U*Zyf1dN:669fcf4512dc71b65037e1a6 peer=16U*Zyf1dN closed=false
TRC 2024-07-23 15:43:23.027+00:00 running heartbeat                          topics="libp2p gossipsub" tid=7 file=behavior.nim:775 instance=140219024908368

Nodes-81

4SY3zo.csv

4SY3zo.txt

Two biggest time jumps in log:

2024-07-23 15:42:32.021000+00:00 to 2024-07-23 15:43:23.019000+00:00 -> 0:00:50.998000
2024-07-23 15:41:44.020000+00:00 to 2024-07-23 15:41:45.025000+00:00 -> 0:00:01.005000

Log line with the issue:

TRC 2024-07-23 15:42:32.021+00:00 sending msg to peer                        topics="libp2p pubsubpeer" tid=7 file=pubsubpeer.nim:484 peer=16U*9qPNke rpcMsg="(subscriptions: @[], messages: @[], control: (ihave: @[(topic: \\"/waku/...rs/2/0\\", messageIDs: @[\\"f5b56dcc4d7b...6ee6542c0b25\\", \\"9fc8ea4f66cd...0f6fd53b22de\\", \\"1c9f39238410...1e410ae22fa7\\"])], iwant: @[], graft: @[], prune: @[]))"
TRC 2024-07-23 15:43:23.019+00:00 running heartbeat                          topics="libp2p gossipsub" tid=7 file=behavior.nim:775 instance=140312026923088

Sender logs

senders.zip