How the experiment is done

Constant experiment parameters

Experiments description:

Current issues:

After doing a rebuild in the lab, we are facing a discrepancy in bandwidth measurements.

Whereas in the previous version of the lab the results were very stable:

image.png

We started seeing variance on the new lab version (v1.11) was run in this new lab. After some investigation, we decided to re-run all the versions, and we obtained this:

image.png

The results are much sparse, and some deployments were even failing to run all requested nodes. We are currently investigating this.

Also, as a note, Quic GC memory usage looks weird. Noting this here to compare once the lab is more stable:

image.png


We decided to update only one machine (Al), and run a small subset of the experiment only there. Then, to compare the results, we ran another batch in a non-upgraded machine (Ambellina).

Al

Al

Ambellina

Ambellina

We obtained the opposite that we expected, Ambellina performs a bit more stable than Al.