I have created a benchmark suite (in MPI) to test message latencies in absence and presence of contention. It shows various interesting results for different supercomputers (being published right now). Please download the tarball here.

The plots provided here were generated by running Matt Reilly's benchmarks from his blog on ANL's Blue Gene/P, ORNL's XT4 and NCSA's Abe on 512 nodes, 512 nodes and 512 cores respectively.