EPYC / Skylake vs Power9 STREAM Memory bandwidth comparison via Zaius / Barreleye G2

Using stream benchmark for measuring memory bandwidth is a industry standard practice and I followed the same. For the x86 systems, to be unbiased, I picked the ‘Stream Triad’ results from a reputable benchmarking org (Anandtech).

Power9 CPU Config used for STREAM testing:

root@ubuntu:/home/ubuntu# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 176
Thread(s) per core: 4
Core(s) per socket: 22
Socket(s): 2
NUMA node(s): 2
Model: 2.2 (pvr 004e 1202)
Model name: POWER9, altivec supported

Memory Config used for STREAM testing:

16x   16GiB RDIMM DDR4 2666 MHz (0.4ns)

Theoretical Memory bandwidth:

Theoretical Memory Bandwidth Calculation on Barreleye G2:

=8(ch)*8(transaction_to_byte)*2.666(GHz)*2(socket)

= 8*8*2.666*2 = 341.248 GB/s

Compiler and run instructions for measurement:

wget http://www.cs.virginia.edu/stream/FTP/Code/stream.c

gcc -m64 -O3 -fopenmp -DSTREAM_ARRAY_SIZE=536895856 -DNTIMES=20 -mcmodel=large stream.c -o stream

OMP_NUM_THREADS=44 GOMP_CPU_AFFINITY=0-175:4 ./stream

Results:

Stream Application Barreleye G2 – 2 x22 core (2400 MHz)

gcc

Barreleye G2 2x 22 core (2666 MHz)

gcc

AMD EPYC 32c 7601 (Anandtech)      2x Intel Skylake 8176 (Anandtech)
Stream Copy (MB/s) 217909.8 241641.7
Stream Add (MB/s) 240561.6 253784
Stream Scale (MB/s) 245069.7 268929.6
Stream Triad (MB/s) 247078.8 270000.4 207000 165000

 

Pictorial Representation of results:

Screen Shot 2018-07-19 at 9.41.34 AM.png

OS support for Linux on Power (Little Endian)

OpenPOWER movement is gaining pace. Linux community has been overwhelmingly supportive of Little Indian port of Power (PPC64LE).  As a result, most of the leading Linux server distributions have built server images in the past couple of years.

Through this post I want to aggregate the installer links to server install images for PPC64LE. Where available I also listed cloud images used to boot up VM’s / (OpenStack guests).

OS Hypervisor Install image Guest Installer Image
Fedora 23 Refer to Link Link
Fedora 22 Refer to Link Link
Fedora 24 Refer to Link Link
Ubuntu 15.10 Refer to Link Link
Ubuntu 15.04 Refer to Link Link
Ubuntu  14.04 LTS Refer to Link Link
Centos 7.2 Refer to Link NA (But you can use init from RHEL)
OpenSUSE 13.2 Refer to Link Link
Debian 8 Refer to Link NA
PowerKVM 3.1 Refer to Link NA
Ubuntu 16.04 Refer to Link Link
RHEL 7.2 Refer to Link (Needs Login) Link
SLES 12 Refer to Link (Needs Login) Link

Hopefully this table helps the audience who are just getting started on power servers, to finish a quick install.

Update Model /Serial number of OpenPOWER server

I’ve found the necessity of updating the Model / Serial numbers of our openPOWER servers at Rackspace. And here’s how I did it:

Using LC system VPD update tool
The VPD update tool is used to make VPD modifications to POWER Systems from the host OS of the machine.

For Ubuntu distributions:
ophtk-lht_20160128-1.ub14.04_ppc64el.deb

For RHEL distributions:
ophtk-lht-20160128-1.el7.ppc64le.rpm

Ubuntu
  • To install, run the following:
  • apt-get install libxml2-utils
  • sudo dpkg -i <deb downloaded above>
  • If you encounter a dependency error when installing the tool, run the following to install dependencies and complete the install:
  • sudo apt-get -f install
RHEL
  • To install, run the following as root:
  • yum install ./<rpm downloaded above>
Execution
  • To update the machine serial number:
  • sudo /opt/openpower/lht/bin/updateserial <0000000000000000>
  • Note: <0000000000000000> to be replaced with your Serial Number
  • To update the machine model:
  • sudo /opt/openpower/lht/bin/updatemodel <8335-GTA>
  • Note: <8335-GTA> to be replaced with your model