How to mine Monero (XMR) on Rackspace Barreleye G2 Server (IBM POWER9)

Previous post, I’ve documented process of mining ETH / Ethereum on GPUs on POWER processor based server (called Barreleye ). Barreleye G2 is a open-source hardware server developed by Rackspace / Google.

In this post I explain, how to mine crypto-currency on CPU and still be profitable:

A. You pick the IBM POWER8 / 9 processors because of vector instructions supported on this CPU, helping the hashing algorithms (Specifically cryptonight algo – basis for Monero). (In your leisure, read this cryptonight standard that explains the basis of the algorithm itself )

B. You mine #MONERO – XMR because of its relative GPU / ASIC resistance. Monero has become one of the most popular cryptocurrencies due to its excellent privacy. Its website describes it as “secure, private, untraceable currency.”

For my guide I tested mining speeds on Rackspace server called : Barreleye G2 .
Example Speeds:  Hash-rate on Barreleye server running 2 POWER8 /POWER9 (10 core chip) (~5 KH/s) —- Equivalent to 9 x GeForce 1070 Mining hash-rate (475 H/s)

 

Currently verified operating systems are :

A. 16.04

B. CentOS 7.x

Here are the steps to follow for mining Monero on Barreleye G2 on Ubuntu 16.04. The only changes for CentOS will be in installing AT10 (look below for what AT10 is) and dependency package names (usually different for debain vs RHEL based repos):

Before starting make sure your SMT level is at 4 and not 8 (4-way multi-threading yields the best results – Discovery through experimentation)

ppc64_cpu –smt=4

1. Clone the xmr-stak-power repository for POWER based on xmr-stack-cpu and cd into source directory

git clone https://github.com/agangidi53/xmr-stak-power

cd xmr-stak-power

      2. Building a binary for mining on POWER needs gcc 6.3.1 which is available for public via Advanced toolchain 10. Install AT10.0 (Follow instructions below)

2a. Change Source list to add AT10 path

nano /etc/apt/sources.list

add “deb ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu xenial at10.0” at the end of the file

apt-get update

2b. Install the 4 AT10.0 packages and add them to the PATH:

apt-get install advance-toolchain-at10.0-runtime \
advance-toolchain-at10.0-devel \
advance-toolchain-at10.0-perf \
advance-toolchain-at10.0-mcore-libs
export PATH=/opt/at10.0/bin:$PATH

3. Install dependencies before building the mining binary:

apt install libmicrohttpd-dev -y
apt install libhwloc-dev -y
apt install libssl-dev -y

4. Configure, Make and Build the XMR-STAK-POWER binary

export LD_LIBRARY_PATH=’/usr/lib/powerpc64le-linux-gnu/’

cmake

make

make install

Binary will be built and saved as <source-directory>/bin/xmr-stak/power

5. Create a Monero wallet and make not of wallet address and private keys

Use https://moneroaddress.org/

6. Replace above the Wallet address and your email in following file <source-directory>/bin/config.txt in below format.

We are using supportxmr pool so use the below template for pool address.

Only replace your email / wallet address. Follow template for everything else.

“pool_address” : “pool.supportxmr.com:5555”,
“wallet_address” : “4945WAJVEC6A3ZM8hwWMrV15VSJeeAvUv3fRbwwMajToCQ2usQa2tefGyx6PFQwXqMfpk7dVdxX6BBqZfYibx3JD3UKzrFk”,
“pool_password” : “MinerName:<email-address>”,

Please note that the default config file assumes you have

8. Start mining by executing the binary you previously built. This binary will refer to config.txt to determine your wallet address, the pool you are joining and number of CPUs in the server etc. ( It assumes you have 2 x 10 core POWER proc. If its different config, you need to change the config file to reflect that )

Use a screen session to run the mining binary so that binary runs in background (and you don’t have to baby sit it)

apt install screen

screen

<hit enter for the message>

<source-director>/bin/xmr-stak/power

7. After mining for 5 minutes, login to supportxmr.com using these default credentials ::

username:     <your-wallet-address>

password:     <your-email-address provided in above config.txt>

8. Change your password to your choice after getting in to the website.

9. At this point the Top Right Corner on supportxmr.com shows your current hash-rate mined via the pool. Here’s test output using 2 Barreleye G2 servers:

Screen Shot 2017-11-20 at 4.00.15 AM

Advertisements
How to mine Monero (XMR) on Rackspace Barreleye G2 Server (IBM POWER9)

How to mine Ethereum on Rackspace BarreleyeG2 OpenPOWER server with Nvidia P100 GPUs

OpenPOWER CPUs and Nvidia GPUs represent the best in class computing Power per $

To test that, I thought I’ll give compiling Ethereum mining suite a try on our Barreleye G2 server. As background information Barreleye G2 is a server Rackspace is building in collaboration with Google, IBM and Ingrasys

Screen Shot 2017-09-02 at 9.56.50 PM

To compile Ethereum on Power9 with Nvidia P100 GPUs you have 2 options:

A)  Compile it for CUDA

B)  Compile it for OpenCL

In this post we will go with Option A (Compile it with CUDA). To do this you need to first get Nvidia Drivers and CUDA working on your openPOWER server. The link to blog post on how to get these working is here:

Once you have CUDA working (run an example CUDA application to make sure), install the build dependencies necessary for compiling ethereum. We will use cmake to configure and build.

sudo apt-get install libleveldb-dev libmicrohttpd-dev cmake

After looking around I realized that main github source for ethereum “ethereum/cpp-ethereum” no longer supports mining for GPUs. So instead I used: https://github.com/ethereum-mining/ethminer for my mining experiments with Nvidia P100 GPUs. I found the release “0.12.0.dev2” to be stable.

Download the latest official release zip and unzip

wget https://github.com/ethereum-mining/ethminer/archive/v0.12.0.dev2.zip

unzip v0.12.0.dev2.zip

cd ethminer-0.12.0.dev2/

Make a build directory for your build activities and make that your current directory

mkdir build
cd build

Since I’m building for CUDA and NOT OpenCL, I made sure, my cmake configuration reflects that. This step 3-4 minutes on Barreleye G2 server

cmake .. -DETHASHCUDA=ON -DETHASHCL=OFF

Once cmake configure is done, we get to the fun part, building:

cmake –build .

If your build from source looks good, your screen should roughly look like this:

Screen Shot 2017-09-02 at 8.00.00 PM

Make install the ethminer binary

sudo make install

At this point you have successfully built your ”ethminer” binary and it should reside in “/usr/local/bin/ethminer”

Play around with your newly built binary with command-line options:

/usr/local/bin/ethminer –help

Before actually mining you can benchmark / simulate to see how many Hash Rate (MH/s) can you hit

/usr/local/bin/ethminer -M -U

Screen Shot 2017-09-02 at 9.22.26 PM

Once you have simulated and your expected MH/s match your expectations. Its time to actually mine ethereum.

You need to create a ethereum wallet and get your own address to send your mined coins to. Go to the below address, create and save your credentials safely

https://www.myetherwallet.com/

Once you have you created <ethereum address> and think of <miner tag> to identify your machine (helpful you are mining via multiple machines)

/usr/local/bin/ethminer –farm-recheck 200 -U -S us1.ethermine.org:4444 -FS eu1.ethermine.org:4444 -O <wallet address>.<tag>

/usr/local/bin/ethminer –farm-recheck 200 -U -S us1.ethermine.org:4444 -FS eu1.ethermine.org:4444 -O 0x4ff2de61282aa5da02E5F8399DB7d47A66Be1465.barreleyeg2

Soon your will receive console output indicating completed shares and your Hash rate:

Screen Shot 2017-09-02 at 9.44.26 PM

Now how do you confirm / check the number of shares / amount of ethereum you have mine ? Got to below website and look up your wallet address from above:

https://ethermine.org

You will be able to see details like below:

Screen Shot 2017-09-02 at 9.48.41 PM

You can use below command to check how much power your nvidia GPUs are consuming

nvidia-smi

Please comment below if you have questions or are interested in price of this server / setup OR profitability.

How to mine Ethereum on Rackspace BarreleyeG2 OpenPOWER server with Nvidia P100 GPUs

Barreleye G2 User-guide: Setup OpenPOWER P100 GPU Server with Ubuntu 17.04 Zesty

HOST Console (Serial Console):

To get to the host console via serial / VGA use the following connections on front panel:

a) VGA + USB on the front panel.

B) Host serial console on the front panel (RJ-45 Serial)

 

Screen Shot 2017-05-11 at 3.27.44 PM

HOST Console (via BMC / OOB network):

You can get to host console via BMC / OOB IP Address. By default BMC will DHCP for IP Address. For getting the BMC IP , simply look up BMC IP corresponding to BMC dedicated Mac (for Example : 00:1f:e2:fd:f4:32)  or BMC shared Mac (for example:  00:1f:e2:fd:f4:33)

one you have the BMC IP, log into host console via following command

(Password: 0penBmc ):

ssh -p2200 root@<bmc-ip-address>

BMC / OOB Power-on : Power-off:

For powering the system on / off via BMC console, first ssh into BMC via following command (Password: 0penBmc  ):

ssh root@<bmc-ip-address>

Once logged into BMC , use the following commands to:

Check the state of the host:  obmcutil state

Power on the Host:                obmcutil poweron

Power off the Host:                obmcutil poweroff

SSH into host shell:

In order to ssh into the host , notice the host is going to default to DHCP for Broadcomm LOM Mac address(for example:  00:1f:e2:fd:f4:31) by default.

Look up your dhcp server for this mac to get Host IP

Log into host shell with below ssh command: (with password:  passw0rd)

ssh nvidia@<host-ip>

How to install P100 GPUs into server:

Screen Shot 2017-05-11 at 4.25.39 PM
Remove 4 screws around GPU metal base to release it from server tray. You will still have the PCIe cable connected
Screen Shot 2017-05-11 at 4.25.51 PM
With the released metal base (PCIe cable still connected), Start sliding the P100 GPU until it “clicks” with PCIe connector.
Screen Shot 2017-05-11 at 4.25.58 PM
Put 2 screws in the front of the metal base so that GPU sits on PCIe connector firmly
Screen Shot 2017-05-11 at 4.26.07 PM
Put the metal base with P100 GPU on Barreleye G2 Tray. Allign the metal base so that screw holes align with server tray 
Screen Shot 2017-05-11 at 4.26.15 PM
Put 4 screws back to secure the metal base (with GPU) to the server tray

 

Errata on Barreleye G2:

a) 17.04 daily builds are unstable, they fail to boot intermittently

b) 17.04 Serial console has a bug where it flashes the login screen continuously making it impossible to login. When this happens , the host is still up and functioning properly. Just SSH instead using Host IP to execute your commands.

c) Gen3 link training fails on boot up occasionally and link falls to Gen2. If you want GPU to train Gen3 , reboot until both GPUs train as Gen3 (Check via lspci) . Once link trains to Gen3 speed there are no issues running tests. The issue is only with the training algorithm in processor, once it trains the speed you want, things should be stable.

How to Install 17.04

Here’s how to install Ubuntu 17.04 on PPC64LE server. In my case Barreleye G2, our openPOWER server is running OPAL as BIOS firmware with Petitboot bootloader:

Here are steps through install:

1) Login to Out of band management and get into host console

  • Logging into host OOB console  is important since Ubuntu 17.04 PPC64EL installer may not have  ASPEED VGA / Graphics drivers. So you cannot see the install progress through the VGA console.
  • You can connect a host serial console via RJ-45 Serial on front panel OR
  • As indicated before you can login into host console via network
    • (Password: 0penBmc ):

      ssh -p2200 root@<bmc-ip-address>

2) Power on the server and get to Petitboot bootloader. Which should look like below:

Screen Shot 2016-05-06 at 1.37.12 PM

2) Now type “n” to have add a new boot option.

We are netboot installing 17.04. So let’s grab and add the kernel and initrd url’s for Zesty (17.04 code name) respectively:

Kernel:

http://ports.ubuntu.com/ubuntu-ports/dists/zesty/main/installer-ppc64el/current/images/netboot/ubuntu-installer/ppc64el/vmlinux

Initrd:

http://ports.ubuntu.com/ubuntu-ports/dists/zesty/main/installer-ppc64el/current/images/netboot/ubuntu-installer/ppc64el/initrd.gz

Your screen should look like this:

Screen Shot 2016-05-06 at 1.38.58 PM

3) Press OK and you should return to Petitboot main screen. The boot option you just added should come up as “User Item 1”:

Screen Shot 2016-05-06 at 1.39.25 PM

4) Select and enter “User Item 1” for starting the install. Following which you will go through standard Ubuntu Install process, that you should be familiar with.

Thanks I hope this post helps you Install not just Ubuntu 17.04 but any OS of your choice using Petitboot.

Barreleye G2 User-guide: Setup OpenPOWER P100 GPU Server with Ubuntu 17.04 Zesty