Difference between revisions of "Projects (USIU)"

From Raspberry Pi Min-Grant project
Jump to: navigation, search
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
=== Project: Pi Based Cluster ===
 
=== Project: Pi Based Cluster ===
  
<u>Course: APT 4030 - Parallel Computing</u>
+
''Course: DST4030 - Parallel Computing''
 +
 
 +
''Members''
 +
 
 +
#''Waituru Mwangi''
 +
#''Mercy Nekesa''
 +
#''Frank Rutahiwa''
 +
#''Bruce Kwiringira''
 +
#''Joseph Manywanda''
 +
#''Hungai Kevin''
 +
#''Trevis Sinatra''
  
 
'''About:'''
 
'''About:'''
Line 17: Line 27:
 
*Familiarity with raspberry pi micro computers useful for rapid hardware prototyping.
 
*Familiarity with raspberry pi micro computers useful for rapid hardware prototyping.
  
Building the system:
+
'''Building the system:'''
  
 
Components:
 
Components:
Line 38: Line 48:
 
'''Resources:'''
 
'''Resources:'''
  
There are many resources that we will use including Raspberry Pi Foundation https://www.raspberrypi.org/
+
There are many resources that we will use including Raspberry Pi Foundation [https://www.raspberrypi.org/ https://www.raspberrypi.org/]
  
Prof Simon Cox, Making a Raspberry pi super computer, University of Southampton http://coen.boisestate.edu/ece/raspberry-pi/
+
Prof Simon Cox, Making a Raspberry pi super computer, University of Southampton [http://coen.boisestate.edu/ece/raspberry-pi/ http://coen.boisestate.edu/ece/raspberry-pi/]
  
 
Configuring the nodes, step by step guide
 
Configuring the nodes, step by step guide
Line 48: Line 58:
 
When this is done, its easier to clone as many nodes as wanted.
 
When this is done, its easier to clone as many nodes as wanted.
  
1.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Get the OS image raspberrypi.org/download
+
 
 +
 
 +
'''1. Get the OS image raspberrypi.org/download'''
  
 
After many false starts, we were content to just use Rasbian Weezy 5.5
 
After many false starts, we were content to just use Rasbian Weezy 5.5
  
2.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Get image into a the sdcard
+
 
 +
 
 +
'''2. Get image into a the sdcard'''
  
 
On Linux:
 
On Linux:
Line 58: Line 72:
 
<span style="font-family:courier new,courier,monospace">#dd if=/media/yourMachine/Images/2015-05-05-raspbian-wheezy.img of=/dev/sdb bs=512 conv=noerror,sync</span>
 
<span style="font-family:courier new,courier,monospace">#dd if=/media/yourMachine/Images/2015-05-05-raspbian-wheezy.img of=/dev/sdb bs=512 conv=noerror,sync</span>
  
3.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Boot the Pi
+
 
 +
 
 +
'''3. Boot the Pi'''
  
 
Great suspense it anything did not go as planned.
 
Great suspense it anything did not go as planned.
Line 64: Line 80:
 
If there is an error with the card or any other thing, nothing will show up on the screen, if the Pi is overwhelmed it will take forever to boot. Rasbian is ideal because it has been tested and has a descent first boot time.
 
If there is an error with the card or any other thing, nothing will show up on the screen, if the Pi is overwhelmed it will take forever to boot. Rasbian is ideal because it has been tested and has a descent first boot time.
  
4.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Configuration on first boot
+
 
 +
 
 +
'''4. Configuration on first boot'''
  
 
These configurations can be done later with raspi-config or ideally done on the first boot.
 
These configurations can be done later with raspi-config or ideally done on the first boot.
  
•&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Expand image to fill card
+
*Expand image to fill card
 +
*Change the password<br/><span style="font-family:courier new,courier,monospace">laxmi</span>
  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Change the password
+
• Change hostname (node1 or nodex)
  
<span style="font-family:courier new,courier,monospace">laxmi</span>
 
  
•&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Change hostname (node1 or nodex)
 
  
5.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Re boot
+
'''5. Re boot'''
  
 
user: <span style="font-family:courier new,courier,monospace">pi</span> (has root priviledges) password: <span style="font-family:courier new,courier,monospace">laxmi</span>
 
user: <span style="font-family:courier new,courier,monospace">pi</span> (has root priviledges) password: <span style="font-family:courier new,courier,monospace">laxmi</span>
Line 90: Line 107:
 
you only get version 23, will have to compile to use version 24)
 
you only get version 23, will have to compile to use version 24)
  
6. Refresh and update the softwares, if not done yet.
+
 
 +
 
 +
'''6. Refresh and update the softwares, if not done yet.'''
  
 
<span style="font-family:courier new,courier,monospace">#sudo apt-get update && sudo apt-get upgrade -y</span>
 
<span style="font-family:courier new,courier,monospace">#sudo apt-get update && sudo apt-get upgrade -y</span>
  
7. Get Fortran, (strange but we need it)
+
 
 +
 
 +
 
 +
 
 +
'''7. Get Fortran, (strange but we need it)'''
  
 
<span style="font-family:courier new,courier,monospace">#sudo apt-get install gfortran</span>
 
<span style="font-family:courier new,courier,monospace">#sudo apt-get install gfortran</span>
  
8. Before getting Argonne MPICH
 
  
Resource: http://www.mpich.org/documentation/guides/
+
 
 +
 
 +
 
 +
'''8. Before getting Argonne MPICH'''
 +
 
 +
Resource: [http://www.mpich.org/documentation/guides/ http://www.mpich.org/documentation/guides/]
  
 
The have great resources not just the the MPICH library
 
The have great resources not just the the MPICH library
Line 106: Line 133:
 
<span style="font-family:courier new,courier,monospace">#mkdir /home/pi/mpich3 $ cd ~/mpich3</span>
 
<span style="font-family:courier new,courier,monospace">#mkdir /home/pi/mpich3 $ cd ~/mpich3</span>
  
9. Get MPICH sources from Argonne
 
  
<span style="font-family:courier new,courier,monospace">#wget http://www.mpich.org/static/downloads/3.1.4/mpich-3.1.4.tar.gz</span>
 
  
Resource: http://www.mpich.org/downloads - Get latest stable
+
'''9. Get MPICH sources from Argonne'''
 +
 
 +
<span style="font-family:courier new,courier,monospace">#wget [http://www.mpich.org/static/downloads/3.1.4/mpich-3.1.4.tar.gz http://www.mpich.org/static/downloads/3.1.4/mpich-3.1.4.tar.gz]</span>
 +
 
 +
Resource: [http://www.mpich.org/downloads http://www.mpich.org/downloads] - Get latest stable
 +
 
 +
 
  
10. Unpack them.
+
'''10. Unpack them.'''
  
 
<span style="font-family:courier new,courier,monospace">#tar xfz mpichXXX.tar.gz</span>
 
<span style="font-family:courier new,courier,monospace">#tar xfz mpichXXX.tar.gz</span>
  
11. Planning a clean place for install
+
 
 +
 
 +
 
 +
 
 +
'''11. Planning a clean place for install'''
  
 
<span style="font-family:courier new,courier,monospace">#sudo mkdir /home/rpimpi/</span>
 
<span style="font-family:courier new,courier,monospace">#sudo mkdir /home/rpimpi/</span>
  
#sudo mkdir /home/rpimpi/mpich3-install
+
<span style="font-family:courier new,courier,monospace">#sudo mkdir /home/rpimpi/mpich3-install</span>
 +
 
 +
 
 +
 
 +
 
  
12. Make a build directory, and go to the build
+
'''12. Make a build directory, and go to the build'''
  
 
<span style="font-family:courier new,courier,monospace">#mkdir /home/pi/mpich_build $ cd /home/pi/mpich_build</span>
 
<span style="font-family:courier new,courier,monospace">#mkdir /home/pi/mpich_build $ cd /home/pi/mpich_build</span>
  
13. Configure the build
+
 
 +
 
 +
 
 +
 
 +
'''13. Configure the build'''
  
 
This will take a while, you can get the cards ready, you can play while the configurations takes place.
 
This will take a while, you can get the cards ready, you can play while the configurations takes place.
Line 132: Line 175:
 
<span style="font-family:courier new,courier,monospace">#sudo /home/pi/mpich3/mpichXXX/configure -prefix=/home/rpimpi/mpich3- install</span>
 
<span style="font-family:courier new,courier,monospace">#sudo /home/pi/mpich3/mpichXXX/configure -prefix=/home/rpimpi/mpich3- install</span>
  
14. Make
+
 
 +
 
 +
 
 +
 
 +
'''14. Make'''
  
 
What ever it took the last, you can +1
 
What ever it took the last, you can +1
Line 138: Line 185:
 
<span style="font-family:courier new,courier,monospace">#sudo make</span>
 
<span style="font-family:courier new,courier,monospace">#sudo make</span>
  
15. Install the files
+
 
 +
 
 +
 
 +
 
 +
'''15. Install the files'''
  
 
It can take a bit of time, but not any way close to the last two stages.
 
It can take a bit of time, but not any way close to the last two stages.
Line 144: Line 195:
 
<span style="font-family:courier new,courier,monospace">#sudo make install</span>
 
<span style="font-family:courier new,courier,monospace">#sudo make install</span>
  
16. Add the place that you put the install to your PATH
+
 
 +
 
 +
 
 +
 
 +
'''16. Add the place that you put the install to your PATH'''
  
 
<span style="font-family:courier new,courier,monospace">#export PATH=$PATH:/home/rpimpi/mpich3-mstall/bin</span>
 
<span style="font-family:courier new,courier,monospace">#export PATH=$PATH:/home/rpimpi/mpich3-mstall/bin</span>
  
17. Or Note to permanently put this on the PATH you will need to edit .profile
+
 
 +
 
 +
'''17. Or Note to permanently put this on the PATH you will need to edit .profile'''
  
 
<span style="font-family:courier new,courier,monospace">#emacs -/.profile</span>
 
<span style="font-family:courier new,courier,monospace">#emacs -/.profile</span>
Line 156: Line 213:
 
<span style="font-family:courier new,courier,monospace">#Add MPI to path (This is just a comment for later) PATH="$PATH:/home/rpimpi/mpich3-install/bin"</span>
 
<span style="font-family:courier new,courier,monospace">#Add MPI to path (This is just a comment for later) PATH="$PATH:/home/rpimpi/mpich3-install/bin"</span>
  
18.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Verify if the install were succesful
+
 
 +
 
 +
'''18. Verify if the install were succesful'''
  
 
<span style="font-family:courier new,courier,monospace">#which mpicc /home/rpimpi/mpich3-install/mpicc $ which mpiexec</span>
 
<span style="font-family:courier new,courier,monospace">#which mpicc /home/rpimpi/mpich3-install/mpicc $ which mpiexec</span>
  
19. Go /home and set a place for your first test
 
  
<span style="font-family:courier new,courier,monospace">#cd ~<br/></span>
+
 
 +
'''19. Go /home and set a place for your first test'''
 +
 
 +
<span style="font-family:courier new,courier,monospace">#cd ~</span>
  
 
<span style="font-family:courier new,courier,monospace">#mkdir mpi_first_test $ cd mpi_first_test</span>
 
<span style="font-family:courier new,courier,monospace">#mkdir mpi_first_test $ cd mpi_first_test</span>
  
20.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Now testing MPI on single node
+
 
 +
 
 +
'''20. Now testing MPI on single node'''
  
 
<span style="font-family:courier new,courier,monospace">#mpiexec -f machinefile -n &lt;number&gt; hostname</span>
 
<span style="font-family:courier new,courier,monospace">#mpiexec -f machinefile -n &lt;number&gt; hostname</span>
Line 172: Line 235:
 
where machine file contains a list of IP addresses (in this case just one) for the machines
 
where machine file contains a list of IP addresses (in this case just one) for the machines
  
21. How this supposed to be done
+
 
 +
 
 +
'''21. How this supposed to be done'''
  
 
a) Get your IP address
 
a) Get your IP address
Line 186: Line 251:
 
<span style="font-family:courier new,courier,monospace">192.168.1.161 [or the ip is ... ]</span>
 
<span style="font-family:courier new,courier,monospace">192.168.1.161 [or the ip is ... ]</span>
  
22.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Now test if the machinefile
+
 
 +
 
 +
'''22. Now test if the machinefile'''
  
 
<span style="font-family:courier new,courier,monospace"># mpiexec -f machinefile ~n 1 hostname</span>
 
<span style="font-family:courier new,courier,monospace"># mpiexec -f machinefile ~n 1 hostname</span>
Line 192: Line 259:
 
Output should be: node1 ('hostname')
 
Output should be: node1 ('hostname')
  
23. Little C code using MPI on Pi to calculate Pi
+
 
 +
 
 +
'''23. Little C code using MPI on Pi to calculate Pi'''
  
 
Don't worry, we shall not write the c code our selves, but MPICH has some example codes we can run.
 
Don't worry, we shall not write the c code our selves, but MPICH has some example codes we can run.
Line 206: Line 275:
 
pi is approximately 3.141 5926544231318, Error is 0.0000000008333387
 
pi is approximately 3.141 5926544231318, Error is 0.0000000008333387
  
24.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Celebrate!
+
 
 +
 
 +
'''24. Celebrate!'''
  
 
This calls for a celebration! (Seriously!)
 
This calls for a celebration! (Seriously!)
  
25.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Shut down
+
 
 +
 
 +
'''25. Shut down'''
  
 
<span style="font-family:courier new,courier,monospace"># sudo poweroff</span>
 
<span style="font-family:courier new,courier,monospace"># sudo poweroff</span>

Latest revision as of 12:45, 30 July 2015

Project: Pi Based Cluster

Course: DST4030 - Parallel Computing

Members

  1. Waituru Mwangi
  2. Mercy Nekesa
  3. Frank Rutahiwa
  4. Bruce Kwiringira
  5. Joseph Manywanda
  6. Hungai Kevin
  7. Trevis Sinatra

About:

In this project, we will build a cluster of identical raspberry pi nodes, networked together and running parallel processing software that allows each node in the cluster to share data and computation.

Objective(s):

Building a cluster computer powered by raspberry Pi that could be used to develop and run parallel and distributed programs. In doing so, the following goals could be achieved.

Practical understanding of building parallel systems.

  • Experiment with different configurations to achieve better performance.
  • Familiarity with MPI (Message Passing Interface) API for parallel programming.
  • Familiarity with raspberry pi micro computers useful for rapid hardware prototyping.

Building the system:

Components:

  • Raspberry Pi Model B
  • Storage - SD cards
  • Ethernet cables
  • Power supply
  • Linux OS
  • There is an array of different possibilities.
  • MPI library
  • MPICH or OpenMPI
  • Ethernet switch and Router
  • A switch that we can use without interruption during the lab works.

Implementation:

Ideally the project will be implemented in stages, starting with configuring the first two nodes and then scaling to add all other additional nodes.

Resources:

There are many resources that we will use including Raspberry Pi Foundation https://www.raspberrypi.org/

Prof Simon Cox, Making a Raspberry pi super computer, University of Southampton http://coen.boisestate.edu/ece/raspberry-pi/

Configuring the nodes, step by step guide

  • First, configure the first node

When this is done, its easier to clone as many nodes as wanted.


1. Get the OS image raspberrypi.org/download

After many false starts, we were content to just use Rasbian Weezy 5.5


2. Get image into a the sdcard

On Linux:

#dd if=/media/yourMachine/Images/2015-05-05-raspbian-wheezy.img of=/dev/sdb bs=512 conv=noerror,sync


3. Boot the Pi

Great suspense it anything did not go as planned.

If there is an error with the card or any other thing, nothing will show up on the screen, if the Pi is overwhelmed it will take forever to boot. Rasbian is ideal because it has been tested and has a descent first boot time.


4. Configuration on first boot

These configurations can be done later with raspi-config or ideally done on the first boot.

  • Expand image to fill card
  • Change the password
    laxmi

• Change hostname (node1 or nodex)


5. Re boot

user: pi (has root priviledges) password: laxmi

Refresh update repo packages and update the software

#sudo apt-get update && sudo apt-get upgrade -y

Choose your poison: I prefer having my emacs

#sudo apt-get install emacs

you only get version 23, will have to compile to use version 24)


6. Refresh and update the softwares, if not done yet.

#sudo apt-get update && sudo apt-get upgrade -y



7. Get Fortran, (strange but we need it)

#sudo apt-get install gfortran



8. Before getting Argonne MPICH

Resource: http://www.mpich.org/documentation/guides/

The have great resources not just the the MPICH library

#mkdir /home/pi/mpich3 $ cd ~/mpich3


9. Get MPICH sources from Argonne

#wget http://www.mpich.org/static/downloads/3.1.4/mpich-3.1.4.tar.gz

Resource: http://www.mpich.org/downloads - Get latest stable


10. Unpack them.

#tar xfz mpichXXX.tar.gz



11. Planning a clean place for install

#sudo mkdir /home/rpimpi/

#sudo mkdir /home/rpimpi/mpich3-install



12. Make a build directory, and go to the build

#mkdir /home/pi/mpich_build $ cd /home/pi/mpich_build



13. Configure the build

This will take a while, you can get the cards ready, you can play while the configurations takes place.

#sudo /home/pi/mpich3/mpichXXX/configure -prefix=/home/rpimpi/mpich3- install



14. Make

What ever it took the last, you can +1

#sudo make



15. Install the files

It can take a bit of time, but not any way close to the last two stages.

#sudo make install



16. Add the place that you put the install to your PATH

#export PATH=$PATH:/home/rpimpi/mpich3-mstall/bin


17. Or Note to permanently put this on the PATH you will need to edit .profile

#emacs -/.profile

and add lines below:

#Add MPI to path (This is just a comment for later) PATH="$PATH:/home/rpimpi/mpich3-install/bin"


18. Verify if the install were succesful

#which mpicc /home/rpimpi/mpich3-install/mpicc $ which mpiexec


19. Go /home and set a place for your first test

#cd ~

#mkdir mpi_first_test $ cd mpi_first_test


20. Now testing MPI on single node

#mpiexec -f machinefile -n <number> hostname

where machine file contains a list of IP addresses (in this case just one) for the machines


21. How this supposed to be done

a) Get your IP address

#ifconfig

b) Put this into a single file called machinefile

#emacs machinefile

c)            Add this line:

192.168.1.161 [or the ip is ... ]


22. Now test if the machinefile

# mpiexec -f machinefile ~n 1 hostname

Output should be: node1 ('hostname')


23. Little C code using MPI on Pi to calculate Pi

Don't worry, we shall not write the c code our selves, but MPICH has some example codes we can run.

# cd /home/pi/mpi_fist_test

# mpiexec -f machinefile -n 2 /home/pi/mpich3/examples/cpi

Output is should be

Process 0 of 2 is on raspberrypi Process 1 of 2 is on raspberrypi

pi is approximately 3.141 5926544231318, Error is 0.0000000008333387


24. Celebrate!

This calls for a celebration! (Seriously!)


25. Shut down

# sudo poweroff