|
|
Student Intro
Code of Practice
Human Relationships
Security
General information on research
General Usage of Our Beowulf Cluster
Linux and Command-Line Utilities
X11 Forwarding
Handling TAR-files
Firefox Problem Tied with NFS and lightening
Riken
Kepler
Programming
General Programming
Fortran
Gnuplot
GPU
Problem Solving
Running MPI Code
FDTD-related
Skills for presentation and technical report writing
LaTeX
To BSc and MSc Students
Living in Manchester
Accommodation Hunting
Fumie's availability
Code of Practice
Human Relationships
When you talk to me either during the meeting or in non-meeting hours, you
should clearly show me your
own understanding of the problem. I will
not judge you for your mistakes but support you and correct your
understanding. You should share you research understanding and problems.
You
should not act as you understand everything if you do not. Ultimately,
when it comes to research you and I are co-workers. We stand at the same
level and you should treat me as your collaborator.
Security
In the past there were some incidents involving access to various machines in
our cluster and entering permission-unprotected directories as well as viewing
files whose owner was different from the accessing person.
The cluster administrator is capable of tracking all actions of all
users. Ignorance and lack of Unix system knowledge will not be considered as
an excuse for violating these security rules.
Unless you execute the parallel computing programs in our computer cluster or
belong to the cluster service providers you are expected to login and use only
your own machine and access only your own or shared files.
Increased security constrains put more burden on the cluster
administrator. Therefore, students demonstrating strong Unix skills might be
asked to help the senior cluster administrators due to the current staff
shortage.
Please read, sign and return to me the local security policy document.
Your university card can be activated
to enter the Sackville street building
after 17:00 on weekdays and all day long on weekends
by swiping your card.
Please give me your card number so that
I can ask john.ashton-2@manchester.ac.uk
to activate your card.
General information on research
The following reference will give you the overview of the research
General Usage of Our Beowulf Cluster
Linux and Command-Line
Utilities
The Key F1 raises the window under the mouse.
The Key F2 lowers the window under the mouse.
The Key F3 iconizes the window under the mouse.
The Key F4 kills the window under the mouse.
Linux as all Unix systems is famous for the powerful command line utilities it
offers. Refer to the following articles if you have little knowledge
about
Linux
,
LinuxPlusShell
and
Command-line interface.
Please go through
Online Introduction to Unix
to acquire the minimum skills on Unix.
Then go through
Linux Shell
Scripting Tutorial.
Refer to the openSUSE 10.3 Reference for the introduction to the important
Linux commands and secure network operations.
Reading entire Chapters 19 and 36 is strongly advisable:
19: Working with the Shell |
pp. 271-296 |
36: SSH-Secure Network Operations |
pp. 589-596 |
The most important reference Sections are:
19.3: Important Linux Commands |
pp. 280-290 |
36.2: The ssh Program |
p. 590 |
36.3: spc-Secure Copy |
p. 590 |
Refer to the Wikipedia article below for a list of frequently used Unix commands.
Bash is a command-line interpreter provided with the majority of Linux
distributions. It allows you to execute the command-line utilities and
programs:
If you face the bash message "command not found" your shell did not
recognise the command you executed. It might occur because of the incorrect
spelling of command name, e.g. "gfrtran" instead of "gfortran". In the
other case the executed command might not be present in your system or you do
not have sufficient user rights to run it.
X11 Forwarding
The following IP addresses are available in our cluster.
As for 32 bits machines:
130.88.108.197
130.88.154.10
130.88.154.26
130.88.154.29
130.88.116.228
130.88.116.188
130.88.116.73
As for 64 bits machines:
130.88.118.133
130.88.154.30
130.88.154.31
130.88.154.34
130.88.154.35
130.88.154.36
130.88.154.177
130.88.154.197
130.88.118.141
The machine
130.88.154.30
is the NFS server and
/home is NFS-mounted
in our cluster apart from Sun workstations.
/home is daily backed up.
Therefore
please do not run any computation and do not store the data which can be
reproduced by the simple computation under the home directories.
Only source code, shell scripts, reading materials, LaTex files,
figures for publication should be stored under your home
directories.
The most of the computation should be done under
the local directories such as
/local ,
/scratch ,
/scratch1 ,
/scratch2
on each machine.
You should find the location and spare space for computation
on each machine by running
df -ha apart from
130.88.116.120
and
130.88.116.237 .
On 130.88.116.120 and
130.88.116.237 ,
you have to create your own directory at
/tmp
and transfer the data to one of the local
directories in other machines as soon as you finish the computation.
Please run the memory hungry program on
130.88.154.177 (8 GB),
130.88.116.120
(96 GB,Dell Precision T7500),
130.88.116.237
(96 GB,Dell Precision T7500).
If they are left unused with Windows OS, then
please go to E16 whose code is C1982
and boot the machines by choosing Linux(Latest kernel).
The code for E24 is 1965 for fridge and microwaves.
The code for E20 is C2854.
The code for E23 is 0258.
The code for E1 is 3419.
The code for E6 is 7346.
The code for D45a is 6601#
The code for D45b is 4419#
When you compile your program using Makefile on
the dual core machines of
130.88.154.34 ,
130.88.154.35 ,
130.88.154.36 ,
130.88.154.177 ,
please use make -j2 ,
On 130.88.154.177 , you
can use make -j4 .
On 130.88.116.120 ,
you can use make -j8
for the fast compilation.
A short step-by-step guide explaining how to export graphical output from a
remote Linux machine to your local Linux computer.
Handling TAR-files
Compressing and decompressing data to and from TAR-file archives:
tar -cjvf <archive_name> <file_list> |
tar -cjvf archive.tar.bz2 file1.txt file2.txt |
This command will compress two files into an archive. You may also use common Unix wildcards and type "*" or "file?.txt" to compress all files in the directory or all files with a variable single symbol value.
Follow the command below to decompress a TAR-archive file into a current directory:
tar -xjvf <archive_name> |
tar -xjvf archive.tar.bz2 |
Firefox Problem Tied with NFS and lightening
Firefox is available only 32 bits machines.
Home directories in our cluster are mounted via NFS and backed up daily. NFS
might cause problems running Firefox web browser on a certain machine. Please
follow the steps below to restart Firefox:
cd ~/.mozilla/firefox/<letter_sequence>.default
ls -lrt lock to find the machine's IP address where you run Firefox at the moment
rm lock to delete the lock
- Login to the machine where you were running Firefox
- Find the process number which is related to Firefox
kill all processes associated with Firefox on that particular machine
Please perform the following to reduce the workload of the system:
- run
firefox
- Find
tools
- select
add-ons
- search
noscript
- install
noscript
Riken
The following is the procedure to get started
- Java version check by doing
java
-version
The version has to be higher than 1.4.
- Firefox version check
help -> about
firefox
If the version is higher than 3, that's fine but lower than this also works.
- Just under your home directory in our cluster,
rsync -auvz -e ssh
mehshan@130.88.154.30:/home/mehshan/keys .
If the file's permission is not 600, set the permission to 600.
- Access
https://riccvpn.riken.jp
- Click
start application
and click always trust and run
- Click
ok
- Alternatively access
https://riccvpn.riken.jp/+CSCOE+/logon.html
and choose the group of ricc and put the username of
fumie
and the password from RIKEN.
- Open a new firefox terminal and access
https://127.0.0.1:4430
- Otherwise skip 1-7 steps and login to --
--130.88.154.34
- From local X-terminal in our cluster
ssh
-l fumie -p 2200 -i ~/keys/riccken.key
127.0.0.1 . Here USER means your username in our
cluster. And if someone has already one this procedure on
130.88.154.34, then you can start this procedure, not
from the first procedure by logging into 130.88.154.34.
-
cd /data/fumie
If you use the Riken's facility first time,
please do mkdir /data/fumie/USER .
USER
can be anything, but your given name is preferred to identify the
owner of the directory.
-
Please do not use
/home/fumie to run jobs.
If you do have some files in /home/fumie in Riken please
move the files by mv /home/fumie/yourdirectory/datadirectories
/data/fumie/yourname/yourpreferreddirectory
-
As for the data transfer
between our local cluster and RIKEN,
you initiate the data transfer always from our cluster.
From local X-terminal in our cluster,
in order to transfer the data from our side:
scp -C -r -P 2200 -i ~/keys/riccken.key
nameofthelocaldirectorywithpath
fumie@127.0.0.1:/data/fumie/USER/rikendirectory
in order to transfer the data from Riken: stay in
the x-window in our local cluster and then
scp -C -r -P 2200 -i ~/keys/riccken.key
fumie@127.0.0.1:/data/fumie/yourname/rikendirectory
nameofthelocaldirectorywithpath
- Try to compile a small program in the
remote terminal (which is the one you opened to access riken via
ssh ) with the compiler option such as -ax which is
written on page 22 of the quick referece
- Tailor
g00.sh
which is
130.88.154.30:/software/FDTDstandard/
and submit the job via qsub g00-username.sh for example.
Please make sure that the job script has your name in it; otherwise
the jobs are killed by Fumie.
The way to write the Makefile and job submission script
is explained in
RikenArchitecture
-
qstat with some option ( -x for example) gives us
how busy each supercomputer is and we identify the
calm one and re-compile the program with the specific machine option
and submit the job via qsub go.sh
Further detail on the usage of the RIKEN's cluster should be obtained
from
Users Guide
Kepler
The following is the procedure to get started
-
login to one of the machines in our cluster
ssh -XC -l
username -p 22111 130.88.xxx.xxx
-
login to the GPU cluster
ssh -XC -p 22 -l h70180b tatara.cc.kyushu-u.ac.jp
-
Move to your own directory in the GPU cluster
cd /homecx/usr0/h70180b/yourname
-
In order to compile your CUDA code,
you set the compilation environment as follows
bash ;
export PATH=/usr/local/cuda-5.0/bin:$PATH ;
export PATH=/usr/local/pgi/linux86-64/13.3/bin:$PATH ;
export LM_LICENSE_FILE=$LM_LICENSE_FILE:/usr/local/pgi/license.dat
-
Modify your Makefile such as the one at /home/usr0/h70180b/mschinis and run
make -F Makefile
-
In order to perform the job submission,
you need to modify your job submission script such as
/home/usr0/h70180b/mschinis/run-mschinis-gpu.sh
-
In order to submit the job, run
pjsub run-mschinis-gpu.sh
-
Never ever run any jobs at the front end in Kepler and RIKEN!
Programming
Use a text editor emacs for the
programming activity. The information on
emacs is
reference file
General Programming
For the introduction to the basic programming concepts and style.
Programming skills are acquired in practise and experimentation. Do not rely
on pure book reading or asking people about each error message.
Programming as well as LaTeX typesetting and shell scripting involves debugging of your code so that
you can compile the program without errors and obtain the expected result.
By facing and tackling the error messages from compilers you actually learn
the techniques for programming. When you are stuck with the compilation or
receive strange error messages, which you can not fix, you can produce a
little program which mimics your current problem in the large production code.
It is much simpler to debug and fix the "extracted code". Once you make it
work you can transfer the problem solution to the major code.
If you still can not solve the problem, search for the exact error message on the Internet using Google. This activity will also give you an extra knowledge around your problem.
Please do not ask other students in the group to solve your programming
problems unless you tried all the above methods. If you need to ask a question
be as specific as possible. Respect other people's time since all of us work
under tight schedules.
Fortran
GNU Fortran compiler gfortran is used in our cluster. Prior to running
any program code, it has to be compiled. Here are some links which will help
you further:
The wrap-up script mpif90 binds gfortran and mpich to
provide a convenient complication tool for the distributed memory architecture
programs.
Use mpif90intel instead of mpif90 if you need to test compile your code with the Intel Fortran compiler.
The 32-bit machines in our cluster are also suitable for running parallel programs.
Output to stdout and stderr is not printed out immediately after
calling the print statement in GNU Fortran. Instead the output is
buffered (which results in output delays) unless the program believes the
output is done to a terminal. To force immediate output from your program you
have to use the call flush statement in the code straight after the
program action, e.g.:
c = a + b
print *, ";c: ", c
call flush
d = a * b
print *, "d: ", d
call flush
Extra attention should be paid to the following points:
- when your code is under the development, gfortran option
-fbounds-check should be used for compilation
-
put "IMPLICIT NONE" at each program including the subroutines
- ALL the variables are defined and initialised
- do not presume a variable retains its value between invocations of
a procedure unless it has the SAVE attritubute
- the USE of modules is encouraged; the use of INCLUDE is okay
for variable declaration but not for including blocks of executable
statements within a code
- for F90 codes, the INTENT of each subprogram's dummy argument should
be explicitly declared
Gnuplot
gnuplot is used in our cluster for
data visualisation.
Please do go through
Gnuplot practice
in order to acquire the minimum skills you need for our research.
The data which you need to do the practice is
data for Gnuplot practice.
These are offered by
Source of Gnuplot practice.
GPU
In order to understand GPGPU,
please read
GPU intro 1
and
GPU intro 2 .
In order to understand CUDAFortran , please read
Portland
and for optimisation, please read
Optimisation
and
documentation by NVIDIA .
Fermi Architecture
is also very informative.
Problem Solving
The best way to approach a problem is to decompose it into smaller simpler
parts. Decide on what parts you can solve, which tools you are going to use
for a solution. Then think and write a draft algorithm for your solution. It
can be made on paper. Afterwards you can start implementing it using the tools
you have chosen. It is always a good idea to implement the algorithm in
smaller chunks. Where each chunk is performing a specific function of the main
algorithm. You can always write a very basic program testing a specific tool
or algorithm functionality. If your prototype works correctly you can increase
its functionality and integrate into the main program.
To solve the file opening and reading problem in Fortan. Decompose it into
simpler tasks:
- generating target file name,
- opening target file,
- reading data required,
- closing target file
All of those tasks can be written as 4 separate Fortran programs. Afterwards
you can put it all together into one main code. Finally, you may address the
loop issue-and wrap the 4 functions up with a triple loop for i, j and k
values.
As a rule of thumb try to search for problem solutions on the Internet using
Google. This is the quickest way to find answers. Extract the error message of
your code, paste it in Google and search for answers. Finding information on
your own will make you learn and remember. It is much better for you, than me
telling you the answer straight away.
Running MPI Code
When you need to learn MPI
please go through
http://staff.cs.manchester.ac.uk/~fumie/internal/MPI/
Before running a parallel program with MPI an MPI service (daemon) has to be
launched. Running on the background this daemon provides Message Passing
functionality to your programs.
Prior to do any MPI activity, you have to do the following(just once
only when you join our cluster)
$USERNAME has to be changed to
your own username to carry out the following procedure.
Step 1
echo " MPD_SECRETWORD=treta " > /home/$USERNAME/.mpd.conf |
Step 2
chmod 700 /home/$USERNAME/.mpd.conf |
Starting an MPI daemon:
This daemon has to be started on each machine, where you intend to execute parallel program. All MPI daemons have to be connected to each other. This daemon ring connection is set up using the port number and IP address of the first machine, where "mpd" runs. This computer is called "MPI-host".
Find out port number and machine name of mpd-host:
The output is the following:
<mpd_host_name>_<port> (<IP_address>) |
130-88-154-30_20115 (130.88.154.30) |
Extract port number, e.g. 20115 and IP address, e.g. 130.88.154.30.
Login to another machine in the cluster using SSH:
ssh -XC -p 22111 <username>@<IP_address> |
ssh -XC -p 22111 maxim@130.88.154.30 |
Copy files/directories to another machine in the cluster using SCP:
scp -P 22111 -C -r -p <filenameYouWantCopy>
<username>@<IP_address>:<directoryname> |
scp -P 22111 -C -r -p /home/fumie/testdir tadashi@130.88.154.25:/local/tadashi/. |
Enter your password on SSH prompt. Now you have to start MPI daemon on this machine using port number and IP address extracted from the MPI-host.
mpd -h <IP_address> -p <port> |
mpd -h 130.88.154.30 -p 20115 |
The procedure mentioned above can be automated using
/software/64/bin/initRing written by Mr.
Philip Clapham.
Step 1
~/mpd.hosts
should have
the ipaddresses of slave/client machines you
would like to use for computation.
~/mpd.hosts should not have the ipaddress of the host
machine.
Step 2
Login to the host machine
Step 3
Run /software/64/bin/initRing
The MPI environment is ready. Time to compile a parallel code. Compiling MPI code involves calling of a wrapper script "mpif90". This script launches ordinary Fortran compiler (gfortran), but also links the code with MPI libraries.
mpif90 <source> -o <executable> |
As an example,
mpif90 source_code.F90 -o executable_code |
This name of the code can be anything of course.
This command will compile a Fortran source code source_code.F90 into an executable file called executable_file .
To run a parallel code on a number of processors write:
mpiexec -n <proc_number> <executable> |
mpiexec -n 2 ./executable_file |
In general you should keep your program source code in your home directory because it will be backed up every day. Designate a different output directory to e.g.
/local/<user_name>/<prog_name>/ |
/local/maxim/fdtd/ |
Specifying a local output directory will force your code to use the hard drive of a certain machine only. Using the home directory for output will synchronising the data files between all machines of the cluster and considerably slow down other people's computers. Please refrain from doing this.
The IBM Redbook on "Practical MPI Programming" is a recommended reading.
The most important Chapters are:
1: Introduction to Parallel Programming |
pp. 1-10 |
2: Basic Concepts of MPI |
pp. 11-40 |
Appendix B: Frequently Used MPI Subroutines Illustrated |
pp. 161-206 |
Although the environment setting and program compilation are strongly site-dependent, you can get a flavour of parallel programming by reading the following information:
FDTD_related
How many timesteps you need to calculate ?
If the frequency range of interest is from f1 Hz to f2 Hz,
the FDTD calculation has to be done much more than
1/(f1 * delta_t) timesteps.
Thus if you get N timesteps from the FDTD calculation
and assess the accuracy in the frequency domain,
you can take the frequency range starting
more than 1/(N * delta_t) Hz.
Skills for presentation and technical report
writing
Presentation is the activity to
give(present) the audience your knowledge as a gift,
not to speak about your idea which is speech.
- Attract the audience at the first page of
the slide by showing the aim and the outline of the talk
- Keep your eye contact with the audience
confidently
- Use main and second title on each slide
- Smile to make the mood relaxed
- Slow down your talk
so that the audience have time to guess what happens next
- Make your conclusion concise
- Attract readers with the title of your
report.
- Make the structure of your report
crystal clear.
- Use as figures and tables
as much as possible to support
understanding of readers.
- Produce the logical and accurate sentenses
(not spoken English).
- Check your English using
ispell
and
plagiarism checker .
- Conclude your report with concise statements.
LaTeX
The basic guideline on the technical report writing
and
The best method
for presentation of
research results
are very useful stuff to read before you gain good amount of experience in writing technical report.
Please check the marking schemes thoroughly prior to your writing.
Prior to present your manuscript to me, please go through
a check list for your manuscript preparation .
You do not have to worry about the formatting as LaTex does the formatting based on
your style file such as muthesis.cls .
An article on Wikipedia gives a short introduction on
LaTeX — a document markup language extensively used in a scientific domain.
Wikipedia also holds an entire LaTeX
textbook
which you can browse by topics.
There's a University LaTeX users list. If you're interested, send an email from your University email account to LISTSERV@listserv.manchester.ac.uk with the body text "subscribe latex-users".
The referencing is one of the very important part of
your documentation.
Please use the reference abbreviation based on
List of IEEE Transactions
and
IEEE Editorial Style Manual.
Movie can be included in LaTeX beamer as follows:
-
Include the lines
\mode<article>{
\usepackage{fullpage,hyperref,flashmovie}
}
\usepackage{media9}
in your latex source code
-
Copy
/software/sl6.3/noarch-software/atbegshi.sty
/software/sl6.3/noarch-software/infwarerr.sty
/software/sl6.3/noarch-software/ltxcmds.sty
into your directory which has your slide latex file exists
-
Convert your images to avi format via
convert *.ppm %03d.png
ffmpeg -f image2 -r 8 -i %03d.png input.avi
-
Convert your movie to swf format via
ffmpeg -i input.avi -r 6000 output.swf
The value of 6000 can be changed depending on the desired speed of
your movie (the higher the value, the faster the movie).
-
Call the movie like
\begin{frame}{1D FDTD Simulation}
\begin{center}
\includemedia[activate=onclick, width=\linewidth,height=0.6\linewidth]{}{wave.swf}
\end{center}
\end{frame}
inside your Latex source code (see
Media 9 Package )
-
Compile the latex file as usual using
pdflatexfull
-
View the pdf file using
acroread
which has to be version 9.4.1 under Unix/Linux.
You can copy the template of this type of presentation
from the cluster
/software/FDTDstandard/latexslidemovie
-
In order to produce the movie,
copy files
/software/FDTDstandard/TadashiMursoftMPI/output_FDTD-2d-xy-3-*.f90
/software/FDTDstandard/TadashiMursoftMPI/output_FDTD-2d-yz-3-*.f90
/software/FDTDstandard/TadashiMursoftMPI/output_FDTD-2d-xz-3-*.f90
/software/FDTDstandard/TadashiMursoftMPI/qualitymovieshell
/software/FDTDstandard/TadashiMursoftMPI/movieshell
/software/FDTDstandard/TadashiMursoftMPI/showshell
/software/FDTDstandard/TadashiMursoftMPI/pgm2ppm.f
into the directory of your FDTD code.
qualitymovieshell is the 2D image movie production and
it needs pgm2ppm which is from pgm2ppm.f (gfortran -O4 -o pgm2ppm
pgm2ppm.f )
movieshell is the low-quality but quick movie production
in 2D. showshell is the quick movie production in 1D.
-
If you need to produce a movie in gif format, just run
convert -resize 200x400 -delay 0.2 *.ppm test.gif
To BSc and MSc Students
Practical skills such as the operating system, programming and documentation
language knowledge are not going to be forced by me unless your work has a
significant impact on the mainstream of our research.
In this case, you can learn Unix, Fortran, LaTeX and shell-scripting including the command-line utilities as
awk , sed , tgif and gnuplot independently.
If you import some image into tgif environment,
what you need to do to produce a pdf file is
- In
tgif environment,file -> print/export format->select EPSI
- file -> print
- In a x-terminal,
epstopdf something.eps
For students in EEE, the risk assessment form is
assessment
form.
You need to change the name of the student to your name obviously.
As for the technical writing, please see
LaTeX section.
For MSc students, the binding information is in
handbook
and
guideline
For BSc students, the Latex style files for final report is in
3rd year UG Latex Template.
Accommodation Hunting
Via the Accommodation Office at the University. This is the expensive but easiest option. You just need to go to the Office in person. It is located in the University Place building on Oxford road. Ask people for directions. University owned halls of residence are well maintained and provide central location and pastoral support. Also in halls you will have an opportunity to meat people from all over the world. Another benefit of the University hall is that the hall contract is a package deal. You will not have to worry about the Internet, water and electricity bills. Everything is provided by the University without any headache. But the drawback of the University hall might be a slightly higher price and
noise by other residential students.
List of all university halls with prices:
http://www.manchester.ac.uk/undergraduate/accommodation/list/
Pick the city centre location. It is closer to our office. George Kenyon Hall, Grosvenor Street Building, Opal Hall, Bowden Court, Ronson Hall and Grosvenor Place are recommended.
Manchester Student Homes. Privately owned, but University recommended accommodation. The University checks the flats and landlords before suggesting their flats and rooms to students. This option is relatively safe, but you might need a fried to share a flat or house with. Also you will have to sign contracts with electricity, water and internet provider companies. This might require some time to arrange. You may see the Manchester Student Homes offers online:
http://www.manchesterstudenthomes.com/
Student Village Manchester. Private company providing accommodation in central location, but possibly cheaper than the University-owned halls. They also provide packaged (all inclusive) contracts. No separate contracts for water, electricity and internet.
http://www.mcrstudents.com/
Gumtree. A universal website to advertise flat or room rentals. It is unpredictable. You can find a good deal, but also get into trouble with an unpleasant landlord:
http://manchester.gumtree.com/
International Society of the Univeristy. You may visit the Society and look on their notice board for flat or room rentals. Also ask at the counter. These are very friendly people. There might be many international students looking for a flatmate there.
http://www.internationalsociety.org.uk/
Fumie's availability
Until 9th of May, 2014 (i.e., till the end of 2nd semester
(27/Jan-9/May) )
Fumie is not available at the following time slots:
Mondays: -10:00, 17:00-
Tuesdays: Whole day long
Wednesdays: -10:00, 17:00-
Thursdays : -10:00, 17:00-
Fridays : Whole day long
Please do respect these time slots as
Fumie also needs some time for her
teaching and research.
During the week commencing on 3/Mar/2014
Fumie is not available during the following timeslots:
Monday:
Tuesday:
Wednesday:10:00-13:00
Thursday : whole day long
Friday : whole day long
|