Texas State University



Texas State University: erwin.tr.txstate.edu

The Schrödinger system is composed of 17 nodes or hosts, each of which is composed of 2 Xeon E5540 central processing units (CPU's) and each CPU contains 4 cores. The host node, erwin.tr.txstate.edu, is dedicated to administration and is the primary node to which all users will login. Ordinarily, a user (schrödinger, say) is set up by staff and the user logs in with their Texas State netID and password. Their home directory, /home/schrödinger, is mounted on all nodes simultaneously and is accessible from any node. The home directory is backed up daily. Schrödinger is normally accessible by ssh (secure shell) login. The accompanying ssh commands, scp(secure copy) and sftp(secure ftp(file transfer protocol)), are the preferred method of access and datafile transfer from other desktops or laptops.

Introduction

Prologue:

Note: ssh, sftp and scp are normally installed and available in any Mac or Linux OS(operating system). In Windows, Putty is a free software source and Qvt-Term is a licensed to Texas State faculty, staff and student source of these clients.

The compute nodes, locally named compute-01-xx where xx varies from 01 to 16, are the preferred execution and compilation hosts as the software libraries are primarily stored on them. All of the nodes are running a Red Hat Linux OS with gridengine queue scheduling.

When erwin_user logs in to erwin.tr.txstate.edu, the user is placed in a shell environment, that is a terminal window into which shell commands are typed. Typing the command

echo $SHELL

followed by pressing the enter/return key will initially return

/bin/bash

indicating the bash login shell. Typing

echo $PATH

and typing enter/return will show the directories which are searched for commands. The echo command is found in /bin/echo. If the PATH environment variable is not set correctly, ordinarily one could use the full pathname /bin/echo for execution. The set or env command will list the environment variables associated with the current shell. Command lines will initially be prefaced with

[erwin_user@erwin ~]$

whose format is set by the PS1 environment variable. Echoing $PS1 will show its current setting. These and other settings are controlled by system login files and personalized by files in your home directory. The command

ls -a

will list .bashrc and .bash_profile text files whose contents serve this purpose. You can use cat

cat .bashrc

or less

less .bashrc

to display these files (press the space bar to continue and q to quit when using less). Help can be found in the man pages. For instance,

man ls

or

man echo

or

man less

will display descriptions of the commands using a less-like context. You will ordinarily need to use one of the available text file editors. The command

man -k editor

will list a selection( man man shows you that this is equivalent to the command

apropos editor

and

man apropos

will give the description “apropos searches a set of database files containing short descriptions of system commands for keywords and displays the result on the standard output”). A few more quick notes, the command

apropos editor | less

will pipe (the name of the vertical bar key in unix systems) the textual output of the apropos command into the less command as if that output were a file. The command

apropos editor > editorlist

will create a file called editorlist (assuming you have write privileges in the current directory) from the output of the apropos command which you can peruse with an editor or show with less,

less editorlist

The command

ls -al

will show (the option l implies long, option a implies all files (i.e., including those starting with dots)) the permissions of the current directory, designated by a dot, the parent directory, designated by two dots and the rest of the files. The chmod command is used to control these permissions if you have permission to use it.

Use the command syntax

export ENV_VARIABLE=”text_here”

in a bash or bash-like shell and

setenv ENV_VARIABLE “text_here”

in csh or a csh-like shell. The manual (man command output) pages for some of these commands may be inside the shell man pages. To find an export description for the bash shell, issue the command

man bash

and type

/export

and the page will immediately move to the first occurrence of the string “export”. Typing

/

repeatedly will move the page to following occurrences of the previous search string. You will eventually find the page which contains

export [-fn] [name[=word]] ...

export -p

and descriptions.

Using the Parallel Environment:

The

qstat -g c

command lists the available queues, all.q, bigmem.q, blades.q and gpu.q. The queue, all.q, is the default queue, accesses all nodes and has a time limit of 15 minutes. Nominally you can choose from 1 to 616 cores, but the diverse set of nodes makes using all of them effectively in any particular job relatively rare. Thus the qstat -g c command shows a bigmem queue for accessing compute-2-1 and compute-2-2, each of which has 40 cores and 1TB of memory. The gpu queue access compute-3-01 and compute-3-02, each of which is loaded with the cuda environment and 2 M2090 Nvidia GPU's. If you feel you have an unusual job case please contact us via the contact page. The qsub command is the preferred entrance into the SGE scheduling system. A submit example is shown below.

#!/bin/bash

#

#$ -cwd . #current working directory

#$ -N srtest #job name

#$ -j yes #merge stderr with stdout

#$ -S /bin/bash #shell

#$ -q all.q #queue

#$ -pe orte 12 #parallel environment and number of processes

#$ -M erwin_user@txstate.edu #mail address for job results

#$ -m as #mailing circumstances

#$ -R yes #reserve (place) in queue

#

#

. /etc/profile.d/modules.sh #setup modules

#

module load mpi/openmpi-mellanox-interconnects-gcc #load openmpi environment

#

#mpicc -O2 c_coded_file.c #source of a.out file

mpirun -np $NSLOTS ./a.out #mpi start and run command

These lines will be in a file, call it jobfile.qsub, and submitted with the command

qsub < jobfile.qsub

The

man qsub

command will show the description of the options. The file a.out is the default output of the mpicc compiler command and the -O2 option is a typical gcc optimization choice. You must use the mpicc wrapper instead of gcc in order to most easily access the mpi libraries. This script assumes that the file, c_coded_file.c and its compiled result a.out, exists in the current working working directory which is specified with the dot above. You must be aware that 16 cores exist on each node filesystem. Thus parallel jobs whose cores use separate files must use unique filenames on the same node in the same filetree. Home directories are shared as is among all hosts/nodes.

Available modules are shown by the

module avail

command, not all of which may be active. These modules will undergo changes as we adapt our library and compiler environment. We have the the Intel compilation environment available, as seen in the modules list, to better optimize execution.

Please let us know if you have software and/or environment requests that do not fit in the present framework.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download