Quick Start

List of Commands for experienced HPC Users

• Programmer Notices

  Unlike the TACC Lonestar system, your code can contain system() and fork()/exec() calls because the Ranger Linux kernel is 2.6.18, and MVAPICH MPI memory routines work correctly with the fork/system/exec calls.

  Do not use the qsub "-S" option to set the shell for batch jobs. Set it with the "shebang" #! line shell invocation (#!/bin/<shell>).

• Suggested default Intel 10 compiler options (version 9.1 is also available):

  ifort	-xW -O2 -ipo prog.f90
  icc	-xW -O2 -ipo prog.c

  mpif90	-xW -O2 -ipo prog.f90
  mpicc	-xW -O2 -ipo prog.c

  Also, try using -xO for SSE3 instructions (not available for version 9.1).

• Suggested default PGI compiler options:

  pgf95	-fast -tp barcelona-64 prog.f90
  pgcc or pgcpp	-fast -tp barcelona-64 prog.c

  mpif90	-fast -tp barcelona-64 prog.f90
  mpicc	-fast -tp barcelona-64 prog.c

• Suggested default gcc compiler options:

  gcc	-O3 -mtune=barcelona -march=barcelona prog.c
  f77	-O3 -mtune=barcelona -march=barcelona prog.f

• Job Script

  #!/bin/tcsh
  #$ -V	# Inherit the submission environment
  #$ -cwd	# Start job in submission directory
  #$ -N myjob	# Job Name
  #$ -j y	# Combine stderr & stdout into stdout
  #$ -o $JOB_NAME.o$JOB_ID	# Name of the output file (eg. myMPI.oJobID)
  #$ -pe 16way 128	# Requests 16 cores/node, 128 cores total
  #$ -q normal	# Queue name
  #$ -l h_rt=00:30:00	# Run time (hh:mm:ss) - 30 min.
  ## -M myemailname@address	# Email notification address (UNCOMMENT)
  ## -m be	# Email at Begin/End of job (UNCOMMENT)
  set echo	# {echo cmds, use "set echo" in csh}
  ibrun ./a.out	# Run the MPI executable named "a.out"

• Job Submission

  qsub job

• Job monitoring, job deletion, queue list, job-run info

  qstat	{info for your jobs; use -u "*" (with quotes) for all users}
  showq	{system-wide job summaries}
  qdel <jobid>	{kill a job}
  qconf -sql	{show queue list}

• File Systems. Variable contains your directory name.

  Directory	Limit (space, inodes)	Access	Policy	Availability
  $HOME	6GB, 100K	Lustre (parallel)	permanent, backed up	all nodes
  $WORK	4TB, 2M	Lustre (parallel)	no purge, no backup	all nodes
  $SCRATCH	400TB, 10M	Lustre (parallel)	10-day purge, no backup	all nodes
  $ARCHIVE	unlimited	DMF archive	Disk-to-Tape Migration	** login nodes

  Use the following command to view your usage and quotas in $HOME, work and scratch, respectively:

  lfs quota -u <username> /share	{/share is the $HOME file system.}
  lfs quota -u <username> /work	{/work is the $WORK file system.}
  lfs quota -u <username> /scratch	{/scratch is the $SCRATCH file system.}

  ** If you need to transfer files from the TACC Archive:

  See the new archive document.