Storage

HPC and Data for Lattice QCD

Storage

Basic rules for storage

Disk space is available for two different kinds of storage:

 - pseudo-temporary storage for running jobs
 - permanent storage

In addition there is a tape robot for long-term storage of data. Please visit dCache and OSM for details.

1) Temporary storage

Each apemaster machine has a *local* /data/ disk intended for temporary storage of data for running jobs of the corresponding crates, units or boards:

 apemaster5:/data/   nc[67]
 apeamster6:/data/   nc[45]
 apemaster7:/data/   nc[23]
 apemaster8:/data/   nc[01]
 apemaster9:/data/   c[0123]
 apemaster10:/data/  c[4567] u8[02] b8[13]0

On each of these local data disks you can create your own directory.

NOTE: This disk space is not in the backup! It will definitely not be appreciated if this disk space is used for other purposes except for storing job input and output data.

NOTE: We expect files on these disks not to be older than 1 monts! Older files may be deleted without further notice in case disks get full!

2) Permanent storage

Each project has its own project file space in the AFS. Please check with the corresponding administrators on how to use this disk space. Note that not all of the AFS space is backuped!

Project	Path	Coordinator
Alpha	/afs/ifh.de/group/alpha	Rainer Sommer
ETMC	/afs/ifh.de/group/nic	Karl Jansen
QCDSF	/afs/ifh.de/group/qcdsf	James Zanotti

Recommendation

Jobs should be started from the AFS disk space. Input files could be copied by the job script from the AFS space to the local /data file system at the beginning of each job using afscp and vice versa after job execution on APEmille finished:

 #!/usr/bin/zsh 
 #$ -V 
 #$ -cwd

 source /zroot/klogin.zsh

 JOBDIR=$PWD
 TMPDIR=/data/my_name

 # Copy files from job directory to temporary run directory
 for file in *; do
    afscp $file $TMPDIR || exit 1
 done

 # Change to temporary directory and execute job
 cd $TMPDIR

 APErun -- -o yz my_program.jex || exit 1

 # Copy result files back to job directory
 for file in *; do
     afscp $file $JOBDIR || exit 1
 done
 cd $JOBDIR
 
 # Remove temporary directory
 rm -rf $TMPDIR