HPC
HPC

| HPC and Data for Lattice QCD

caos

HPC and Data for Lattice QCD

caos

Using 'caos' manually

hs 8/2001 For further documentation see also /kroot/doc/HOWTO-caos.html In the following $CAOS stands for the default 'caos' (currently /kroot/bin/caos), shell is tcsh.

Kernel Drivers:

  - PB driver is in $CAOS/EGADI/ampbdrv22/
  - RB driver is in $CAOS/EGADI/amrbdrv22/

Starting Slaves: (on corresponding unit)

  - cd $CAOS/WORK
  - ./slave "name"  &
  - ./root  "name"  &

where in the 1st case "name" is one of "slave0","slave1",... referring to UID and in 2nd case "name" is one of "pbroot0","pbroot1" referring to CID and this process must only be started on the host connected to the root board. Please note to start slave in background!

Running caos: (on apemaster)

  - cd $CAOS/EGADI
  - source egadi-env
  - cd <run directory> 
  - caos <options> <jex file>
where <options> typically is something like
    -C unit 0 -H -j 0x88 -f 0x2001
       ^^^^^^		
       or any other machine configuration
       like unit 1 or board 5	
[Alternatively, running the Master (on apemaster):
  - cd $CAOS/EGADI
  - source egadi-env
  - $CAOS/EGADI/bin/master.pl
This is almost the same as running caos, but some configuration options are different:
    master.pl -T PB <TID> <CID> <UID> <BID> 

is equivalent to
    caos -C board <global_BID>
Note that this is not the recommended way for using caos.]

Machine configurations:

The mapping between host names and logical names ("slave0" etc.) is defined in the file:
	$CAOS/EGADI/pm/Share/APEMsg.pm
To run on a specific sub-system of the machine, caos needs the global coordinates with respect a virutal 2-crate machine with the following numbering:
   crate	                       0	 
   unit	                  1            0            2            3
   board	 7  6  5  4   3  2  1  0  11 10  9  8  15 14 13 12
   crate	                       1
   unit		          5            4            6            7
   board        23 22 21 20  19 18 17 16  27 26 25 24  31 30 29 28
Running on a single PB:
	caos -C board <BID>       	(e.g. caos -C board 27)
Running on a single Unit:
	caos -C unit <UID>		(e.g. caos -C unit 1)
Running on a single Crate:
	caos -C crate <CID>       	(e.g. caos -C crate 1)
Running interactively (!)

        caos -C <structure> -i          (includes taco-compatible behavior)  

Warning

Make sure the rootmask of the rootboard(s) is cleared before starting caos jobs! ( to clear mask: /zroot/tools/rootmask 0x000000) An improperly set rootmask (e.g., from a previous job with "krun" ) might cause problems using caos on I/O.

Interactive mode

Caos commands can be executed either interactively as
	caos -C <configuration> -i
	(prompts for caos commands)

or using command files as
	caos -C <configuration> -p <commandfile>
Note: Unix pipes or input-redirection for STDIN do not (yet) work.

Taco-like Syntax:

  o Write access to devices:

       w <device> <nodes> <addr> <num> : <data>

  o Read access to devices:
       r <device> <nodes> <addr> <num>
  o Formats:

      <device> can be:
	      ar	Altera Register
	      tr	Tz Register
	      td	Tz Data Memory
	      tp	Tz Program Memory
	      jr	Jn Register     
	      jd	Jn Data Memory   
	      jp	Jn Program Memory
      <nodes> are specified as:
	      all			all nodes
	      n			node specified by node_abs_id
	      [x,y,z]			node specified by triple of 
				      node_abs_x, node_abs_y, node_abs_z
	      [x1,y1,z1][x2,y2,z2]	slice specified by two triples of
				      node_abs_x, node_abs_y, node_abs_z
	      def			default node
      <addr> is a decimal or 0x-prefixed hex value. For Jn Data Memory
      it refers to 32-bit words while for Program Memories it refers 
      to 96-bit words.

      <num> is a decimal or 0x-prefixed hex value and always refers 
      to the number of 32-bit words (in particular for Program Memories 
      and Jn Data Memory).
      <data> can be a space- or newline-separated list of decimal or 
      0x-prefixed hex values. For Program Memories, the LSW comes before
      the MSW (in the spirit of a 32-bit word sequence).