SG++
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
python.uq.uq_setting.remote_worker Namespace Reference

Functions

def choose_host
 
def dist_main
 
def do_dist_run
 
def free_host
 

Variables

dictionary hosts
 
string username = "franzefn"
 

Detailed Description

Distribution for shared-disk systems. Code/Working directory should live on a
Filesystem visible from any host (e.g. NFS-home)

The UQSetting need to have an attribute 'setupCommand' which is a string
with python code. The code should load a UQSetting with proper
Simulation set.

Function Documentation

def python.uq.uq_setting.remote_worker.choose_host ( )
selects a host randomly. Every host is replicated
with respect to the number of cores it has.
allslots = []
for h, props in hosts.iteritems():
    #allslots = allslots + [h] * props['cores']
    allslots = allslots + [h] * props['free_cores']

while True:
    if not allslots == []:
        host = random.choice(allslots)
        # hosts[host] = hosts[host] - 1
        hosts[host]['free_cores'] = hosts[host]['free_cores'] - 1
        sleep(2)  # do not kill the server
        return host
    else:
        sleep(2)  # wait for cores, that may become free
def python.uq.uq_setting.remote_worker.dist_main (   pwd,
  setup,
  uq_in,
  uq_out,
  samples,
  starti,
  host,
  scratch_path,
  sample_number 
)
Main program for the worker.
@param pwd: string, working directory
@param setup: string, python command that builds new UQSetting
@param uq_in:
@param uq_out:
@param samples:
@param starti:
def python.uq.uq_setting.remote_worker.do_dist_run (   host,
  uqsetting,
  outfile,
  samplelist,
  starti,
  scratch_path,
  sample_number 
)
Calculate the given samples on a remote machine.
exec's, call after fork'ing. (Does not return)
@param host: string, name of the host
@param uqsetting: UQSetting
@param outfile: string, location where the UQSetting of the result is
                 stored
@param samplelist: Samples, which samples should be run
@param starti: int

References python.uq.operations.general.join().

def python.uq.uq_setting.remote_worker.free_host (   host)

Variable Documentation

dictionary python.uq.uq_setting.remote_worker.hosts
Initial value:
1 = {
2 # 'localhost': {'cores': 4,
3 # 'free_cores': 2,
4 # 'scratch': '/data2/scratch/Fabian/localhost'},
5 # 'helium': {'cores': 36, # actually 42 cores, but I don't use all of them
6 # 'free_cores' : 4,
7 # 'scratch': '/data2/scratch/%s/helium' % username},
8  'kepler': {'cores': 24,
9  'free_cores' : 24,
10  'scratch': '/scratch/sgs/%s' % username}
11  #'vgpu1': {'cores': 12,
12  # 'free_cores' : 12,
13  # 'scratch': '/home/%s/data2/scratch/%s/vgpu1' % (username, username)}
14  # 'vgpu2': 12,
15  # 'pcsgs05': 3,
16  # 'pcsgs02': 4,
17  # 'pcsgs03': 2,
18  # 'pcsgs04': 4,
19  }
string python.uq.uq_setting.remote_worker.username = "franzefn"