This article shows how to submit a simple OGS (SGE) job to the cloud using the Bright CMSUB command.
Let's get started.
Here's the job script we'll use. It's a standard OGS job script with some added #CMSUB directives. These #CMSUB directives are ignored by the shell, and by OGS, but they provide a convenient way to pass arguments to the Bright CMSUB command.
#!/bin/sh
#$ -N ogs-job.sh
#$ -j yes
#$ -cwd
#$ -S /bin/bash
#CMSUB --input=/home/rstober/ogs/cloud/test.txt
#CMSUB --output=/home/rstober/ogs/cloud/analysis.output
#CMSUB --regions=us-west-1
#CMSUB --arguments="-q cloud"
#CMSUB --wlmanager=sge
date >> test.txt
echo "This job is running on node: `hostname`"
echo
echo "In the directory: `pwd`"
echo
echo "As the user: `whoami`"
echo "this is a file created by the job" > analysis.output
echo
echo "Here is a listing of the current directory:"
ls -l
echo
echo "This is the input file upload by cmsub, and modified by this job:"
echo
cat test.txt
sleep 30
Submit the job using the Bright cmsub command.
[rstober@atom-head1 cloud]$ cmsub ogs-job.sh
Upload job id: 75
User job id: 76
Download job id: 77
The qstat command shows the current status of the available SGE queues and the jobs associated with the queues. When called without arguments, qstat displays a list of running and pending jobs without queue status information.
[rstober@atom-head1 ogs]$ qstat
job-ID prior name user state submit/start at queue slots ja-task-ID
-----------------------------------------------------------------------------------------------------------------
75 0.00000 ogs-job.sh rstober qw 01/21/2013 10:44:05 1
76 0.00000 ogs-job.sh rstober hqw 01/21/2013 10:44:05 1
77 0.00000 ogs-job.sh rstober hqw 01/21/2013 10:44:05 1
The qstat -f option causes summary information on all queues to be displayed along with the queued job list.
[rstober@atom-head1 ogs]$ qstat -f
queuename qtype resv/used/tot. load_avg arch states
---------------------------------------------------------------------------------
all.q@atom01.cm.cluster BIP 0/0/4 0.00 linux-x64
---------------------------------------------------------------------------------
all.q@atom02.cm.cluster BIP 0/0/4 0.00 linux-x64
---------------------------------------------------------------------------------
all.q@atom03.cm.cluster BIP 0/0/4 0.00 linux-x64
---------------------------------------------------------------------------------
all.q@atom04.cm.cluster BIP 0/0/4 -NA- -NA- au
---------------------------------------------------------------------------------
cloud@cnode1.cm.cluster BI 0/1/4 0.09 linux-x64
76 0.55500 ogs-job.sh rstober r 01/21/2013 10:44:31 1
---------------------------------------------------------------------------------
cloudtransfers@master.cm.clust BI 0/0/1 0.28 linux-x64
############################################################################
- PENDING JOBS - PENDING JOBS - PENDING JOBS - PENDING JOBS - PENDING JOBS
############################################################################
77 0.00000 ogs-job.sh rstober hqw 01/21/2013 10:44:05 1
qstat -j provides detailed information about the specified job.
[rstober@atom-head1 ogs]$ qstat -j 76
==============================================================
job_number: 76
exec_file: job_scripts/76
submission_time: Mon Jan 21 10:44:05 2013
owner: rstober
uid: 1001
group: rstober
gid: 1001
sge_o_home: /home/rstober
sge_o_log_name: rstober
sge_o_path: /cm/shared/apps/gcc/4.7.0/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/sbin:/usr/sbin:/cm/shared/apps/openmpi/gcc/64/1.4.5/bin:/cm/local/apps/cluster-tools/bin:/cm/shared/apps/slurm/2.3.4/bin:/cm/shared/apps/slurm/2.3.4/sbin:/home/rstober/bin:/cm/shared/apps/sge/2011.11/bin/linux-x64
sge_o_shell: /bin/bash
sge_o_workdir: /home/rstober/ogs
sge_o_host: atom-head1
account: sge
cwd: /home/rstober/ogs
merge: y
hard resource_list: cloudregion=us-west-1
mail_list: rstober@master.cm.cluster
notify: FALSE
job_name: ogs-job.sh
jobshare: 0
hard_queue_list: cloud
shell_list: NONE:/bin/bash
env_list:
script_file: /tmp/tmpd62nmm.job
jid_predecessor_list (req): 75
jid_successor_list: 77
usage 1: cpu=00:00:00, mem=0.00000 GBs, io=0.00000, vmem=N/A, maxvmem=N/A
scheduling info: queue instance "all.q@atom04.cm.cluster" dropped because it is temporarily not available
We're done.
