Getting Started#
This guide contains the basic codes to get you started on NGS on the cluster at GIS. It is based on the following guides:
Account#
Apply for an account by emailing GIS Scientific Support and Shared Services scientificsupport@gis.a-star.edu.sg or sharedservices@gis.a-star.edu.sg. You will need to complete a questionnaire process.
Once completed, you will receive an email with log in information and instructions:
Disk Space#
Request for space from shared services or from members in the group. The group is allocated a total space, and each project under the group is also allocated a fixed amount of space. Since the space allocation overlaps, it is unclear which one sets the space limit.
To check the amount space available, use the code df -h .
.
Research folder#
To access the group's storage (not cluster) disk, use smb://nlsmb.gis.a-star.edu.sg
and log in with your GIS username and password.
Location of metasub sequencing filenames.
smb://nlsmb.gis.a-star.edu.sg/Research/CSB/CSB5/Eliza/Food microbiome 2019/MetaSub/MetaSub Analysis
Start up script#
You can configure bash startup scripts in either ~/.bashrc
or ~/.bash_profile
.
VPN#
GlobalConnect#
- Connect to vpn server at vpngw.gis.a-star.edu.sg
- Log in using your A*STAR ID and Password
OpenConnect#
- View hipreport file from GlobalConnect under the troubleshoot setting (PanGPS.log).
- Download, compile and install openconnect for your platform
- (For Linux) download the vpnc script and run
./configure --with-vpnc-script=[location of vpnc-script] make sudo make install
- Edit
/trojan/hipreport.sh
- Update the field DOMAIN, COMPUTER, and HOSTID
- Update the field
<entry name="anti-malware">
and<network-interface>
- Run the command below
- (For Linux) download the vpnc script and run
sudo openconnect -v --protocol=gp vpngw.gis.a-star.edu.sg --csd-wrapper=./trojans/hipreport2.sh
Cluster Access#
ssh <username>@<node>.gis.a-star.edu.sg
<username>
is the username given by GIS. It will typically start with your last name. <nodes>
include aquila
, aquilaln2
, and ionode
.
Aquila
is the login node and should not be used to run jobs or processes. To do that, (submit jobs)[##submitting_jobs] to other nodes via qsub
or open an interactive session as follow:
qrsh -l h_rt=23:59:59 -l mem_free=8G -pe OpenMP 4 -q interactive.q
Change h_rt
and mem_free
depending on your own needs.
SFTP#
Download filezilla or any other file manager.
sftp <username>@ionode.gis.a-star.edu.sg
X Forwarding / Graphical Interface#
To enable X forwarding on a mac, download xquartz.
To enable X forwarding on a windows using WSL, refer to this. Substitute localhost
for 127.0.0.1
instead.
Set up ssh configuration file accordingly as follow:
~/.ssh/config
/etc/ssh/ssh_config
Run
# -Y enables graphical interaction over ssh
ssh -Y <username>@<node>.gis.a-star.edu.sg
# Once logged in test with
echo $DISPLAY
xterm
xclock
Submitting Jobs#
On the cluster, check out /opt/uge-8.1.7p3/examples/jobs/array_submitter.sh and /opt/uge-8.1.7p3/examples/jobs/jobnet_submitter.sh for examples on how to submit jobs.
For a simple way to submit jobs, use
# standard multithread
qsub -pe OpenMP 4 -l h_rt=23:59:59 -l mem_free=16G SCRIPT.sh
# Add env var PATH to the job
qsub -pe OpenMP 4 -l h_rt=23:59:59 -l mem_free=16G -v PATH SCRIPT.sh
# Note: If a command runs in interactive mode but not when submitted as a job, it is likely due to differences in environment.
# To copy the environment over when submitting a job, use `-V`.
qsub -pe OpenMP 4 -l h_rt=23:59:59 -l mem_free=16G -V SCRIPT.sh
# Add a name to the job
qsub -N $NAME_OF_THE_JOB -pe OpenMP 4 -l h_rt=23:59:59 -l mem_free=16G -V SCRIPT.sh
# Send input output jobs to ionode.q specifically
qsub -q ionode.q -N $NAME_OF_THE_JOB -pe OpenMP 1 -l h_rt=23:59:59 -l mem_free=16G -V SCRIPT.sh
# Add log files
qsub -q ionode.q -N $NAME_OF_THE_JOB -pe OpenMP 1 -l h_rt=23:59:59 -l mem_free=16G -V -e $DIR/log -o $DIR/log SCRIPT.sh
# Submitting through echo STDIN instead of using a file
echo "commands1; comamnds2; commands3" | qsub -N $NAME_OF_THE_JOB -pe OpenMP 4 -l h_rt=23:59:59 -l mem_free=16G -V -e $DIR/log -o $DIR/log
Input/Output#
- Use the ionode instead of aquila to transfer large amounts of data out of GIS server.
- Method 1 - Create a script using
scp
with a ssh key to transfer file. Since the submitted job does not allow us to input a password when submitting a job, create a passphraseless private/public key instead. Use the-i
option to select the private key. - Method 2 - Create a script using
rsync
with a ssh key to transfer file. Use passphraseless private/public key again to log in without a password. - Submit a job.
- Remove the private key after
# transfer.sh
## Method 1 - scp
scp -r -i [private key file] [source directory or file] [target]:[target directory]
## Method 2 - rsync
rsync -a [source directory or file] [target]:[target directory]
rsync -anv [source directory or file] [target]:[target directory] #to check files that'll be transferred
rsync --log-file [file] --stats --sizeonly -a [source directory or file] [target]:[target directory]
#### Examples
#Transfer only *fq.gz
rsync -e "ssh -i $HOME/.ssh/jonai" -anvm --include="*.fq.gz" --include="*/" --exclude="*" jon@jonai.teojy.com:/media/JonData/NEA/data/processed /home/users/astar/gis/teojyj/scratch/nea/data # --include="*/" needed for recurssion
# To rsync a list of files, use --files-from. Add r for recursive
rsync -ar --stats --size-only --log-file ~/transfer_general.rsync.log --files-from=isolate_temp.txt jon@jonai.teojy.com:/media/JonData/CRE/data/raw/isolate_reads /home/teojyj/nagarajan_pool/CRE/DATA/ISOLATE_TEMP
qsub -q ionode.q -N transfer_files -pe OpenMP 1 -l h_rt=95:59:59 -l mem_free=16G -V transfer.sh
Troubleshooting#
Log files are incredibly important to troubleshoot issues with your code. Below are a few tips that I use.
- When submitting a job, add the options the
-o
and-e
options to write the stdout and stderr streams into their respective log files. - If a job failed but stdout and stderr didn't provide any clues, use to check the exit code.
qacct -j <job-id>