Setting up AWS EC2 with docker#

Attach and format harddisk#

Refer to Guide Ubuntu. Skip the formatting step if your harddisk already has data.

####### Formatting #######
# Check disk
sudo fdisk -l #For AWS, /dev/sdb will become/dev/xvdb. Check with fdisk
sudo parted -l #Double check
# Create partition
sudo parted /dev/xvdb
(parted) mklabel gpt
(parted) mkpart primary ext4 0% 100%
quit
sudo mkfs.ext4 /dev/xvdb
#mount /dev/xvdb [target_folder]
#sudo chown -R $(id -u):$(id -g) [target_folder]

Mount on startup

vi /etc/fstab

#
# /etc/fstab
# Created by anaconda on Sun Aug  4 04:28:13 2019
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
/dev/xvdb     /mnt/data     ext4        defaults      0       0

Install nextflow#

curl -s "https://get.sdkman.io" | bash #Install sdk
source "$HOME/.sdkman/bin/sdkman-init.sh"
sdk install java 17.0.6-amzn #Install java
wget -qO- https://get.nextflow.io | bash #Install nextflow

Install docker#

Refer to guide.

# Install docker (yum method)
sudo yum update
sudo yum search docker #search for docker package
sudo yum info docker #get version info
sudo yum install docker
# Set up user so root is not required
sudo usermod -a -G docker ec2-user
id ec2-user
newgrp docker
# Enable socker service at boot time
sudo systemctl enable docker.service
sudo systemctl start docker.service
# Check docker status
sudo systemctl status docker.service

Install Programs#

#Install git
sudo yum install git

Transfer file#

Source

rsync -avzn -e "ssh -i /path/to/key.pem" /path/to/file.txt  <username>@<ip/domain>:/path/to/directory/
rsync -avzn -e "ssh -i /path/to/key.pem" <username>@<ip/domain>:/path/to/directory/file.txt  /path/to/directory/

Load from AMI#

Instead of installing docker, nextflow etc., load from AMI instead.

AMI name Jon-Docker-Kraken-Nextflow
AMI ID ami-0ab649c42b9a41556