AWS

The instructions to set up an AWS instance here is adapted from EECS 485’s tutorial.

Create an account

Create an AWS account at the AWS Registration. You should be eligible for their free tier, which means that you will be able to run an instance for free for the duration of the course.

Despite that, you will need to enter a credit card number on the account, even if you only use free tier resources. This is how Amazon charges, in case you request more resources than provided in the free tier. Do not launch any additional instances other than the one we specify in order to avoid additional charges.

Optionally, you may redeem extra benefits as a student, including $100 in AWS credits.

Start instance

Navigate to the AWS Management Console. Select the “Services” dropdown menu, then “EC2”. An EC2 “instance” is a virtual machine running on Amazon AWS hardware.

Click launch an instance. It may take a few minutes to initialize before it’s running.

Select the “Ubuntu Server 24.04 LTS” Amazon Machine Image (AMI).

Choose instance

As discussed in the “Hosting cost” section of the spec, for this course, we assume you are running a free-tier backend and will provide only instructions to set up a free-tier backend, which can only run the tinyllama LLM model. If you want to run other LLM models, choose the VM instance with enough resources to host your model of choice. VM instances with more resources cost more.

Select the “t2.micro” or “t3.micro” instance type. You should see “free tier eligible”. When you create an instance, AWS automatically creates user “ubuntu” for you on the instance.

Setup security

Create a key pair for user “ubuntu”. The private key will be automatically downloaded for you once the key pair is created. You’ll use this private key later to ssh to the instance. Click on Create new key pair:

Then enter “reactive” as the key pair name and click Create key pair:

Allow SSH, HTTP, and HTTPS and click Edit on the top right:

Verify if three security group rules have been added. After this, Click on Add security group rule.

Edit the “Security group rule 4 (TCP, 0)” section, and add “Custom TCP” for port 8000 to allow server traffic in and out of your instance. [Thanks to Z. Liang and R. Nagavardhan ‘23.]

Configure storage

Students have reported that 10GB is not enough to install Ollama. AWS free tier allows up to 30 GB EBS storage per month. However, this 30GB is across all your instances, and you cannot shrink a volume once created, you can only expand it. If you already have a free AWS instance in another course or project, you will likely be billed if your total is above 30 GB.

Set your storage to 30 GB:

Startup Script

At the bottom of the page, click Advanced details.

Scroll down to find User data - optional.

Set up a 4GB swapfile so that your server can survive load spikes. Paste the following script:

#! /usr/bin/env bash

fallocate -l 4G /swapfile  # free -h to see allocated size
chmod 600 /swapfile
mkswap /swapfile
swapon /swapfile

Please keep the box of User data has already been base64 encoded unchecked.

Click “Launch Instance”.

Instance status

Navigate to the AWS Management Console. Select the “Services” dropdown menu, then “EC2”.

Select one of the instances and view its status and Public DNS.

In the remainder of this spec, and this term, we will refer to your “IPv4 Public IP” as YOUR_SERVER_IP (in the image it’s 52.90.192.32).

ssh to instance

On your development host (laptop):

If AWS gave you a reactive.cer instead of reactive.pem, just use reactive.cer everywhere you see reactive.pem in this spec.

MacOS on Terminal:

#         👇👇👇👇👇👇👇👇👇👇
laptop$ cd YOUR*LABS*FOLDER
laptop$ mv ~/Downloads/reactive.pem reactive.pem
laptop$ chmod 400 reactive.pem
laptop$ ssh -i reactive.pem ubuntu@YOUR_SERVER_IP
#                                 👆👆👆👆👆👆👆👆👆

Windows on PowerShell [thanks to Jad Beydoun (F21) for use of icacls]:

#            👇👇👇👇👇👇👇👇👇👇
PS laptop> cd YOUR*LABS*FOLDER
PS laptop> mv ~\Downloads\reactive.pem reactive.pem
PS laptop> icacls reactive.pem /grant "$($env:username):(r)" /inheritance:r
PS laptop> ssh -i reactive.pem ubuntu@YOUR_SERVER_IP
#                                    👆👆👆👆👆👆👆👆👆

  laptop$ sudo vi /etc/wsl.conf

  [automount]
  options = "metadata"

  #          👇👇👇👇👇👇👇👇👇
  laptop$ cd YOUR*LABS*FOLDER
  #                       👇👇👇👇👇👇👇👇👇👇👇👇👇
  laptop$ mv /mnt/c/Users/YOUR_WINDOWS_USERNAME/Downloads/reactive.pem reactive.pem
  laptop$ chmod 400 reactive.pem
  laptop$ ssh -i reactive.pem ubuntu@YOUR_SERVER_IP
  #                                 👆👆👆👆👆👆👆👆👆

In both cases, what the above does:

1. change working directory to YOUR*LABS*FOLDER,
2. move the private ssh key you created and downloaded earlier into YOUR*LABS*FOLDER,
3. set its permissions to read-only, and
4. ssh to your AWS instance as user “ubuntu” using the downloaded private key. (Make sure your instance is running. See Instance status.)

Persistent swapfile

To mount the 4G swapfile we created above persistently after each reboot, do:

server$ sudo su
server# echo '/swapfile none swap sw 0 0' >> /etc/fstab
server# exit

Increasing the disk size

If you have set up your instance with less than the 30 GB suggested above and got an error message that you don’t have enough storage, to increase your storage, go to the EC2 Dashboard, select “Volumes” under “Elastic Block Store”. Select the volume attached to your instance, and click “Actions > Modify Volume”. Note that there’s an upper limit of 30 GB of total free-tier storage across all your AWS instances.

Enter a larger size, e.g., 20 GB, for the volume and click “Modify”. You may need to wait a few minutes for the modification to take effect.

Once the volume size modified, you also need to resize the partition and filesystem inside the VM: ssh to your sever, then check your storage devices (disk, partitions, and logical volumes).

server$ lsblk

From the list of block devices, look for xvda or nvme0n1. This is your disk storage and it should show the modified size (20GB) of your volume. Your root partition would be listed as xvda1 or nvme0n1p1. Expand your root partition (xvda1 or nvme0n1p1) to use all of the free space on the volume (xvda or nvme0n1):

server$ sudo growpart /dev/xvda 1A  # or /dev/nvme0n1
server$ sudo resize2fs /dev/xvda1  # or /dev/nvme0n1p1

The above commands grow the root partition and resize the filesystem.

You can verify the updated partition size by:

server$ df -h

You should see /dev/root mounted at / and that its usable space is as you’ve set.

Stop instance

server$ sudo apt install cloud-utils awscli

server$ ec2metadata
server$ aws configure
server$ aws ec2 help

server$ aws iam list-access-keys

server$ aws ec2 authorize-security-group-ingress --group-id YOUR_GROUP_ID --ip-permissions IpProtocol=tcp,FromPort=8000,ToPort=8000,Ipv6Ranges=[{CidrIpv6=::/0}]