#ThisWeekILearned: What the heck are all these AWS acronyms

If you’re like me and have depended on the magic of Heroku to deploy applications, this article is for you. I learnt a few things about AWS this week and I hope to share these learnings in this article. Specifically, I’ll share a high-level overview of: (1) What does Amazon Web Services (AWS) do, (2) what the AWS alphabet soup (EC2, ECS, ECR, IAM, what what?) mean and what they do (3) how to get started with AWS and (4) how to not accidentally incur insane charges when learning how to use AWS.

In this post, I prioritise brevity over details since the article is intended to give someone who’s new to AWS (like me) a clearer picture of what it is, not scare them with even more details.

In my next post, I hope to write about Terraform, which is an awesome way of provisioning AWS services by writing code, rather than clicking buttons in the Amazon Console. Why this is important? Terraform allows us to provision AWS services using code, and thereby allowing us to (1) have version control (e.g. git), (2) have consistent configurations across all our machines and avoid the nightmares associated with snowflake servers, and (3) scale! Imagine if you had to provision 40 servers — would you prefer clicking around 400+ times, or would you prefer to run one command? More on that later. But first, let’s get some context of what AWS is and how you can make it work for you.

What does AWS do?

AWS provides a suite of cloud computing services that enables on-demand delivery of compute power, database storage, applications, and other IT resources through a cloud services platform via the internet with pay-as-you-go pricing.

Amazon hosts their server farms in 14 regions, and each region has multiple Availability Zones (which is basically a data center) to ensure 99.999999999% availability. There are 38 availability zone in total. For me in Singapore, my region is ap-southeast-1 and my Availability Zones are ap-southeast-1a and ap-southeast-1b. When running projects on AWS, pick the AWS region closest to you / your users.

AWS Regions

What do all these acronyms stand for?

The alphabet soup of acronyms (e.g. EC2, S3, IAM, SQS) are basically different services that do different things (I am less vague when I elaborate on what some of these key services do later in the article). I find it less daunting when I think of them in groups:

I found it even less daunting when I heard someone say that it’s not possible to know everything (that’s really comforting to know!). For us to be effective developers/devops people, it would be good (dare I say sufficient?) to know/master the following categories of services:

• Compute. This category of services is responsible for running your application. Under this category, you’ll find services such as EC2 (Elastic Compute Cloud) which allows you to spin up Virtual Machines (VMs) for running your application. This is where you choose the right amount of CPU, memory, storage, and networking capacity to ensure that your application scales accordingly. Next, ECS (EC2 Container Service) is a container management service for deploying containers (e.g. Docker). Elastic Beanstalk is where the magic happens — you just need to upload your application, and Elastic Beanstalk will provision the server automatically; Lambda is a compute service that lets you run code (in an event-driven way) without provisioning or managing servers. You can see common use cases of AWS Lambda here.
• Storage. AWS offers 4 services for 4 kinds of storage. S3 (Simple Storage Service) allows you to create buckets to store files (e.g. word files, images, presentations, etc). EFS (Elastic file service) allows you to store and install databases and applications, and you can share this volume with multiple VMs. Glacier is a cheaper service for archiving files. Storage Gateway is a way of connecting S3 to your (non-AWS) server farms.
• Databases. AWS offers 4 database services: RDS (Relational Database Service) for relational databases such as MySQL, Postgres & PostgresQL, SQLServer, and Oracle. DynamoDB for non-relational (NoSQL) databases. Redshift for data warehousing. It copies your data to another database where you can run your analyses (Because running data analysis on your production database can slow down your production database). ElastiCache is a web service that for deploying, operating, and scaling an in-memory data store or cache.
• Networking and Content Delivery: This category includes services such as Virtual Private Cloud (VPC) (the docs explain it better than I can), Route53, which is a Domain Name System (DNS) web service where you can purchase your own domain name and configure how you want user requests to be routed to the your AWS infrastructure (e.g. to EC2 instances, Elastic load balancers, or S3 buckets). CloudFront is a service for caching assets (e.g. videos or large media files)
• Security and Identity. There are many aspects to security, but I just want to touch on one of the coolest things I’ve learnt. AWS uses IAM (Identity Access Management) to ensure that only the people with the right credentials can do what they’re supposed to do on your AWS account. The general pattern is this: For any service (e.g. EC2, or S3), you create user(s), assign them permission(s) (e.g. Admin, read-only, etc) either by directly specifying what each user can do, or by creating a group and specifying its permissions, and then adding the user(s) to the group. Otherwise, you would be using these services as the root user (the sudo guy), which is dangerous because anyone who somehow gets your credentials can do anything and everything on your AWS account as the root user. IAM prevents that. AWS also has a Certificate Manager, where you can get free SSL certificates for your domain names.
• Management tools (e.g. monitoring, logging). Cloud Watch is a service that allows you to monitor performance of your AWS environment (e.g. EC2 RAM / CPU / HD utilisation). Cloud Formation offers a way to read and write your AWS infrastructure as code. There are Cloud Formation templates that you can use to provision an entire production environment. In one single command line command, you can provision 50 servers.
• Messaging. SNS (Simple notification service) allows you to easily push real-time notification messages to interested subscribers over multiple delivery protocols (e.g. HTTPS). SES (Simple Email Service) is a service for receiving and sending emails from AWS. SQS (Simple Queue System) is a service that stores messages as they travel between applications or microservices. SQS moves data between distributed application components and helps you decouple these components. For example, if you have a application that creates memes, if 100 users make a request to create a meme, SQS will create a queue of jobs. If your EC2 instance dies, the jobs will still be stored in the queue and it will resume when another EC2 instance is spun up.

Getting started with AWS

To get your hands dirty and actually launch some AWS services on the internet, I highly recommended AWS’ getting started guides (especially this one) and Udemy’s Certified Solutions Architect course. Be prepared for a dependency tree when going through the AWS guides (i.e. to do step X1, I first have to do steps W1–5, and to do step W3, I first have to do steps U1–6). Personally I found getting started to be a pain, but once your account is set up, you’ll start noticing patterns that repeat across services and it gets easier over time.

Also you can install the AWS CLI so that you can access AWS programmatically (i.e. via your shell terminal, instead of via the AWS GUI Console).

1. Open up your Terminal, and run brew install awscli
2. To enable autocomplete of aws commands, run source /usr/local/bin/aws_zsh_completer.sh
3. To make the console output more readable, run aws configure and skip the first 3 fields by hitting Enter (You can fill in your access keys and region later) and when it prompts you for “Default Output Format”, enter json. This will json-prettify AWS CLI’s output and make it more readable
4. Run aws help. This gives you a good overview of what options and commands are available to you. The general pattern of commands is:aws <aws-service-short-hand> <command> <subcommand> [parameters]
5. To see what commands are available to you in each service (e.g. S3, EC2, etc), you can run aws <aws-service-short-hand> help (e.g. aws s3 help, or aws ec2 help).

How to not accidentally incur insane charges when learning how to use AWS.

While I knew that most popular AWS services contain free-tier options that I can try out for free while learning AWS, I couldn’t help feeling a latent fear that I will suddenly get some massive bill because I accidentally launched some super expensive server or because someone hacked into my account. There are two things you can do to prevent this from happening so that you can continue to learn without fear:

1. Set up billing alarm (instructions here). You can configure AWS to notify you as soon as your AWS bill goes above a certain amount (e.g. $10). I set mine to $5.
2. Set up Multi-Factor Authentication (instructions here). AWS allows you to use multi-factor authentication and your mobile phone can become a second layer of security for your AWS account. I was speaking with a DevOps guru last week and he suggested that MFA is a must-do for our own sake, so that if someone were to somehow get your password (e.g. through CloudBleed), they still won’t be able to log in to your AWS account without your mobile phone. I already use Google authenticator for my Gmail and Facebook, and adding AWS was a (10-minute) breeze.

I hope this is helpful! Your feedback, suggestions and comments are welcome! Please post them in the comments section below :-)