Contents
Introduction
Objective
The objective of this project is to demonstrate the working
of Amazon ECS by deploying a simple nodeJS app and scaling it.
Goal
The goal of this project is fully demonstrate the usage of
ECS container service to deploy a web app but using docker containers. The
tutorial aims to give insight on the simplicity of scaling a web app when using
Amazon ECS.
Amazon ecs overview
Amazon EC2 Container Service (Amazon ECS) is a highly
scalable, fast, container management service that makes it easy to run, stop,
and manage Docker containers on a cluster of Amazon Elastic Compute Cloud
(Amazon EC2) instances. Amazon ECS lets you launch and stop container-based
applications with simple API calls, allows you to get the state of your cluster
from a centralized service, and gives you access to many familiar Amazon EC2
features.
benefits of using ecs
Some of the benefits of using ec2 are:
_
Elastic web-scale computing
_
Completely controlled
_
No additional charges
_
Supports docker-compose style yaml files
_
Multiple web apps under same cluster
ECS Features
ECS, just like EC2, is a regional service that leverages the
underlying technology of docker and makes it simpler to deploy applications
that span multiple availability zones in a highly scalable manner.
A brief overview of ECS architecture is as bellows:
This diagram gives a well-defined view of how ECS feature
work together.
The following sections dive into these individual
elements of the Amazon ECS architecture in more detail.
CONTAINERS and images
ECS makes
use of docker as core underlying technology to drive the deployment engine and
mechanism using docker containers and images. A container is like a running
instance of app whereas an image is immutable form of container, when its not
started yet.
To deploy
applications on Amazon ECS, your application components must be architected to
run in containers. A Docker container is a standardized unit of software
development, containing everything that your software application needs to run.
Images are
typically built from a Dockerfile, a plain text file that specifies all of the
components that are included in the container. These images are then stored in
a registry from which they can be downloaded and run on your container
instances.
Task definitions
Task
definitions are textual description of the actual tasks that need to run for
your webapp. You define various parameters in a YAML or JSON file about what
docker images its going to use, what ports should be open, what data volumes
need to be mounted etc. To prepare your
application to run on Amazon ECS, you create a task definition. The task
definition is a text file in JSON format that describes one or more containers
that form your application. It can be thought of as a blueprint for your
application.
Clusters
Clusers are
a logical grouping of EC2 instances, where you running tasks are placed by ECS.
An application images are downloaded from whatever the registry specified to
the EC2 instances within the cluster and containers are then started from those
images.
tasks and scheduling
ECS task
scheduler is responsible for placing tasks on container instances. A task is
simply an instantiated, running form of task definition on a container
instance.
There are
several different scheduling options available. For example, you can define a
service that runs and maintains a specified number of tasks simultaneously.
container agent
A container
agent is a process that runs on each instance with the cluster and is
responsible for sending information to ECS about our tasks. It runs in the form
of a container and is responsible for sending information received from client
about start/stop of tasks.
Demonstration of Working application
In this
demo, we will show you how to deploy a scalable,load-balanced and high
available simple nodejs app. The demo
will have multiple instances of nodejs app each running inside a container and
load-balanced through haproxy.
The basic
architecture looks like following:
The app as a whole consists of four components that will be
deployed:
_
Haproxy(for load-balancing)
_
App(Simple node JS app that prints out the port
it started on)
_
MondoDB as backend
The demo will first walk-through setting ECS cluster from
Web UI of AWS and later how to use ECS CLI.
i) Creating a ECS Cluster:
First step
in deploying any web application to create an ECS cluster from aws console.
This is a fairly simple step as show in picture below:
ii) Install & Configure ECS CLI:
Now that a
bare minimum cluster is created its time to use ECS CLI. ECS CLI is a command
line tool that you can use with given credentials to manage you cluster and
prepare to run tasks and services on it.
Installing the ECS CLI:
Configure the ECS-CLI with credentials:
ecs-cli configure --region us-west-2 --access-key <key> --secret-key <secret> --cluster cs6320
INFO[0000] Saved ECS CLI configuration for cluster (cs6320)
iii)
Creating the ECS Cluster from CLI:
iv) Use a docker-compose YAML file for task definition:
Declaring tasks can be done from the Console or CLI. ECS
CLI has a neat feature of supporting docker-compose , which are docker native
way of declaring services. We avoided the hassle by reusing same compose based
definitions as ECS CLI already supported them
The compose for our app looks like:
version: '2'
version: '2'
services:
mongo:
build:
mongo
image:
hayderimran7/mongo:cs6320
container_name:
mongo
volumes:
-
/tmp/mongo_logs:/logs
-
/tmp/mongo_data:/data
ports:
-
"27017:27017"
haproxy:
build:
haproxy
image:
hayderimran7/haproxy:cs6320.4
container_name: haproxy
volumes:
-
/tmp/haproxy_logs:/logs
ports:
-
"80:80"
redis:
build:
redis
image:
hayderimran7/redis:cs6320
container_name: redis
volumes:
-
/tmp/redis_data:/data
ports:
-
"6379:6379"
app1:
build: app
image:
hayderimran7/app:cs6320
container_name: app1
ports:
-
"81:80"
links:
-
haproxy
app2:
build: app
image:
hayderimran7/app:cs6320
container_name: app2
ports:
-
"82:80"
links:
-
haproxy
app3:
build: app
image:
hayderimran7/app:cs6320
container_name: app3
ports:
- "83:80"
links:
-
haproxy
app4:
build: app
image:
hayderimran7/app:cs6320
container_name: app4
ports:
-
"84:80"
links:
-
haproxy
v) Start a
task and services from YAML file:
Now is the time to finally start our webapp from the
compose file.
Once that’s successful, you will see the status using ecs
cli.
vi)
Verify cluster status and containers:
Now we verify the status of containers by running “ps”
command from ecs-cli as:
Name
State Ports TaskDefinition
a48573dc-7961-4d9a-afb5-8b24f1aec12f/redis RUNNING
54.202.144.143:6379->6379/tcp ecscompose-cs6320:2
a48573dc-7961-4d9a-afb5-8b24f1aec12f/app3 RUNNING
54.202.144.143:83->80/tcp
ecscompose-cs6320:2
a48573dc-7961-4d9a-afb5-8b24f1aec12f/app2 RUNNING
54.202.144.143:82->80/tcp
ecscompose-cs6320:2
a48573dc-7961-4d9a-afb5-8b24f1aec12f/haproxy RUNNING
54.202.144.143:80->80/tcp
ecscompose-cs6320:2
a48573dc-7961-4d9a-afb5-8b24f1aec12f/mongo RUNNING
54.202.144.143:27017->27017/tcp
ecscompose-cs6320:2
a48573dc-7961-4d9a-afb5-8b24f1aec12f/app1 RUNNING
54.202.144.143:81->80/tcp
ecscompose-cs6320:2
vii)
Verify services and task from Console:
We can also verify that all services are up and cluster
status from console.
viii)
Scaling to multiple web apps from ECS CLI:
Scaling a webapp to multiple instances is simply one
command. Lets say we want to scale to 5 , we can then issue the command as:
ecs-cli compose --file docker-compose.yml --project-name cs6320_1 scale 5
ecs-cli compose --file docker-compose.yml --project-name cs6320_1 scale 5
Resources and Links:
The tutorial was possible due to several helpful
resources we found online as cited below:
[1]. ECS CLI Tutorial http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_CLI_tutorial.html
Synopsis: this is the official link to how to run ecs
cli.
Reliability: Its from official AWS so its reliable
[2]. Amazon ECS Overview
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html
Synopsis: this is the official link about the overview of
ECS
Reliability: Its from official AWS so its reliable
[2]. Amazon ECS Overview
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html
Synopsis: this is the official link about the overview of
ECS
Reliability: Its from official AWS so its reliable
Technologies Used
We have made use of
following technologies for this project
1.1
NodeJS
1.2
Amazon ECS
1.3
Docker
1.4
Docker-compose
1.5
Mongo
