AWSLAB — Lab Infrastructure Automation using AWS CloudFormation

This project aims to provide users with hands-on training on various AWS services. A platform for enthusiasts to perform hands-on labs designed for different architectures. AWS CloudFormation is used to automate the deployment of AWS resources or services in the backend. When a lab is started, all the resources/services required for the specific lab are provisioned automatically, and thus the user can easily complete the lab by just following the provided step-by-step instructions.

Here, is a snippet of what our project looks like.

AWS CloudFormation

AWS CloudFormation is a service used to implement Infrastructure as a code in a cloud environment. It provides a simpler way for you to create, model, and provision AWS resources and third-party applications in an automated and efficient manner. It eliminates the need and time required to manage those resources individually rather, lets you focus on your infrastructure designing. Thus, AWS CloudFormation simplifies your infrastructure management by replicating your infrastructure quickly and provides an ease of controllability over your resources.

A stack is basically a collection of AWS resources and is responsible for the creation or deletion of the infrastructure. Also, it automatically manages the dependencies between your resources. So you don’t need to worry about the specific order in which resources are created or deleted as CloudFormation itself determines the correct sequence while performing stack operations. You are authorized to manage and provision stacks across multiple AWS accounts and AWS Regions.

AWS CloudFormation allows you to model your entire cloud environment in form of text files or templates. It also provides a better way of visualizing these templates as a diagram with the help of a graphic designer tool — AWS CloudFormation Designer. It helps you with editing, modifying, and updating templates using the drag and drop interface in a controlled and predictable way.

To begin with AWS CloudFormation, you need to code your infrastructure from scratch and create your own template in either of the open-source declarative languages, YAML or JSON format, or you can simply use any of the sample AWS CloudFormation templates available for different infrastructures.

AWS CloudFormation Template

A CloudFormation template describes all your desired resources and their dependencies so you can launch and configure them together as a stack. These templates can be used to create, update, or delete an entire stack. AWS CloudFormation constructs and configures those stack resources that you have specified in your template. Here, the following syntax shows the AWS CloudFormation template structure and their sections in both JSON and YAML format respectively.

JSON Format

{
  "AWSTemplateFormatVersion" : "version date",
  "Description" : "JSON string",
  "Parameters" : {
    set of parameters
  },
  "Mappings" : {
    set of mappings
  },
  "Resources" : {
    set of resources
  },
  "Outputs" : {
    set of outputs
  }
}

YAML Format

AWSTemplateFormatVersion: "version date"
Description: string
Parameters:
  set of parameters
Mappings:
  set of mappings
Resources:
  set of resources
Outputs:
  set of outputs

Here’s a short explanation of what each of the important sections of AWS CloudFormation Templates means:

AWSTemplateFormatVersion: Specifies the AWS CloudFormation template version.

Description: Any comment that describes the template can be specified in the description section.

Metadata: Metadata can be used in the template to provide further information using JSON or YAML objects.

Parameters: Templates can be customized using parameters. Each time you create or update your stack, parameters help you give your template custom values at runtime.

Mappings: Mapping enables you to map keys to a corresponding named value that you specify in a conditional parameter.

Conditions: In a template, conditions are the statements used to perform certain actions when the statement is true.

Resources: Using this section, you can specify the stack resources and their properties that you want to create and specify in the stack, such as an Amazon S3 bucket. The Resources section is the only required section in the CloudFormation template.

Outputs: The output section describes the output values that are returned whenever you view your stack’s properties. This gets displayed in the AWS CloudFormation Console.

Snippet of a Sample Template just for your reference

AWSTemplateFormatVersion: 2010-09-09
Parameters:
  NameOfService:
    Default: RDS
    Description: "The name of service this stack is used for."
    Type: String
  KeyName:
    Description: Name of existing EC2 KeyPair 
    Type: AWS::EC2::KeyPair::KeyName
  DBName: 
    Default: MyDatabase
    Description: "The database name"
    Type: String
    MinLength: 1
    MaxLength: 64
    AllowedPattern: "[a-zA-Z][a-zA-Z0-9]*"
    ConstraintDescription: "contain only alphanumeric characters."
Mappings:
  RegionMap:
    us-east-1:
      AMI: ami-1853ac65
    us-west-1:
      AMI: ami-bf5548df
    eu-west-1:
      AMI: ami-3bfab942
    ap-southeast-1:
      AMI: ami-e2adf99e
    ap-southeast-2:
      AMI: ami-43874721
    ap-south-1:
      AMI: ami-0e306788ff2473ccb
Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16
      EnableDnsHostnames: true
      Tags:
      - Key: Name
        Value: Lab VPC
  MyDBSubnetGroup: 
    Type: AWS::RDS::DBSubnetGroup
    Properties: 
      DBSubnetGroupName: DBSubnetGroup
      DBSubnetGroupDescription: "Subnets for the RDS DB Instance"
      SubnetIds: 
      - !Ref PrivateSubnet1
      - !Ref PrivateSubnet2
  DBInstance: 
    Type: AWS::RDS::DBInstance
    Properties:
      DBName: !Ref DBName
      AllocatedStorage: !Ref DBAllocatedStorage
      DBInstanceClass: !Ref DBClass
      Engine: MySQL
      EngineVersion: 8.0.20
      MasterUsername: !Ref DBUsername
      MasterUserPassword: !Ref DBPassword
      MultiAZ: false
      Port: 3306
      PubliclyAccessible: false
      DBSubnetGroupName: !Ref MyDBSubnetGroup
      DBSecurityGroups: 
      - !Ref DBSecurityGroup
Outputs:
  RDSInstanceId:
    Description: "Instance Id of RDS Instance"
    Value: !Ref DBInstance

AWS CLI

The AWS Command Line Interface (CLI) is an open-source tool used for managing your AWS services from a terminal session on your own client, allowing you to interact and configure multiple AWS services. In this blog, you can learn how to implement AWS CloudFormation templates using AWS CLI.

Installation of AWS CLI

To begin with AWS CLI, you need to first download the AWS CLI. Depending on your operating system, it will require a different method. Hereby, I have mentioned the steps for 64-bit Windows.

STEP 1. Download the 64-bit Windows MSI installer from the following link

https://awscli.amazonaws.com/AWSCLIV2.msi

STEP 2. Run the downloaded MSI installer and follow the installation instructions

Once you are done with all the steps, to confirm the proper installation, use the aws –version command at the command prompt window.

Configuration of AWS CLI

Once you have successfully installed the AWS CLI, you now need to configure it so as to connect to your AWS account. Thus, to do so enter the following command in your command prompt:

aws configure

The AWS CLI outputs lines of text, prompting you to enter four additional information. The first two are the required ones — the AWS Access Key ID and the AWS Secret Access Key. These credentials are used to authenticate your AWS account and can be generated within AWS Identity and Access Management (IAM) from your AWS Console. The other two pieces of information are the AWS region name and the output format (JSON or YAML), which could be of your own choice. For the time being, you can leave them as default. Below is a snapshot of the following command.

Command to configure AWS CLI

Using AWS CLI for AWS CloudFormation Stacks

With the AWS Command Line Interface, you can create, monitor, update, and delete stacks from your system’s terminal. To create an AWS CloudFormation stack, you need to run the aws cloudformation create-stack command. Wherein, you need to provide the stack name, the location of a valid template, and the input parameters. The following create-stack command creates a stack with the name mystack using the template.json template.

aws cloudformation create-stack --stack-name mystack --template-body file://template.json --parameters ParameterKey=KeyPairName,ParameterValue=TestKey

The output for the above command would be in the specified output format you had configured before.

To delete a stack, you run the aws Cloudformation delete-stack command. Wherein, you specify the name of the stack that you want to delete. While deleting the stack, you automatically delete all the stack resources as well. The following delete-stack command deletes the specified stack named mystack.

aws cloudformation delete-stack --stack-name mystack

This delete-stack command produces no output.

Implementation of the project

Now, learn about the implementation of the above knowledge in order to execute this project.

Authentication for AWSLAB — AWS Cognito

For authentication purposes, I have used the service — AWS Cognito with a User Pool. Amazon Cognito is a user authentication service that enables user sign-up and sign-in, and access control for mobile and web applications, easily, quickly, and securely. AWS Cognito ties itself to an authentication directory, where user account data is stored, without needing to monitor or manage the underlying infrastructure. Alternatively, you can federate AWS Cognito, so the actual authentication is performed by social media authentication engines like Google, Facebook, etc. Hereby, I have provided a federated authentication provider — Google as an addition.

Authentication for AWSLAB

Architecture of AWSLAB

After a successful sign-in, users are welcomed to AWSLAB. On the Homepage, we have provided numerous labs for hands-on practice on different AWS Services. The UI for the homepage is kept simple and effective using a grid layout. On clicking ‘View Lab’, the user will be re-directed to the instructions page for the selected lab.

Homepage of AWSLAB — Hands-On Training

On the instructions page, one can find detailed instructions for completing the lab successfully. From starting the lab to ending the lab, all minute steps are covered in the instructions. Also, a quick brief of all the tasks is provided in the task menu. To begin with the lab, one needs to click on the ‘Start Lab’ button, then by clicking on the ‘AWS Console’ button, one will be directed to the AWS sign-in page, wherein the user needs to sign-in to the AWS console using the credentials provided on the instructions page.

Instructions Page of Lab 3 — Introduction to Amazon Relational Database Service (Linux)

Wondering, how exactly this works?

Basically, on clicking the ‘Start Lab’ button, a CloudFormation template designed specifically for that particular lab is launched in the backend. AWS CloudFormation templates are designed in such a way, that all the necessary resources for that lab are provisioned automatically by just launching those templates. Similarly, on clicking the ‘End Lab’ button, the launched CloudFormation template will be deleted instantly and thus all the created resources will be deleted.

For instance, in this particular lab ‘Introduction to Amazon Relational Database Service (Linux)’, as you can view the below snapshots, this is how necessary resources like VPC along with subnets and route tables, Ec2 instance, RDS security group, etc. are automatically created when the template for that lab is launched. Thus, it eliminates the time required for launching those resources individually and simply provides a way to just focus on the main task.

Creation and deletion of Stack along with all the resources

“Cloud is about how you do computing, not where you do computing.” ~ Paul Maritz, CEO of VMware