CI/CD using AWS Code Deploy
In this post, I explain how to use the Jenkins open-source automation server to deploy with AWS CodeDeploy and creating a functioning CI/CD pipeline. When properly implemented, the CI/CD pipeline is triggered by code changes pushed to your GitHub repo, automatically fed into Jenkins then the application is deployed using CodeDeploy. As an added feature, one can auto-scale the application and also put the whole setup behind a load balancer for failover benefits.
Such people are in high demand in the IT industry. Because of their wide array of skill sets (coding + cloud based knowledge) they have an upper hand in the market as compared to conventional developers.
In today’s DevOps age, constant code revision is a part of developmental practice. To take advantage of changing times it is beneficial to bring in some DevOps tools and methodology to help in this regard. This article will help your journey down that path easier with complete end-to-end automation.
Challenges:
1) Manually work to run test cases and pull the latest code on all the servers.
2) Since the servers were auto scaled we had to pull the latest code on one server , take image of that server, re configure it with Auto Scaling.
3) Roll back to last successful pull was a painful task.
Though the above challenges are not the blockers but we have to bring up a technique that must save high amount of time and make our life easy with automating all the process from CI to CD.
We choose to use Jenkins as CI tool and AWS Code Deploy as a CD tool.
Lets walk through the flow before we implement it all.
When a new code is pushed to a particular GIT repo branch.
1) Jenkins will run the test cases, (Jenkins listening to a particular branch through git web hooks )
2) If the test cases fail. It will notify us and stop the further after build actions.
3) If the test cases are successful , it will go to post build action and trigger aws code deploy.
4) Jenkins will push the latest code in the zip file format to AWS S3 on the account we specify.(Optional for code backup purpose)
5) AWS Code Deploy will pull the code in all the Auto Scaled servers that have been mentioned.
6) Once the latest code is copied to the application folder , it will once again run the test cases.
7) If the test cases fail it will roll back the deployment to previous successful revision.
8) If it is successful , it will run post deployment build commands on server and ensure that latest deployment does not fail.
Now lets follow above steps and configure everything from scratch.
Part-1: GitHub: For Source Code Management You can clone the below mentioned repository on your machine for testing out this mechanism.
Part-2: https://www.digitalocean.com/community/tutorials/how-to-install-jenkins-on-ubuntu-18-04 For installing Jenkins on Ubuntu-18.04 machine refer this link.
After installing jenkins we have to unlock jenkins using initial admin password. you can find the password here /var/lib/jenkins/secrets/initialAdminPassword
On the Customize Jenkins page, choose Install suggested plugins.
Wait until Jenkins installs all the suggested plugins. When the process completes, you should see the check marks alongside all of the installed plugins.
- On the Create First Admin User page, enter a user name, password, full name, and email address of the Jenkins user.
- Choose Save and continue, Save and finish, and Start using Jenkins.
After you install all the needed Jenkins plugins along with their required dependencies, the Jenkins server restarts. This step should take about two minutes. After Jenkins restarts, refresh the page. Your Jenkins server should be ready to use.
Create a project and configure the CodeDeploy Jenkins plugin
Now, to create our project in Jenkins we need to configure the required Jenkins plugin.
- Sign in to Jenkins with the user name and password that you created earlier and click on Manage Jenkins then Manage Plugins.
2. From the Available tab search for and select the below plugins then choose Install without restart:
AWS CodeDeploy Plugin
GitHub Plugin.
3. Select the Restart Jenkins when installation is complete and no jobs are running.
Jenkins will take couple of minutes to download the plugins along with their dependencies then will restart
4. Login then choose New Item, Freestyle project.
5. Enter a name for the project (for example, CodeDeployApp), and choose OK.
6. On the project configuration page, under Source Code Management, choose Git. For Repository URL, enter the URL of your GitHub repository with Credentials.
7. In the Build Triggers options you can select Build when a change is pushed to GitHub
What this will do is whenever a new update is pushed to the specified branch in my case */feature/cicd, Jenkins will automatically get triggered, pull the latest updates from the git repo and perform build actions.
The Build option contains build steps, in my case i have selected Execute Shell.
The Command box contains a shell script saved in my jenkins home directory.
This script contains commands to run test case for the application.
If the test case fail then Jenkins would not go further to deploy or won’t perform any further actions and report as Build Failed.
In other case if the test case run successful then you are done Continuous Integrating.
Jenkins will go to next step that is Continuous Deployment.
8. Now for CD, under Add Build Step, select Deploy an application to AWS Code Deploy
In AWS CodeDeploy Application Name : Fill in your AWS CodeDeploy application name that you have configured before.
In AWS CodeDeploy Deployment Group : Fill in your AWS CodeDeploy Deployment Group name that you have configured before.
In AWS CodeDeploy Deployment Config : Paste the Deployment Config you have selected in application configuration. In my case CodeDeployDefault.OneAtATime
In the AWS Regions option : Choose your specific region for your sufficient.
In the S3 Bucket : Enter your S3 bucket name.
In the S3 Prefix : Enter your directory name under the S3 bucket
Keep everything else to default.
Except you need to give your aws credentials
Fill in you Secret Key and Access Key
11. Choose Deploy Revision. This option registers the newly created revision to your CodeDeploy application and gets it ready for deployment.
12. Select the Wait for deployment to finish? check box. This option allows you to view the CodeDeploy deployments logs and events on your Jenkins server console output.
SETUP GITHUB AND WEBHOOK
13. To finish setting up the integration and deployment
Go to the GitHub repo, and click Settings. Click the Webhooks & Services tab, and then the Add service drop-down. Select the Jenkins (GitHub plugin) service.
Click Add service. Our project is now ready for its first continuous integration test!
Finally Save your project.
Part-3: CONFIGURE AWS CodeDeploy Application:
I assume you have all the access required to configure AWS CD
- We need to configure 2 IAM Roles that would be used to boot your ec2 server / access S3 and then for Code Deploy to talk to ec2 server.
We will name them CodeDeploy and EC2CodeDeploy
To create the role go to IAM Service select Roles -> Create New Role
Enter the name CodeDeploy and select Next
Select Amazon EC2 as Role Type and select Next
Under Set Permissions choose Custom Policy and select
Enter the name CodeDeploy and paste the below policy.
CodeDeploy Policy:
Select Next and Create Role. This show the list of roles created.
Now we need to select CodeDeploy and select Edit Trust Relationship
Replace the policy with below policy
Policy Trust for CodeDeploy
and Update Trust Policy.
2. Now you need to follow the above steps to create EC2CodeDeploy policy
Paste the below Policy.
No need to edit the Trust Policy.
3. Now you need to launch Ubuntu or Amazon Linux (contains aws cli pre installed) or any other instance, in my case Ubuntu attached with IAM role EC2CodeDeploy and install aws cli with aws code deploy agent.
To Install CodeDeploy agent follow below steps
Considering you have installed aws cli
aws configure : Fill in Secret Key, Access Key and Region.
aws s3 cp s3://aws-codedeploy-us-east-1/latest/install . — region [As u wish]
chmod +x ./install
sed -i “s/sleep(.*)/sleep(10)/” install
./install auto
service codedeploy-agent status
Remember that you need to place the appspec.yml file in your application repo.
here is one sample appspec.yml file
The cleanup-instance.sh script files contains all the commands that needs to be done before the code is deployed to the application folder.
In my case I create a folder that will just take a backup of some files.
install-dependencies.sh file contains commands that will run some test cases to check if the application is deployed properly.
4. Now we will create AWS CodeDeploy Application
Go to AWS CodeDeploy -> Get Started -> Custom Deployment
This will bring you to Create New Application page.
Fill in the Application Name and Deployment Group Name
The next interesting part is selecting the instances where you want to deploy the application.
In Add Instances part you can search and select your Instances or AutoScaling Group by tag names.
Create Deployment
Here we select the instance that we had launched.
Specify your revision type either S3 or GitHub — Select GitHub since ours is stored in GitHub. Enter your github credentials and connect to github
Next Select the kind of load balancer you want to setup. I have set Application Load Balancer which is sufficient for our purposes.
Next
Deployment Config , here we can set it to One at a Time since we have only one server for now.
Service Role , here we attach the tole CodeDeploy Role that we had created.
Finally select Create Application.
Here we finish with configuring AWS CodeDeploy.
Testing the whole CI/CD pipeline
To test the whole solution, put an application on your GitHub repository. You can download the sample from https://github.com/Yuvaraj421/code-deploy-chatapp
The following screenshot shows an application tree containing the application source files, including text and binary files, executables, and packages:
In this example, the application files are the templates directory, test_app.py file, and web.py file.
The appspec.yml file is the main application specification file telling CodeDeploy how to deploy your application. Jenkins uses the AppSpec file to manage each deployment as a series of lifecycle event “hooks”, as defined in the file. For information about how to create a well-formed AppSpec file, see AWS CodeDeploy AppSpec File Reference.
The buildspec.yml file is a collection of build commands and related settings, in YAML format, that CodeBuild uses to run a build. You can include a build spec as part of the source code, or you can define a build spec when you create a build project.
The scripts folder contains the scripts that you would like to run during the CodeDeploy LifecycleHooks execution with respect to your application requirements.
Conclusion
In this post, I outlined how you can use a Jenkins open-source automation server to deploy with CodeDeploy. I showed you how to construct a functioning CI/CD pipeline with these tools. I walked you through how to build the deployment infrastructure and automatically deploy application version changes from GitHub to your production environment.
Hopefully, you have found this post informative and the proposed solution useful.
Feel free to share your feedback
About the Author
Yuvaraj Yadav is a DevOps Engineer at Teqfocus. He is a subject matter expert for AWS CodeDeploy and Jenkins. In his role, he enjoys supporting customers with their CodeDeploy and other DevOps configurations. Outside of work he enjoys Swimming and Chilling out with his pals.
