aws-senior.com

www.aws-senior.com
Visite out website www.aws-senior.com
http://www.aws-senior.com
www.aws-senior.com
AWS Certified Cloud Practitioner
AWS Certified Solutions Architect Associate
AWS Certified Developer Associate
AWS Certified SysOps Administrator Associate
AWS Certified Solutions Architect Professional
AWS Certified DevOps Engineer Professional
AWS Certified Big Data Specialty
AWS Certified Advanced Networking Specialty
AWS Certified Security Specialty www.aws-senior.com
/ http://www.aws-senior.com
aws-senior.com
www.aws-senior.com
Visite out website www.aws-senior.com
www.aws-senior.com Automating your software build is an important step to adopt [120]DevOps best practices. To help you with that, we built [121]AWS CodeBuild, a fully managed continuous integration service that compiles source code, runs tests, and produces packages that are ready for deployment, and [122]AWS CodePipeline, a fully managed continuous delivery service to automate your release pipelines.

However, there are so many possible customizations in our customersâ€™ build processes, and we have seen developers spend time in creating their own custom workflows to coordinate the different activities required by their software build. For example, you may want to run, or not, some tests, or skip [123]static analysis of your code when you need to deploy a quick fix. Depending on the results of your unit tests, you may want to take different actions, or be notified via [124]SNS.

CodePipeline is currently optimized for releasing software and infrastructure in production. It includes release safety features that donâ€™t exist in general workflow automation tools. On the other side, CodePipeline isnâ€™t currently optimized for non-release scenarios such as orchestrating validation of a change prior to release, or for running multiple builds in parallel. To simplify those use cases, we are launching today a new [125]AWS Step Functions [126]service integration with CodeBuild. Now, during the execution of a state machine, you can [127]start or [128]stop a build, [129]get build report summaries, and [130]delete past build executions records.

In this way, you can define your own workflow-driven build process, and trigger it manually or automatically. For example you can: * Use [131]Amazon EventBridge rules to start the build workflow periodically (for nightly builds) or when something happens (such as a a pull request to an [132]AWS CodeCommit repository). * Build a [133]webhook that can be called by services such as GitHub using the [134]Amazon API Gateway, either with a [135]direct integration to a state machine, or via a [136]AWS Lambda function that checks the validity of the input payload before starting the workflow. * Customize the behavior of [137]AWS CodePipeline actions, as [138]described in this post.

With this integration, you can use the full capabilities of Step Functions to automate your software builds. For example, you can use a [139]Parallel state to create parallel builds for independent components of the build. Starting from a list of all the branches in your code repository, you can use a [140]Map state to run a set of steps (automating build, unit tests, and integration tests) for each branch. You can also leverage in the same workflow other Step Functions [141]service integrations. For instance, you can send a message to an [142]SQS queue to track your activities, or start a containerized application you just built using [143]Amazon ECS and [144]AWS Fargate.

Using Step Functions for a Workflow-Driven Build Process I am working on a Java web application. To be sure that it works as I add new features, I wrote a few tests using [145]JUnit Jupiter. I want those tests to be run just after the build process, but not always because tests can slow down some quick iterations. When I run tests, I want to [146]store and view the reports of my tests using CodeBuild. At the end, I want to be notified in an [147]SNS topic if the tests run, and if they were successful.

I created a repository in [148]CodeCommit and I included two [149]buildspec files for CodeBuild: * buildspec.yml is the default and is using [150]Apache Maven to run the build and the tests, and then is storing test results as reports.

version: 0.2 phases: build: commands: - mvn package artifacts: files: - target/binary-converter-1.0-SNAPSHOT.jar reports: SurefireReports: files: - '**/*' base-directory: 'target/surefire-reports' * buildspec-notests.yml is doing only the build, and no tests are executed.

version: 0.2 phases: build: commands: - mvn package -DskipTests artifacts: files: - target/binary-converter-1.0-SNAPSHOT.jar

To set up the CodeBuild project and the Step Functions state machine to automate the build, I am using [151]AWS CloudFormation with the following template: AWSTemplateFormatVersion: 2010-09-09 Description: AWS Step Functions sample project for getting notified on AWS CodeB uild test report results Resources: CodeBuildStateMachine: Type: AWS::StepFunctions::StateMachine Properties: RoleArn: !GetAtt [ CodeBuildExecutionRole, Arn ] DefinitionString: !Sub - |- { "Comment": "An example of using CodeBuild to run (or not run) test s, get test results and send a notification.", "StartAt": "Run Tests?", "States": { "Run Tests?": { "Type": "Choice", "Choices": [ { "Variable": "$.tests", "BooleanEquals": false, "Next": "Trigger CodeBuild Build Without Tests" } ], "Default": "Trigger CodeBuild Build With Tests" }, "Trigger CodeBuild Build With Tests": { "Type": "Task", "Resource": "arn:${AWS::Partition}:states:::codebuild:startBui ld.sync", "Parameters": { "ProjectName": "${projectName}" }, "Next": "Get Test Results" }, "Trigger CodeBuild Build Without Tests": { "Type": "Task", "Resource": "arn:${AWS::Partition}:states:::codebuild:startBui ld.sync", "Parameters": { "ProjectName": "${projectName}", "BuildspecOverride": "buildspec-notests.yml" }, "Next": "Notify No Tests" }, "Get Test Results": { "Type": "Task", "Resource": "arn:${AWS::Partition}:states:::codebuild:batchGet Reports", "Parameters": { "ReportArns.$": "$.Build.ReportArns" }, "Next": "All Tests Passed?" }, "All Tests Passed?": { "Type": "Choice", "Choices": [ { "Variable": "$.Reports[0].Status", "StringEquals": "SUCCEEDED", "Next": "Notify Success" } ], "Default": "Notify Failure" }, "Notify Success": { "Type": "Task", "Resource": "arn:${AWS::Partition}:states:::sns:publish", "Parameters": { "Message": "CodeBuild build tests succeeded", "TopicArn": "${snsTopicArn}" }, "End": true }, "Notify Failure": { "Type": "Task", "Resource": "arn:${AWS::Partition}:states:::sns:publish", "Parameters": { "Message": "CodeBuild build tests failed", "TopicArn": "${snsTopicArn}" }, "End": true }, "Notify No Tests": { "Type": "Task", "Resource": "arn:${AWS::Partition}:states:::sns:publish", "Parameters": { "Message": "CodeBuild build without tests", "TopicArn": "${snsTopicArn}" }, "End": true } } } - {snsTopicArn: !Ref SNSTopic, projectName: !Ref CodeBuildProject} SNSTopic: Type: AWS::SNS::Topic CodeBuildProject: Type: AWS::CodeBuild::Project Properties: ServiceRole: !Ref CodeBuildServiceRole Artifacts: Type: NO_ARTIFACTS Environment: Type: LINUX_CONTAINER ComputeType: BUILD_GENERAL1_SMALL Image: aws/codebuild/standard:2.0 Source: Type: CODECOMMIT Location: https://git-codecommit.us-east-1.amazonaws.com/v1/repos/binary -converter CodeBuildExecutionRole: Type: "AWS::IAM::Role" Properties: AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: "sts:AssumeRole" Principal: Service: states.amazonaws.com Path: "/" Policies: - PolicyName: CodeBuildExecutionRolePolicy PolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: - "sns:Publish" Resource: - !Ref SNSTopic - Effect: Allow Action: - "codebuild:StartBuild" - "codebuild:StopBuild" - "codebuild:BatchGetBuilds" - "codebuild:BatchGetReports" Resource: "*" - Effect: Allow Action: - "events:PutTargets" - "events:PutRule" - "events:DescribeRule" Resource: - !Sub "arn:${AWS::Partition}:events:${AWS::Region}:${AWS::Acc ountId}:rule/StepFunctionsGetEventForCodeBuildStartBuildRule" CodeBuildServiceRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: "sts:AssumeRole" Effect: Allow Principal: Service: codebuild.amazonaws.com Path: / Policies: - PolicyName: CodeBuildServiceRolePolicy PolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: - "logs:CreateLogGroup" - "logs:CreateLogStream" - "logs:PutLogEvents" - "codebuild:CreateReportGroup" - "codebuild:CreateReport" - "codebuild:UpdateReport" - "codebuild:BatchPutTestCases" - "codecommit:GitPull" Resource: "*" Outputs: StateMachineArn: Value: !Ref CodeBuildStateMachine ExecutionInput: Description: Sample input to StartExecution. Value: > {}

When the [152]CloudFormation stack has been created, there are two CodeBuild tasks in the state machine definition: * The first CodeBuild task is using a synchronous integration (startBuild.sync) to automatically wait for the build to terminate before progressing to the next step:

"Trigger CodeBuild Build With Tests": { "Type": "Task", "Resource": "arn:aws:states:::codebuild:startBuild.sync", "Parameters": { "ProjectName": "CodeBuildProject-HaVamwTeX8kM" }, "Next": "Get Test Results" } * The second CodeBuild task is using the BuildspecOverride parameter to override the default buildspec file used by the build with the one not running tests:

"Trigger CodeBuild Build Without Tests": { "Type": "Task", "Resource": "arn:aws:states:::codebuild:startBuild.sync", "Parameters": { "ProjectName": "CodeBuildProject-HaVamwTeX8kM", "BuildspecOverride": "buildspec-notests.yml" }, "Next": "Notify No Tests" },

The first step is a [153]Choice that looks into the input of the state machine execution to decide if to run tests, or not. For example, to run tests I can give in input: { "tests": true }

This is the visual workflow of the execution running tests, all tests are passed.

I change the value of "tests" to false, and start a new execution that goes on a different branch.

This time the buildspec is not executing tests, and I get a notification that no tests were run.

When starting this workflow automatically after an activity on GitHub or [154]CodeCommit, I could look into the last commit message for specific patterns, and customize the build process accordingly. For example, I could skip tests if the [skip tests] string is part of the commit message. Similarly, in a production environment I could skip code static analysis, to have faster integration for urgent changes, if the [skip static analysis] message in included in the commit.

Extending the Workflow for Containerized Applications A great way to distribute applications to different environments, is to package them as [155]Docker images. In this way, I can also add a step to my build workflow and start the containerized application in an [156]Amazon ECS task (running on [157]AWS Fargate) for the Quality Assurance (QA) team.

First, I create an image repository in ECR and add permissions to the service role used by the CodeBuild project to upload to ECR, as [158]described here.

Then, in the code repository, [159]I follow this example to add: * A Dockerfile to prepare the Docker container with the software build, and start the application. * A buildspec-docker.yml file with the commands to create and upload the Docker image.

The final workflow is automating all these steps: 1. Building the software from the source code. 2. Creating the Docker image. 3. Uploading of the Docker image to ECR. 4. Starting the QA environment on ECS and [160]Fargate. 5. Sending an [161]SNS notification that the QA environment is ready.

[162][step-functions-codebuild-docker-ecs-fargate_1.png]

The workflow and its steps can easily be customized based on your requirements. For example, with a few changes, you can [163]adapt the buildspec file to push the image to Docker Hub.

Available Now The CodeBuild service integration is available in all commercial and GovCloud regions where Step Functions and CodeBuild services are offered. For regional availability, please see the [164]AWS Region Table. For more information, [165]please look at the documentation.

As [166]AWS Serverless Hero Gojko Adzic pointed out on the [167]AWS DevOps Blog, [168]CodeBuild can also be used to execute administrative tasks. The integration with Step Functions opens a whole set of new possibilities.

Let me know what are you going to use this new service integration for!

â€" [169]Danilo

Danilo Poccia

[170]Danilo Poccia

Danilo works with startups and companies of any size to support their innovation. In his role as Chief Evangelist (EMEA) at Amazon Web Services, he leverages his experience to help people bring their ideas to life, focusing on serverless architectures and event-driven programming, and on the technical and business impact of machine learning and edge computing. He is the author of AWS Lambda in Action from Manning. [171]View Comments

Resources

[172]Getting Started [173]What's New [174]Top Posts [175]Official AWS Podcast [176]Case Studies __________________________________________________________________

Follow

[177] Twitter [178] Facebook [179] LinkedIn [180] Twitch [181] RSS Feed [182] Email Updates

[183]Sign In to the Console

Learn About AWS

* [184]What Is AWS? * [185]What Is Cloud Computing? * [186]What Is DevOps? * [187]What Is a Container? * [188]What Is a Data Lake? * [189]AWS Cloud Security * [190]What's New * [191]Blogs * [192]Press Releases

Resources for AWS

aws-senior.com

aws cloudtrail

aws cloudwatch documentation

aws cloudwatch

aws security logs

aws certification cost

amazon web services certification

aws iam audit report

aws cloudtrail ppt

aws cloudtrail tutorial

aws cloudtrail events