2020年11月26日 星期四

Serverless

 最近做的 Serverless 概念的課程簡報

Serverless
 留言

開發工程必備的雲端架構知識

Paul Chen
DevOps @ 104 開發營運部

    


  • Serverless 由來
  • FaaS?SaaS?Serverless?
  • Serverless Architecture
  • Serverless Limits
  • Serverless Pros and Cons
  • Recap
  •  留言

Serverless 由來


Serverless Providers

  • AWS Lambda 
  • Microsoft Azure Function
  • Google Cloud Functions
  • IBM Cloud Functions

FaaS?SaaS?Serverless?

Note:

IaaS example: AWS EC2, OpenStack
PaaS example: Heroku, AWS Elastic Beanstalk
FaaS example: AWS Lambda
SaaS example: Gmail, Youtube, Office365, TravisCI


So… What is Serverless?


Serverless computing is a cloud computing execution model in which the cloud provider runs the server, and dynamically manages the allocation of machine resources. – Wiki


Function as a service (FaaS) is a category of cloud computing services that provides a platform allowing customers to develop, run, and manage application functionalities without the complexity of building and maintaining the infrastructure. Building an application following this model is one way of achieving a “serverless” architecture. – Wiki


Serverless Architecture on AWS


The AWS serverless platform


The AWS serverless platform


Serverless Develop Limits

  • Functional Programming
  • Execution Duration
  • Stateless

Serverless Pros and Cons


Pros : Cost-effective


Pros : Scalability and Availability



Pros of Serverless

  • Cost-effective
  • Efficient scalability
  • High availability
  • Reduced operational costs
  • Focus on business, not on infrastructure

Serverless Cons

and how AWS can help


Cons of Serverless

  • Complex system architecture
    • Latency
    • Difficulty in debugging and running local test
  • Vendor lock-in

AWS X-Ray


AWS X-Ray Traces


Serverless Application Model (SAM)

$ sam build --use-container
$ sam local invoke -e event.json ExampleLambda

Recap

  • Serverless architecture have servers, but they are not managed by you
  • Functions are a core concept
  • Scaling and High Availability is done by cloud providers
  • Keeps cost low, by covering spikes and idle
  • Focus on business, not on infrastructure

End

2019年12月4日 星期三

利用 AWS SAM 部署 Serverless 時,需要什麼權限??

利用 AWS SAM 部署 Serverless 時,需要什麼權限??

每次要部署 SAM Project 時,為了符合 AWS IAM Best Practices 都要調校一次 CI User 的 Policy,耗時耗力,所以整理一份 Sample Policy 以便之後直接取用。
會用到的 SAM command 有 validate, build, package, deploy
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "lambda:CreateFunction",
                "lambda:UpdateFunctionCode",
                "lambda:TagResource",
                "lambda:UntagResource",
                "lambda:ListTags",
                "lambda:GetFunctionConfiguration",
                "lambda:UpdateFunctionConfiguration",
                "iam:GetRole",
                "cloudformation:UpdateStack",
                "cloudformation:DescribeStackEvents",
                "cloudformation:CreateChangeSet",
                "cloudformation:DeleteChangeSet",
                "cloudformation:DescribeChangeSet",
                "cloudformation:ExecuteChangeSet"
                "s3:PutObject",
                "s3:GetObject",
            ],
            "Resource": [
                "arn:aws:lambda:{AWS::Region}:{AWS::AccountId}:function:{STACK_NAME}-{FUNCTION_NAME}*",
                "arn:aws:s3:::{BUCKET_NAME}/*",
                "arn:aws:iam::{AWS::AccountId}:role/{STACK_NAME}-{FUNCTION_NAME}Role*",
                "arn:aws:cloudformation:{AWS::Region}:aws:transform/Serverless-2016-10-31",
                "arn:aws:cloudformation:{AWS::Region}:{AWS::AccountId}:stack/{STACK_NAME}/*"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "cloudformation:ValidateTemplate",
                "cloudformation:GetTemplateSummary",
                "cloudformation:DescribeStacks"
            ],
            "Resource": "*"
        }
    ]
}
  • replace the following:
    • {BUCKET_NAME} with the bucket name you’re using for code upload
    • {STACK_NAME} with your stack name
    • {FUNCTION_NAME} with your function name
  • 如果在 deploy 加上 —debug 參數,則還會需要 cloudformation:DescribeStackEvents 權限
  • 如果用到 Events properties,則還會需要 lambda:AddPermission,以及 Event type 相對應的 Resource Permission
  • 如果有設定 Lambda Role Policies 則還會需要以下 Permission
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "iam:GetRole",
                "iam:PassRole",
                "iam:DeleteRolePolicy",
                "iam:CreateRole",
                "iam:DeleteRole",
                "iam:PutRolePolicy",
                "iam:GetRolePolicy"
            ],
            "Resource": "arn:aws:iam::{AWS::AccountId}:role/{STACK_NAME}-LambdaRole*"
        }
    ]
}

Reference

2019年9月27日 星期五

How to delete AWS Config service

How to delete AWS Config service

最近在用 AWS Config,玩過以後想說要刪掉,結果發現在 console 找不到 delete config 功能,查了文件後,看起來只能用 cli 才砍得到,而且還不是一個 command,而是 ConfigurationRecorderDeliveryChannel 兩個 resources 分開,如下:
$ aws configservice delete-configuration-recorder --configuration-recorder-name default
$ aws configservice delete-delivery-channel --delivery-channel-name default

Reference

2019年9月24日 星期二

AWS Application Load Balancers Rule Condition Types

AWS Application Load Balancers Rule Condition Types

host-header
Route based on the host name of each request.
path-pattern
Route based on path patterns in the request URLs.
http-header
Route based on the HTTP headers for each request.
http-request-method
Route based on the HTTP request method of each request.
query-string
Route based on key/value pairs or values in the query strings.
source-ip
Route based on the source IP address of each request.

Reference

2019年9月9日 星期一

AWS SAM (Serverless Application Mode)

What Is the AWS Serverless Application Model (AWS SAM)

The AWS Serverless Application Model (AWS SAM) is an open-source framework that you can use to build serverless applications on AWS.
A serverless application is a combination of Lambda functions, event sources, and other resources that work together to perform tasks. Note that a serverless application is more than just a Lambda function—it can include additional resources such as APIs, databases, and event source mappings.
簡單來說,就是結合 Lambda function 跟事件來源、其他資源等 serverless 的組合技

AWS SAM template specification.

顧名思義,不解釋,自己看比較快:AWS Serverless Application Model Specification
下面提供兩個範例
example: s3 create object send event to lambda function
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: s3 create object send event to lambda function
Resources:
  MyLambda:
    Type: AWS::Serverless::Function
    Properties:
      Handler: my_lambda.my_handler
      Timeout: 300
      Runtime: python3.6
      Role: arn:aws:iam:::role/LambdaRole
      Events:
        S3Event:
          Type: S3
          Properties:
            Bucket: !Ref MyBucket
            Events: s3:ObjectCreated:*
  MyBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: 
example: cloudwatch schecduled event (每天凌晨兩點) trigger lambda function
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: cloudwatch schecduled event trigger lambda function
Resources:
  MyLambda:
    Type: AWS::Serverless::Function
    Properties:
      Handler: my_lambda.my_handler
      Timeout: 300
      Runtime: python3.6
      Role: arn:aws:iam:::role/LambdaRole
      Events:
        MyEvent:
          Type: Schedule
          Properties:
            Schedule: cron(0 18 * * ? *)

AWS SAM command line interface (AWS SAM CLI)

The AWS SAM CLI lets you locally build, test, and debug serverless applications that are defined by AWS SAM templates.
可以在本機執行 Lambda Function,並且如果用使用 event source 的話,可以產生測試用的 event 方便本機測試;另外,可以結合 CI 服務,執行 build, package, deploy,做到 CI/CD

Install

$ pip install awscli
$ pip install aws-sam-cli

How to use

validate

顧名思義,驗證 template 寫法是否正確

build

可以配合使用 --use-container 參數解決第三方程式庫引入問題

--use-container

If your functions depend on packages that have natively compiled dependencies, use this flag to build your function inside an AWS Lambda-like Docker container

local

example
$ sam local invoke -e event.json MyLambda

invoke

Invokes a local Lambda function once.

generate-event

產生測試用的 event 方便本機測試
You can use this command to generate sample payloads from different event sources such as S3, 
API Gateway, and SNS. These payloads contain the information that the event sources send to your Lambda functions.

Generate the event that S3 sends to your Lambda function when a new object is uploaded
  $ sam local generate-event s3 [put/delete]

  You can even customize the event by adding parameter flags. To find which flags apply to your command,
  run:

  $ sam local generate-event s3 [put/delete] --help

  Then you can add in those flags that you wish to customize using

  $ sam local generate-event s3 [put/delete] --bucket  --key 

  After you generate a sample event, you can use it to test your Lambda function locally
  $ sam local generate-event s3 [put/delete] --bucket  --key  | sam local invoke 
example: s3 create object event
{
  "Records": [
    {
      "eventVersion": "2.0",
      "eventSource": "aws:s3",
      "awsRegion": "ap-northeast-1",
      "eventTime": "1970-01-01T00:00:00.000Z",
      "eventName": "ObjectCreated:Put",
      "userIdentity": {
        "principalId": "EXAMPLE"
      },
      "requestParameters": {
        "sourceIPAddress": "127.0.0.1"
      },
      "responseElements": {
        "x-amz-request-id": "EXAMPLE123456789",
        "x-amz-id-2": "EXAMPLE123/5678abcdefghijklambdaisawesome/mnopqrstuvwxyzABCDEFGH"
      },
      "s3": {
        "s3SchemaVersion": "1.0",
        "configurationId": "testConfigRule",
        "bucket": {
          "name": "my_bucket",
          "ownerIdentity": {
            "principalId": "EXAMPLE"
          },
          "arn": "arn:aws:s3:::my_bucket"
        },
        "object": {
          "key": "test.txt",
          "size": 19,
          "eTag": "8a6da8606b1e063921d61dbaf8f5b643",
          "sequencer": "0A1B2C3D4E5F678901"
        }
      }
    }
  ]
}
example: cloudwatch scheduled event
{
  "id": "cdc73f9d-aea9-11e3-9d5a-835b769c0d9c",
  "detail-type": "Scheduled Event",
  "source": "aws.events",
  "account": "",
  "time": "1970-01-01T00:00:00Z",
  "region": "us-east-1",
  "resources": [
    "arn:aws:events:us-east-1:123456789012:rule/ExampleRule"
  ],
  "detail": {}
}

package

Package an AWS SAM application. This is an alias for 'aws cloudformation package'.

deploy

Deploy an AWS SAM application. This is an alias for 'aws cloudformation deploy'.