AWS Service Overview

AWS has an enormous number of services, however, for this blog, I will be covering a set number of AWS services

AWS cloud market share

Analytics

Athena

Amazon Athena is a serverless, interactive analytics service built on open-source frameworks, supporting open-table and file formats. Athena provides a simplified, flexible way to analyze petabytes of data where it lives

Kinesis

• collect, process, and analyze streaming data in real-time
• Kinesis Data Streams: capture, process, and store data streams – Lambda,
• Kinesis Data Firehose: load data streams into AWS data stores
• Kinesis Data Analytics: analyze data streams with SQL or Apache Flink
• Kinesis Video Streams: capture, process, and store video streams
• Retention between 1 day to 365 days, replay, immutability
• Upload Data blob (up to 1 MB)
• Each shard gets 1MB/s in (or 1000 records per second)
• Each shard gets 2MB/s out (classic or enhanced fan-out consumer)
• Write throughput: 1 MB/sec or 1000 records/sec per shard

OpenSearchService

Amazon OpenSearch Service makes it easy for you to perform interactive log analytics, real-time application monitoring, website search, and more. OpenSearch is an open source, distributed search and analytics suite derived from Elasticsearch

Application Integration

AppSync

Simple, Secure, Scalable — Fully-Managed Serverless GraphQL API Service for Real-Time Data Queries

EventBridge

• Amazon EventBridge is a serverless event bus that ingests data from your own apps, SaaS apps, and AWS services and routes that data to targets.

SNS

• send one message to many receivers by sending to SNS topic
• JSON policy used to filter messages sent to SNS topic’s subscriptions

SQS

• Unlimited throughput, unlimited number of messages in queue
• Limitation of 256KB per message sent
• Default retention of messages: 4 days, maximum of 14 days
• After a message is polled by a consumer, it becomes invisible to other consumers for bydefault 30sec
• After the MaximumReceives threshold is exceeded, the message goes into a Dead Letter Queue
• Delay a message(consumers don’t see it immediately) up to 15 minutes
• LongPolling decreases the number of API calls made to SQS
• SQS Extended Client to send larger than 256Kb

Step Functions

AWS Step Functions lets you orchestrate multiple AWS services into serverless workflows so that you can build and update applications quickly.

Compute

EC2

• EC2 = Elastic Compute Cloud
• It is possible to bootstrap instances using an EC2 User data script, and this is only run once at the instance’s first start.
• The EC2 User Data Script runs with the root user
• EC2 Instance Type – General purpose – Balance b/w Computing, Networking, and Memory.
• EC2 Instance Type – Compute Optimized – Good for compute-intensive applications such as some scientific modeling or high-performance web servers.
• EC2 Instance Type – Memory Optimized – Used for anything that needs memory-intensive applications, such as real-time big data analytics, or running Hadoop or Spark.
• EC2 Instance Type – Storage Optimized – Ideal for tasks that require huge amounts of storage, specifically with sequential read-writes, such as log processing, and Redis.
• Security Groups: They control how traffic is allowed into or out of our EC2 Instances, and is locked down to region.
• EC2 Instance Purchasing Options: OnDemand, Reserved(Reserved Instances), Savings Plans(Commitment to usage), Spot Instances, Dedicated Hosts(book an entire physical server), Dedicated Instances(no other customers will share your hardware), Capacity Reservations(Reserve On-Demand instances capacity in a specific AZ for any duration)
• An EBS (Elastic Block Store) Volume is a network drive you can attach to your instance(USB), mounted to only one instance, bounded by AZ. deleted when Instance terminated.
• EBS Snapshot: copy snapshots across AZ or Region
• AMI = Amazon Machine Image, customization of an EC2 instance, AMI are built for a specific region (and can be copied across regions)
• 6 EBS Volumes: gp2 / gp3 SSD (balances price and performance) boot volume, io1 / io2 SSD(low-latency or high-throughput workloads) boot volume, st1 HDD(throughput optimised), sc1
• EBS Multi Attach: Attach the same EBS volume to multiple EC2 instances in the same AZ(16 instances)
• EFS: Elastic File System – Managed NFS (network file system) that can be mounted on many EC2 – EFS works with EC2 instances in multi-AZ – Encryption at rest using KMS
• http://169.254.169.254/latest/meta-data
• API for EC2 has a limit of 100 calls per seconds

Elastic Beanstalk

• Automatically handles capacity provisioning, load balancing, scaling, application health monitoring, instance configuration whereas the developer focuses on writing the code.
• Supports Java, .Net, Python, PHP, Ruby, Go,
• Deployment Options – AtOnce, Rolling, Rolling with Additional Batches, Immutables, BG, Traffic splitting
• .ebextensions/ directory in the root of source code
• Relies on CF

Lambda

• From 128MB to 10GB in 1MB increments
• The more RAM you add, the more vCPU credits you get
• Timeout: default 3 seconds, maximum is 900 seconds (15 minutes)
• Concurrency limit: up to 1000 concurrent executions

SAM

AWS SAM templates provide a short-hand syntax, optimized for defining Infrastructure as Code (IaC) for serverless applications

Containers

Copilot 

• CLI tool to build, release, and operate production-ready containerized apps

ECR

• Store container images
• Access is controlled through IAM, supports vulnerability checks

ECS

• Elastic Container Service
• Ec2 launch type means you maintain infra whereas Fargate launch type AWS does
• ECS can be integrated with ELB
• ECS with EFS integration
• ECS autoscalling – increase/ decrease the tasks automatic – CPU based
• ECS rolling based deployment
  • ECS Environment Variables – Hardcoded, SSM Parameter store, Secrets manager

EKS

• Elastic Kubernetes service
• It’s an alternative to ECS,
• EKS supports EC2 & Fargate

Database

Aurora 

• Aurora is a proprietary technology and is “AWS cloud-optimized” and claims 5x better than RDS MySQL, & 3x better than RDS PostgreSQL – Max 128TB, 15 replicas – Cross Region – Cross AZ
• RDS proxy allows apps to pool and share DB connections established with the database – min CPU – openConnections – Enforces IAM authentication

DynamoDB 

• Fully managed, highly available with replication across multiple AZs
• Low cost and auto-scaling capabilities
• Maximum size of an item is 400KB
• DAX: Fully-managed, highly available, seamless in-memory cache for DynamoDB
• Transaction:
• Coordinated, all-or-nothing operations (add/update/delete) to multiple items
• across one or more tables

ElastiCache 

• Elastic Cache – Managed Memcached – No HA, Non Persistent, No backup restore – Redis – high performance, low latency – stateless, BR
• Amazon MemoryDB for Redis – Redis-compatible, durable, in-memory database service – Cross AZ, Max 100TB

RDS 

• Relational Database Service is a managed DB service for DB uses SQL as a query language.
• Supports Postgres, MySQL, MariaDB, Oracle, MSSQL, Aurora
• RDS is a managed service: provisioning, backups, replicas – 15 Replicas – eventually consistent – Cross AZ – Cross Region, DR but no SSH

Developer tools

Amplify 

Scalable and Easy to Use — Build Extensible, Full-Stack Web and Mobile Apps Faster. Easy to Start, Easy to Scale. Develop High Quality, Scalable, Cloud-Connected Mobile and Web Apps With AWS Amplify.

Cloud9 

Get Started with AWS — Write, Run, and Debug Your Code With Just a Browser. Work From Anywhere With AWS Cloud9. Easily Develop Serverless Applications From Anywhere With AWS Cloud9.

CloudShell 

AWS CloudShell is a browser-based, pre-authenticated shell that you can launch directly from the AWS Management Console.

CodeArtifact

AWS CodeArtifact is a managed artifact repository service that lets you securely store, publish, and share software packages.

CodeBuild 

 AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces ready-to-deploy software 

CodeCommit 

AWS CodeCommit is a managed source control repository that makes it easier for teams to securely collaborate on code with contributions encrypted in transit

CodeDeploy 

CodeDeploy is a deployment service that automates application deployments to Amazon EC2 instances, on-premises instances, serverless Lambda functions

CodeGuru 

Amazon CodeGuru helps you improve code quality and automate code reviews by scanning and profiling your Java and Python applications.

CodePipeline 

AWS CodePipeline automates the build, test, and deploy phases of your release process each time a code change occurs.

CodeStar 

AWS CodeStar provides a unified user interface, enabling you to easily manage your software development activities in one place.

CodeWhisperer 

Amazon CodeWhisperer is a general purpose, machine learning-powered code generator that provides you with code recommendations, in real time. As you write code, CodeWhisperer automatically generates suggestions based on your existing code and comments. Your personalized recommendations can vary in size and scope, ranging from a single line comment to fully formed functions.

X-Ray

• Distributed tracing of microservices
• Troubleshooting application performance and errors

Management and governance

AppConfig

AWS AppConfig simplifies the administration of applications at scale by deploying configuration changes from a central location.

CDK

The AWS Cloud Development Kit is an open-source software development framework developed by Amazon Web Services for defining and provisioning cloud infrastructure resources using familiar programming languages

CloudFormation

• CloudFormation is a declarative way of outlining your AWS Infrastructure, for any resources
• Resources: your AWS resources declared in the template (MANDATORY), Ec2Instance – type, properties
• Parameters: the dynamic inputs for your template
• Mappings: the static variables for your template
• Outputs: References to what has been created
• Conditionals: List of conditions to perform resource creation – Fn::And, Fn::If, Fn::Equals
• Metadata
• References: Fn::Ref to reference parameters
• Functions: Fn::FindInMap, Fn::ImportValue, Fn::GetAtt
• Drift: Find if the resources have changed

CloudTrail 

• Internal monitoring of API calls being made
• Audit changes to AWS Resources by your users
• CloudTrail Insights to detect unusual activity
• Events are stored for 90 days in CloudTrail

CloudWatch 

• Metrics: Collect and track key metrics
• Logs: Collect, monitor, analyze and store log files
• Events: Send notifications when certain events happen in your AWS
• Alarms: React in real-time to metrics / events
• EC2 instance metrics have metrics “every 5 minutes”
• Log Insights: Search and analyze log data stored in CloudWatch Logs

Systems Manager

AWS Systems Manager gives you visibility and control of your infrastructure on AWS. Systems Manager provides a unified user interface so you can view operational data from multiple AWS services and allows you to automate operational tasks across your AWS resources.

Networking and content delivery

API Gateway

Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale.

CloudFront 

• Content Delivery Network (CDN)
• Improves read performance,
• content is cached at the edge
• DDOS protection, shield, Origin Access Control
• S3, ALB, EC2, Any Http Backend
• Cloudfront geo restriction
• CloudFront Signed URL / Cookie – Account wide key-pair, only the rootcan manage it, filter by IP, path, date, expiration –
• trusted key group – private key to sign and public to verify
• CloudFront can route requests to multiple origins based on content-type, path-pattern, does failover

ELB

• Load Balancers spread load across multiple downstream instances
• handle failures, health checks, Enforce stickiness, HA
• Classic Load Balancer: HTTP, HTTPS, TCP, SSL (secure TCP)
• Application Load Balancer: HTTP, HTTPS, gRPC, WebSocket – Works on layer 7 – Round-robin – Routing based on path, URL, and query strings.
• Network Load Balancer: TCP, TLS (secure TCP), and UDP, operate at 4 transport layers – Flow hash, Handle millions of req/s,
• Gateway Load Balancer: Supports IP-based routing – Routing table lookup – Works on the network layer, layer 3, and layer 7.
• IP Addresses – must be private IPs, true IP is added as header, X-Forwarded-For, X-Forwarded-Port
• ALBS can attach multiple target groups, and health checks at the target group.
• Sticky Sessions: The same client is always redirected to the same instance behind a load balancer – ALB, NLB & CLB supports this, ALB & CLB uses Cookies for Sticky sessions named AWSALB, AWSALBAPP, AWSALBTG, and AWSELB
• Cross Zone Load Balancing: each load balancer instance distributes evenly across all registered instances in all AZ otherwise some instances take more traffic as Load balances on target groups. APLB – Enabled, free, CLB – Disabled, free, GLB-NLB – Disabled, paid
• SSL -SNI – Helps to load multiple certificates to the server to serve multiple websites.
• Connection Draining: Default 300sec, time to complete the already accepted requests.
• ASG: Auto Scaling Group helps to scale out/ back based on min, desired, and maximum capacity.
• ASG metrics ex – CPUUtilization, RqCountPerTarget, Avg N/w In/Out, custom metric.

Route 53

• Domain Name System – 53 is a reference to the traditional DNS port,
• Types of DNS Record types A(IPv4) / AAAA(IPv6) / CNAME(another hostname, but no root domain) / Alias(only aws domains)
• Routing Policies – Simple/ Weighted – 70, 20, 10, HealthCheck/ Failover/ Latency based – best latency/ Geolocation/ Multi-Value Answer/ Geoproximity
• Health Checks monitor the heath of services – 3 failures makes unhealthy – Passes for 2xx or 3xx- Combine max 256 Child Health checks
• The health check is outside VPC – CloudWatch Alarm + Health Check for inside VPC checks.

VPC 

• VPC: private network to deploy your resources, public subnet, private subnet, routetables: Access b/w subnets, internet
• Internet Gateway: helps VPC connect with Internet, Public subnet has a route table entry to IG, NAT g/w helps private access internet through IG.
• NACL: firewall which controld traffic from to subnet, stateless. Security Group: firewalls which controls traffic from EC2 etc, statefull
• VPC flow logs: captures information about IP traffic going into your interfaces.
• VPC peering: connect two VPCs
• VPC Endpoints: connect two services using private n/w instead of public internet, VPC Endpoint Gatewat – S3, DynamoDB, rest as VPC endpoint interface.
• WS PrivateLink — A technology that provides private connectivity between VPCs and services.
• Site to Site VPN – Connect an on-premises VPN to AWS
• Direct Connect -physical connection between onpremises and AWS

Security, identity, and compliance

ACM

Use AWS Certificate Manager (ACM) to provision, manage, and deploy public and private SSL/TLS certificates for use with AWS services and your internal connected resources. ACM removes the time-consuming manual process of purchasing, uploading, and renewing SSL/TLS certificates.

Cognito

Quick and Easy User Sign-Up, Sign-In, and Access Control to Your Web and Mobile Apps. Simple and Secure User Sign-Up, Sign-In, and Access Control. Get Started for Free.

IAM

• IAM = Identity and Access Management, Global service
• IAM Policy structure consists of
  Effect: whether the statement allows or denies access
  Principal: account/user/role to which this policy applied
  Action: list of actions this policy allows or denies
  Resource: list of resources to which the actions applied to
  Condition: conditions for when this policy is in effect
• IAM roles for the service which allows AWS services to perform actions on your behalf.

KMS

AWS Key Management Service (AWS KMS) lets you create, manage, and control cryptographic keys across your applications and AWS services. 

Private Certificate Authority

AWS Private CA is a highly available certificate authority that helps organizations secure their applications and devices using private certificates.

Secrets Manager

AWS Secrets Manager allows you to rotate, manage, and retrieve database credentials, API keys, and other secrets through their lifecycle.

STS

AWS provides AWS Security Token Service (AWS STS) as a web service that enables you to request temporary, limited-privilege credentials for users.

WAF

AWS WAF helps you protect against common web exploits and bots that can affect availability, compromise security, or consume excessive resources.

Storage

EBS

Amazon Elastic Block Store (Amazon EBS) is an easy-to-use, scalable, high-performance block-storage service designed for Amazon Elastic Compute Cloud 

EFS

Amazon Elastic File System (EFS) provides a simple, scalable fully managed elastic NFS file system for AWS compute instances.

S3

• Simple Storage service stores objects in buckets.
• S3 looks like a global service but buckets are stored in a region.
• Each object have an key i.e folderpath + filename, folder path is just a key with slashes.
• Max 5TB, >5GB use MultiPart upload.
• S3 Security – IAM policies at the user level, Bucket policies from S3 Console allow cross-account, Object ACL, Bucket ACL(not recommend).
• S3 allows versioning, cross-region replication
• S3 Standard(99.99% available), Standard Infrequent Access(less called but same access speed), OneZone IA(1 az), Glacier Instant(mill retrieval once a quarter min 90 days)/Flexibe(varied times starting from 1-5 min min 90)/deep retrieval min 12 hours min 180days, Intelligent tiering(move from tier to tier depending on usage)
• GetObject on S3 has a limit of 5500 GET per second, 3500 rest
• S3 supports transition from one type, expiration of objects after no use, EventNotification
• Increase transfer speed by using an edge location which uses private AWS to increase speed
• S3 Encryption: SSE(Server side encryption )AES256, enableddefault – S3, SSE-KMS aws:kms, user control, SSE-C(Encryption key must provided in HTTP headers, for every HTTP request made), Client Side Encryption
• S3 Access logs, MFA, . S3 Cross Region Replication
• Access Points simplify security management for S3 Buckets – AccessPoint policy

S3 Glacier

S3 Glacier is a secure and durable service for low-cost data archiving and long-term backup using vaults.