AWS Service Overview | Jones J Alapat Tech Blog

Amazon Athena is a serverless, interactive analytics service built on open-source frameworks, supporting open-table and file formats. Athena provides a simplified, flexible way to analyze petabytes of data where it lives

Kinesis

collect, process, and analyze streaming data in real-time
Kinesis Data Streams: capture, process, and store data streams – Lambda,
Kinesis Data Firehose: load data streams into AWS data stores
Kinesis Data Analytics: analyze data streams with SQL or Apache Flink
Kinesis Video Streams: capture, process, and store video streams
Retention between 1 day to 365 days, replay, immutability
Upload Data blob (up to 1 MB)
Each shard gets 1MB/s in (or 1000 records per second)
Each shard gets 2MB/s out (classic or enhanced fan-out consumer)
Write throughput: 1 MB/sec or 1000 records/sec per shard

OpenSearchService

Amazon OpenSearch Service makes it easy for you to perform interactive log analytics, real-time application monitoring, website search, and more. OpenSearch is an open source, distributed search and analytics suite derived from Elasticsearch

Application Integration

AppSync

Simple, Secure, Scalable — Fully-Managed Serverless GraphQL API Service for Real-Time Data Queries

EventBridge

Amazon EventBridge is a serverless event bus that ingests data from your own apps, SaaS apps, and AWS services and routes that data to targets.

SNS

send one message to many receivers by sending to SNS topic
JSON policy used to filter messages sent to SNS topic’s subscriptions

SQS

Unlimited throughput, unlimited number of messages in queue
Limitation of 256KB per message sent
Default retention of messages: 4 days, maximum of 14 days
After a message is polled by a consumer, it becomes invisible to other consumers for bydefault 30sec
After the MaximumReceives threshold is exceeded, the message goes into a Dead Letter Queue
Delay a message(consumers don’t see it immediately) up to 15 minutes
LongPolling decreases the number of API calls made to SQS
SQS Extended Client to send larger than 256Kb

Step Functions

AWS Step Functions lets you orchestrate multiple AWS services into serverless workflows so that you can build and update applications quickly.

Compute

EC2

EC2 = Elastic Compute Cloud
It is possible to bootstrap instances using an EC2 User data script, and this is only run once at the instance’s first start.
The EC2 User Data Script runs with the root user
EC2 Instance Type – General purpose – Balance b/w Computing, Networking, and Memory.
EC2 Instance Type – Compute Optimized – Good for compute-intensive applications such as some scientific modeling or high-performance web servers.
EC2 Instance Type – Memory Optimized – Used for anything that needs memory-intensive applications, such as real-time big data analytics, or running Hadoop or Spark.
EC2 Instance Type – Storage Optimized – Ideal for tasks that require huge amounts of storage, specifically with sequential read-writes, such as log processing, and Redis.
Security Groups: They control how traffic is allowed into or out of our EC2 Instances, and is locked down to region.
EC2 Instance Purchasing Options: OnDemand, Reserved(Reserved Instances), Savings Plans(Commitment to usage), Spot Instances, Dedicated Hosts(book an entire physical server), Dedicated Instances(no other customers will share your hardware), Capacity Reservations(Reserve On-Demand instances capacity in a specific AZ for any duration)
An EBS (Elastic Block Store) Volume is a network drive you can attach to your instance(USB), mounted to only one instance, bounded by AZ. deleted when Instance terminated.
EBS Snapshot: copy snapshots across AZ or Region
AMI = Amazon Machine Image, customization of an EC2 instance, AMI are built for a specific region (and can be copied across regions)
6 EBS Volumes: gp2 / gp3 SSD (balances price and performance) boot volume, io1 / io2 SSD(low-latency or high-throughput workloads) boot volume, st1 HDD(throughput optimised), sc1
EBS Multi Attach: Attach the same EBS volume to multiple EC2 instances in the same AZ(16 instances)
EFS: Elastic File System – Managed NFS (network file system) that can be mounted on many EC2 – EFS works with EC2 instances in multi-AZ – Encryption at rest using KMS
http://169.254.169.254/latest/meta-data
API for EC2 has a limit of 100 calls per seconds

Elastic Beanstalk

Automatically handles capacity provisioning, load balancing, scaling, application health monitoring, instance configuration whereas the developer focuses on writing the code.
Supports Java, .Net, Python, PHP, Ruby, Go,
Deployment Options – AtOnce, Rolling, Rolling with Additional Batches, Immutables, BG, Traffic splitting
.ebextensions/ directory in the root of source code
Relies on CF

Lambda

From 128MB to 10GB in 1MB increments
The more RAM you add, the more vCPU credits you get
Timeout: default 3 seconds, maximum is 900 seconds (15 minutes)
Concurrency limit: up to 1000 concurrent executions

SAM

AWS SAM templates provide a short-hand syntax, optimized for defining Infrastructure as Code (IaC) for serverless applications

Containers

Copilot

CLI tool to build, release, and operate production-ready containerized apps

ECR

Store container images
Access is controlled through IAM, supports vulnerability checks

ECS

Elastic Container Service
Ec2 launch type means you maintain infra whereas Fargate launch type AWS does
ECS can be integrated with ELB
ECS with EFS integration
ECS autoscalling – increase/ decrease the tasks automatic – CPU based
ECS rolling based deployment
- ECS Environment Variables – Hardcoded, SSM Parameter store, Secrets manager

EKS

Elastic Kubernetes service
It’s an alternative to ECS,
EKS supports EC2 & Fargate

Database

Aurora

Aurora is a proprietary technology and is “AWS cloud-optimized” and claims 5x better than RDS MySQL, & 3x better than RDS PostgreSQL – Max 128TB, 15 replicas – Cross Region – Cross AZ
RDS proxy allows apps to pool and share DB connections established with the database – min CPU – openConnections – Enforces IAM authentication

DynamoDB

Fully managed, highly available with replication across multiple AZs
Low cost and auto-scaling capabilities
Maximum size of an item is 400KB
DAX: Fully-managed, highly available, seamless in-memory cache for DynamoDB
Transaction:
Coordinated, all-or-nothing operations (add/update/delete) to multiple items
across one or more tables

ElastiCache

Elastic Cache – Managed Memcached – No HA, Non Persistent, No backup restore – Redis – high performance, low latency – stateless, BR
Amazon MemoryDB for Redis – Redis-compatible, durable, in-memory database service – Cross AZ, Max 100TB

RDS

Relational Database Service is a managed DB service for DB uses SQL as a query language.
Supports Postgres, MySQL, MariaDB, Oracle, MSSQL, Aurora
RDS is a managed service: provisioning, backups, replicas – 15 Replicas – eventually consistent – Cross AZ – Cross Region, DR but no SSH

Developer tools

Amplify

Scalable and Easy to Use — Build Extensible, Full-Stack Web and Mobile Apps Faster. Easy to Start, Easy to Scale. Develop High Quality, Scalable, Cloud-Connected Mobile and Web Apps With AWS Amplify.

Cloud9

Get Started with AWS — Write, Run, and Debug Your Code With Just a Browser. Work From Anywhere With AWS Cloud9. Easily Develop Serverless Applications From Anywhere With AWS Cloud9.

CloudShell

AWS CloudShell is a browser-based, pre-authenticated shell that you can launch directly from the AWS Management Console.

CodeArtifact

AWS CodeArtifact is a managed artifact repository service that lets you securely store, publish, and share software packages.

CodeBuild

AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces ready-to-deploy software

CodeCommit

AWS CodeCommit is a managed source control repository that makes it easier for teams to securely collaborate on code with contributions encrypted in transit

CodeDeploy

CodeDeploy is a deployment service that automates application deployments to Amazon EC2 instances, on-premises instances, serverless Lambda functions

CodeGuru

Amazon CodeGuru helps you improve code quality and automate code reviews by scanning and profiling your Java and Python applications.

CodePipeline

AWS CodePipeline automates the build, test, and deploy phases of your release process each time a code change occurs.

CodeStar

AWS CodeStar provides a unified user interface, enabling you to easily manage your software development activities in one place.

CodeWhisperer

Amazon CodeWhisperer is a general purpose, machine learning-powered code generator that provides you with code recommendations, in real time. As you write code, CodeWhisperer automatically generates suggestions based on your existing code and comments. Your personalized recommendations can vary in size and scope, ranging from a single line comment to fully formed functions.

X-Ray

Distributed tracing of microservices
Troubleshooting application performance and errors

Management and governance

AppConfig

AWS AppConfig simplifies the administration of applications at scale by deploying configuration changes from a central location.

CDK

The AWS Cloud Development Kit is an open-source software development framework developed by Amazon Web Services for defining and provisioning cloud infrastructure resources using familiar programming languages

CloudFormation

CloudFormation is a declarative way of outlining your AWS Infrastructure, for any resources
Resources: your AWS resources declared in the template (MANDATORY), Ec2Instance – type, properties
Parameters: the dynamic inputs for your template
Mappings: the static variables for your template
Outputs: References to what has been created
Conditionals: List of conditions to perform resource creation – Fn::And, Fn::If, Fn::Equals
Metadata
References: Fn::Ref to reference parameters
Functions: Fn::FindInMap, Fn::ImportValue, Fn::GetAtt
Drift: Find if the resources have changed

CloudTrail

Internal monitoring of API calls being made
Audit changes to AWS Resources by your users
CloudTrail Insights to detect unusual activity
Events are stored for 90 days in CloudTrail

CloudWatch

Metrics: Collect and track key metrics
Logs: Collect, monitor, analyze and store log files
Events: Send notifications when certain events happen in your AWS
Alarms: React in real-time to metrics / events
EC2 instance metrics have metrics “every 5 minutes”
Log Insights: Search and analyze log data stored in CloudWatch Logs

Systems Manager

AWS Systems Manager gives you visibility and control of your infrastructure on AWS. Systems Manager provides a unified user interface so you can view operational data from multiple AWS services and allows you to automate operational tasks across your AWS resources.

Networking and content delivery

API Gateway

Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale.

CloudFront

Content Delivery Network (CDN)
Improves read performance,
content is cached at the edge
DDOS protection, shield, Origin Access Control
S3, ALB, EC2, Any Http Backend
Cloudfront geo restriction
CloudFront Signed URL / Cookie – Account wide key-pair, only the rootcan manage it, filter by IP, path, date, expiration –
trusted key group – private key to sign and public to verify
CloudFront can route requests to multiple origins based on content-type, path-pattern, does failover

ELB

Load Balancers spread load across multiple downstream instances
handle failures, health checks, Enforce stickiness, HA
Classic Load Balancer: HTTP, HTTPS, TCP, SSL (secure TCP)
Application Load Balancer: HTTP, HTTPS, gRPC, WebSocket – Works on layer 7 – Round-robin – Routing based on path, URL, and query strings.
Network Load Balancer: TCP, TLS (secure TCP), and UDP, operate at 4 transport layers – Flow hash, Handle millions of req/s,
Gateway Load Balancer: Supports IP-based routing – Routing table lookup – Works on the network layer, layer 3, and layer 7.
IP Addresses – must be private IPs, true IP is added as header, X-Forwarded-For, X-Forwarded-Port
ALBS can attach multiple target groups, and health checks at the target group.
Sticky Sessions: The same client is always redirected to the same instance behind a load balancer – ALB, NLB & CLB supports this, ALB & CLB uses Cookies for Sticky sessions named AWSALB, AWSALBAPP, AWSALBTG, and AWSELB
Cross Zone Load Balancing: each load balancer instance distributes evenly across all registered instances in all AZ otherwise some instances take more traffic as Load balances on target groups. APLB – Enabled, free, CLB – Disabled, free, GLB-NLB – Disabled, paid
SSL -SNI – Helps to load multiple certificates to the server to serve multiple websites.
Connection Draining: Default 300sec, time to complete the already accepted requests.
ASG: Auto Scaling Group helps to scale out/ back based on min, desired, and maximum capacity.
ASG metrics ex – CPUUtilization, RqCountPerTarget, Avg N/w In/Out, custom metric.

Route 53

Domain Name System – 53 is a reference to the traditional DNS port,
Types of DNS Record types A(IPv4) / AAAA(IPv6) / CNAME(another hostname, but no root domain) / Alias(only aws domains)
Routing Policies – Simple/ Weighted – 70, 20, 10, HealthCheck/ Failover/ Latency based – best latency/ Geolocation/ Multi-Value Answer/ Geoproximity
Health Checks monitor the heath of services – 3 failures makes unhealthy – Passes for 2xx or 3xx- Combine max 256 Child Health checks
The health check is outside VPC – CloudWatch Alarm + Health Check for inside VPC checks.

VPC

VPC: private network to deploy your resources, public subnet, private subnet, routetables: Access b/w subnets, internet
Internet Gateway: helps VPC connect with Internet, Public subnet has a route table entry to IG, NAT g/w helps private access internet through IG.
NACL: firewall which controld traffic from to subnet, stateless. Security Group: firewalls which controls traffic from EC2 etc, statefull
VPC flow logs: captures information about IP traffic going into your interfaces.
VPC peering: connect two VPCs
VPC Endpoints: connect two services using private n/w instead of public internet, VPC Endpoint Gatewat – S3, DynamoDB, rest as VPC endpoint interface.
WS PrivateLink — A technology that provides private connectivity between VPCs and services.
Site to Site VPN – Connect an on-premises VPN to AWS
Direct Connect -physical connection between onpremises and AWS

Security, identity, and compliance

ACM

Use AWS Certificate Manager (ACM) to provision, manage, and deploy public and private SSL/TLS certificates for use with AWS services and your internal connected resources. ACM removes the time-consuming manual process of purchasing, uploading, and renewing SSL/TLS certificates.

Cognito

Quick and Easy User Sign-Up, Sign-In, and Access Control to Your Web and Mobile Apps. Simple and Secure User Sign-Up, Sign-In, and Access Control. Get Started for Free.

IAM

IAM = Identity and Access Management, Global service
IAM Policy structure consists of
Effect: whether the statement allows or denies access
Principal: account/user/role to which this policy applied
Action: list of actions this policy allows or denies
Resource: list of resources to which the actions applied to
Condition: conditions for when this policy is in effect
IAM roles for the service which allows AWS services to perform actions on your behalf.

KMS

AWS Key Management Service (AWS KMS) lets you create, manage, and control cryptographic keys across your applications and AWS services.

Private Certificate Authority

AWS Private CA is a highly available certificate authority that helps organizations secure their applications and devices using private certificates.

Secrets Manager

AWS Secrets Manager allows you to rotate, manage, and retrieve database credentials, API keys, and other secrets through their lifecycle.

STS

AWS provides AWS Security Token Service (AWS STS) as a web service that enables you to request temporary, limited-privilege credentials for users.

WAF

AWS WAF helps you protect against common web exploits and bots that can affect availability, compromise security, or consume excessive resources.

Storage

EBS

Amazon Elastic Block Store (Amazon EBS) is an easy-to-use, scalable, high-performance block-storage service designed for Amazon Elastic Compute Cloud

EFS

Amazon Elastic File System (EFS) provides a simple, scalable fully managed elastic NFS file system for AWS compute instances.

S3

Simple Storage service stores objects in buckets.
S3 looks like a global service but buckets are stored in a region.
Each object have an key i.e folderpath + filename, folder path is just a key with slashes.
Max 5TB, >5GB use MultiPart upload.
S3 Security – IAM policies at the user level, Bucket policies from S3 Console allow cross-account, Object ACL, Bucket ACL(not recommend).
S3 allows versioning, cross-region replication
S3 Standard(99.99% available), Standard Infrequent Access(less called but same access speed), OneZone IA(1 az), Glacier Instant(mill retrieval once a quarter min 90 days)/Flexibe(varied times starting from 1-5 min min 90)/deep retrieval min 12 hours min 180days, Intelligent tiering(move from tier to tier depending on usage)
GetObject on S3 has a limit of 5500 GET per second, 3500 rest
S3 supports transition from one type, expiration of objects after no use, EventNotification
Increase transfer speed by using an edge location which uses private AWS to increase speed
S3 Encryption: SSE(Server side encryption )AES256, enableddefault – S3, SSE-KMS aws:kms, user control, SSE-C(Encryption key must provided in HTTP headers, for every HTTP request made), Client Side Encryption
S3 Access logs, MFA, . S3 Cross Region Replication
Access Points simplify security management for S3 Buckets – AccessPoint policy

S3 Glacier

S3 Glacier is a secure and durable service for low-cost data archiving and long-term backup using vaults.

Analytics

Athena