DynamoDB Walkthrough

DynamoDB best practices include:

- Keep item sizes small.

- If you are storing serial data in DynamoDB that will require actions based on data/time use separate tables for days, weeks, months.

- Store more frequently and less frequently accessed data in separate tables.

- If possible compress larger attribute values.

- Store objects larger than 400KB in S3 and use pointers (S3 Object ID) in DynamoDB.

-Serverless

-Suited for Read heavy operation.
-VPC endpoint is avaiable.
Encryption in transit using ssl/tls. at rest using KMS.
Glonal Tables- Cross Region Replication. Useful for low latency, DR purpose, Must enable DaynamoDB stream.
If burst credits are empty, we will get a 'ProvisionedThroughputException'
-Eventual consistency takes around 1 sec. Enable Strong consistency for latest data.
-You are charged on basis of Provisioned throughput configuration.
-Combined key or Range key.

Hash and Range key: This is used when we want primary keys to be dependent on more than one columns. eg. Id and firstname if set both cannot be null

 Hash and Global secondary index . The attribute can be null

Secondary Index:
-A subset of attributes from a table, along with an alternate key to support Query operations.
- A table can have multiple secondary indexes, which give your applications access to many different query patterns.

- The key schema for the index—every attribute in the index key schema must be a top-level attribute 
Global secondary index:
- An index with a partition key and a sort key that can be different from those on the base table ie. different hash/range keys per index.
- Queries on the index can span all of the data in the base table, across all partitions. 
 A global secondary index is stored in its own partition space away from the base table and scales separately from the base table.
-Separate throughput can be provisioned.
-Eventual consistency in  writes due to asynchronously replication .
- You can use either a simple or a composite key schema.
- With global secondary index queries or scans, you can only request the attributes that are projected into the index. DynamoDB will not fetch any attributes from the table.
There are no size restrictions for global secondary indexes.

Local secondary index 
- An index that has the same partition key as the base table, but a different sort key.One hash key per table
- Every partition of a local secondary index is scoped to a base table partition that has the same partition key value.

You do not need to specify read and write capacity unit settings.
DAX:
% mins TTL for cache by default.
Streams has 24 hrs of data retention
Transactions supported/ On Demand.
On Demand: When spikes are unpredicatble or the application is very low throughput. More expensive than Planned capacity provision.

A DynamoDB Accelerator (DAX) cluster is a cache that fronts your DynamoDB tables and caches the most frequently read values. 

AWS Application Services

AWS CloudTrail:  for AWS account related action.Actions taken by User, Role and aws services are recorded as events in CloudTrail. For governance and complaince.

-Can be configured to log Data events and management events.
- CloudWatch Logs: to monitor, store accesslog from EC2, lcoudTrail, Rote 53 and other logs.

Custom Metrics. Metric Resolution Standard 1 min, best upto 1 sec. API: putMetricData.

CloudWatch Dashboard: Are global. Can includes graph from different regions.
- By default, CloudTrail event log files are encrypted using Amazon S3 server-side encryption (SSE).
- You can also choose to encrypt your log files with an AWS Key Management Service (AWS KMS) key.
-Management and Data Events Logs. Management events is enabled by default.
- Past 90 days  history by default.
- Adds checks for Amazon S3, Amazon Redshift, EC2 Reserved Instances, security, and service limits
 You cannot configure global service event logging from the CloudTrail console.
CloudWatch is for capturing performance metrics , monitoring application logs wherein CuoldTrail for tracking user activity.

CloudWatch Log Insights- To query logs and add queries to CloudWatch.

CloudWatch Agent && Unified Agent (lot more capabilities).

CloudWatch Alarm: Alarm state OK/Insufficient data/alarm. status check instance/system status check will have action 'Ec2 Instance Recovery' if configured.

CloudWatch Events:add rules and target. 

AWS Config:

- Possibility of storing the configuration into S3 and analysed by Athena.

 -View compliance.configuration/CloudTrail API calls
Support for Encryption using SSE-KMS –
CloudTrail will encrypt the log files using the KMS key you specify.

Log File Integrity Validation –
validate the integrity of the CloudTrail log files stored in your S3 bucket and detect whether
they were deleted or modified after CloudTrail delivered them to your S3 bucket as part of security and auditing discipline.

Trusted Advisor : (Customized cloud expert)
AWS Trusted Advisor offers a rich set of best practice checks and recommendations across five categories: cost optimization; security; fault tolerance; performance; cost optimization and service limits.

VM Import/Export:
-To easily import virtual machine images from your existing environment to Amazon EC2 instances
and export them back to your on-premises environment.

AWS OpsWorks Stacks :
 you can model your application as a stack containing different layers, such as load balancing, database, and application server.
A common practice is to have multiple stacks that represent different environments.

CloudHSM:
 - To processes cryptographic operations and provides secure storage for cryptographic keys.
 - To encrypt its data, the HSM uses a unique, ephemeral encryption key known as the ephemeral backup key (EBK). It stores the encerypted data key to S3 in the same region as HSM cluster.

 Amazon Polly:
- Text-to-Speech (TTS) cloud service that converts text into lifelike speech
- If single word repeated multiple times t have different speeches, there needs to have multiple      Lexicons created.
 - Using SSML tag, speech is controlled. commas converted into Period and tag words and  Paragraphs as "strong" will help to control the speech speed.

AWS Cloud Support contract:
  Response time  : critical < 15m enterprise a/c ,  urgent < 1 hour business a/c. normal <12 hr developer, basic -n/a

AWS Storage Gateway
To seamlessly links your on-premises environment to Amazon cloud storage.
It offers local storage with highly optimized connectivity to AWS Cloud storage, and helps with migration, bursting and storage tiering use cases.

AWS Pilot Light:
Run a minimal version of the environment in cloud.
r. A small part of your infrastructure is always running simultaneously syncing mutable data (as databases or documents), while other parts of your infrastructure are switched off and used only during testing. RDS/ EC2, AMI,ELB.
WARM Standby:
Apps server already running.Always running the critical system. RTO low/medium, RPO: Low. Only need to simply reroute live traffic using Route 53 .

Multi-Site: (Active-active configuration)
50% weighted routing  using Route 53

AWS Direct Connect:
Alternative to using Internet betwer on-premise data and AWS resources.

Elastic Transcoder:
Media transcoding service in cloud. Media can be converted into anoher format as per device or users.
Amazon Workspaces:
-desktop as a service/virtual desktop. data backup every 12 hours
AWS WorkDocs:
- To Share contants. Only Power users can share publicly

AWS KMS 
Customer managed CMK will generate plain text Data Key & encrypted Data Keys. documents will be encrypted using these plain text Data Keys. After encryption , plain text Data keys needs to be deleted to avoid any inappropriate use & encrypted Data Keys along with encrypted Data is stored in S3 buckets.
Amazon Deploy:
  Amazon Deploy is the simplest way for developers to get paid for Amazon EC2 machine images (AMIs) or applications they build on top of Amazon S3.
   Developers use the simple Amazon Deploy web interface to register their application or AMI with Amazon Deploy and configure their desired pricing.
AWS CloudFormer:
- A template creation tool and it creates AWS CloudFormation template from our existing resources in AWS account.
- We can select any supported AWS resources that are running in our account, and CloudFormer creates a template in an Amazon S3 bucket.

Amazon FPS: Amazon Flexible Payments Service, developers can accept payments on websites. It has several innovative features, including support for micropayments.
Amazon DevPay: supports applications built on Amazon S3 or Amazon EC2 by allowing you to resell applications built on top of one of these services.
AWS Organizations: Have Managed Service Control Policies.

AWS Pipeline:
  Source repository or S3 - console  created t, CodePipeline creates an Amazon CloudWatch Events rule that starts your pipeline when the source changes.
 GitHub repository,: console  created , CodePipeline creates a webhook that starts your pipeline when the source changes.
CICD Pilpleline: CICD orchastration service to deliver all teh way to Elastic ebanstock.
 If created through CLI, disable periodically checking the resources
AWS codecommit: AWS in-house code repositiry: Will be useful when build size is lesser.
AWS code deploy : It is for existing deployment infrastucture. If a new resource needs to be deployed then Elastic Beanstock is a better option.
-Dploying our code directly onto EC2 instance or on premise server. We can define strategy how fast  the rollout of ew code should be.
AWS Glue: ETL, serverless, have integration with Spark etc. Arora,RDS --> S3, Redshift.
EMR : Big Data Processing.
Consolidated Billing: When other accounts have launched unused instances from same A.Z. and should not be part of same VPC.
AWS SSO:
 -To centrally manage SSO access to all of your AWS accounts and cloud application
- Users in your on-premises Active Directory can also have SSO access to AWS accounts and cloud applications in the AWS SSO user portal
Following two options available:
1. Create a two-way trust relationship
2. Create an AD Connector,  A directory gateway that can redirect directory requests to your on-premises Active Directory without caching any information in the cloud.
 If you are connecting AWS SSO to an AD Connector directory, any future user password resets must be done from within Active Directory. This means that users will not be able to reset their passwords from the user portal.

EFS: 
Can be connected across AZ. Encryption at rest duering ceration time. Across VPC, peering required. Port 2049 NFS needed to acess it.At transit encrypt using mount helper.

Enabling encryption of data in transit for your Amazon EFS file system is done by enabling Transport Layer Security (TLS) when you mount your file system using the Amazon EFS mount helper.
Perfomrmance :
-Max I/O mode when too many EC2s (100s, 1000s )are connected. Slight latency cost.
-General Purpose: recommended for low latency.
-Procvisioned : throughput mode =provisioned, it can be incereaed irrespective of file size.
AWS STS:
 Returns a set of temporary security credentials (consisting of an access key ID, a secret access key, and a security token) for a federated user.

Amazon FSx for 

     1)Windows File Server

         -supports DFSN and is the most suitable storage solution for Microsoft filesystems. AWS                    DataSync supports migrating to the Amazon FSx and automates the process.

    --FSx for Windows File Server provides fully managed, highly reliable, and scalable file storage that is accessible over the industry-standard Server Message Block (SMB) protocol

    II) for Lustre is a fully managed shigh performance computing (HPC),

Amazon FSx  

-Systems Manager Parameter Store provides secure, hierarchical storage for configuration data management and secrets management.

When an EBS volume is encrypted with a custom key you must share the custom key with the PROD account. You also need to modify the permissions on the snapshot to share it with the PROD account. 

EBS Performane Tuning

AMI - Template required to launch an instance. Includes customization done on top of base AMI.
Virtualisation: -HVM Hardware assisted VM. AWS  Advised to use this though supports both.
                        -Para Virtualisation
Hypervisor keeps instances separated running on same physical m/c.
Hypervisor  keeps Instances pyhiscal network interface and virtual network interface separated using firewall
You can select AMI with EBS or instance store as root device type. AMI is cretaed from Instance.

EBS can be attched or detached but ephenrmal storage/instance store can't be done so.
AMI convention:
General purpose - T
Compute optimized - C
Memory optimized - X
Accelerated Programming- P
Storage optimized- H
Instance store:
EBS optimized: better perofmance with  additional dedicated throughputs for EBS input/output

More IOPS : SSD based drive
  Genral 3 IOPs/Gib
  Provisioned: more than 100 to 32000  IOPS,
More throughputs required :  HDD based drive. Infrequently used.
     - Throughput optimaised HDD: Throughput intensive workload. Anohter type is cold HDD.
     - Cannot be used as root volume of the Ec2 instance.
Volume
- Root volume cannot be detached.
- Volume and the instances need to be in the same AZ.
- We can have many voume attached to one instance but cannot have many instances attahced to one volume.
- Volume can be upgraded without affecting instances.
AMIs are region specific.
EBS Volume types can be changed at run time without terminating instaces.
EBS volume and instances must be in the same availability zone.

Encryption:
- Uses 256-bit AES alogorithm.
- Uses KMS master key to encrypt the volume and snapshot created from those.
Snapshots:
- Point in time backup data of a EBS volume. Incremental backup.
- Volume created from replciated snaphot loaded lazily.
- Snapshot  is regin specific and supports cross region copy.
- Snapshots are incremental volume and stored in S3.

EBS Volume Performance :
- Use EBS optimized instance.
- Modern Linux Kernel.
-Use RAID 0 to Maximize Utilization of Instance Resource

AWS Cloudwatch Metrics for EBS:
- Provide Volume status check.

Raid 0: Increase Performance
Raid 1: Increase Fault Tolerance

VPC - Deep Dive

NAT instance can work as a bastain server, NAT gateway cannot be used.
NAT instances supports IP faragmented packets of TCP, UDP and ICMP protocol, NAT gateway will drop them.

VPC Peering:
Communication between two VPCs. Route the traffic between them using privare IP addresses of the instances.
- You cannot edit VPC peeering connection once established. Also cannot attach or detach an existing VPC peering connection.
-Requester VPC sends request to the acceptor VPC.
-Requester route table will have an entry with destination as  IP address range of the acceptor and target as VPC perring. Similarly Acceptor will have route table entry for  the other VPC ip address.

-Transitive VPC peering not supported. A<--->B B<---->C  does not mean A<---> C

VPC Endpoints:

- Enable us  connect VPC resources and endpoints privately  without using NAT instance/Internet Gateway/ VPN/AWS direct connection.
-The instances in VPC do not required public IP addresses  to communicate with te reqources in the ed point services.
  1. Interface endpoitns: An elastic network interface with a private IP address that serves as an entry point.eg. cloud formation, cloudwatch etc.
2. Gateway endpoints: Endpoint is a gateway which is  a target for a specific route in the route table.
We have a separate entry in the route table wheere the destination is the prefix ID of endpoint (eg. s3) private ip address of the connection target say S3 endpoint and target is VPC endpoint. Communication is done through aws private IP addresses and not through internet.
- Gateway endpoints are not supported outside VPC.

-Endpoints are supported within the same Region only. You cannot create an endpoint between a VPC and a service in a different Region.

-You cannot transfer an endpoint from one VPC to another, or from one service to another.

- S3 and DynamoDB two services have a Gateway endpoint (remember it), all the other ones have an interface endpoint (powered by Private Link - means a private IP)

How to avoid multiple VPC Peering:

AWS Private Link(VPC end point Services)

VPC Flow Logs: 
- Capture IP traffic going to and from network interfaces in a VPC.
- Flog logs can be sent to S3 buckets or Clousdwatch.
DHCP traffic,  reserved IP addresses for defaut VPC,  endpoint network interface between NLB network interface are not captured.

VPC Gateway:
- Enables you to privately connect your VPC to supported AWS services and VPC endpoint services powered by AWS PrivateLink without requiring an internet gateway, NAT device, VPN connection, or AWS Direct Connect connection.

VPC Direct connect: It does not require Internet.instead rely on dedicatd private network between on-premise internet and AWS VPC.

#When there is need to increase bandwidth throughput, especially when you're working with large data setsand you want lower cost on your bandwidth.
#Maybe sometimes you need more consistent network experience, because you're experiencing data drops,you're experiencing connection shutdowns.
#You want to have real-time data feeds on your application, they're shutting down too often.
Or maybe you want to just have a hybrid environment of on-premise data center and cloud data center.
Because it also supports both IPv4 and IPv6.

Connection Types: Dedicated connections, Hosted connections. Takes 1 month to setup a new connection.

Data in-transit is not encrypted but is private. Use direct connect +  VPN to IPSec  encrypted private connection.

VPC Direct connect Gateway: To connect with multiple VPCs.

The VPCs must not have overlapping CIDERs 

They are not replacing Peering connection.

 
AWS VPN CloudHub:

 When you have multiple branch offices and existing internet connections and 

 would like to implement a convenient, potentially low cost hub-and-spoke model for primary or backup connectivity between these remote offices.

The remote network prefixes for each spoke must have unique ASNs, and the sites must not have overlapping IP ranges. 

Each site can also send and receive data from the VPC as if they were using a standard VPN connection.

AWS Site to site VPN connection

Customer Gateway Device
- A customer gateway device is a physical device or software application on your side of the Site-to-Site VPN connection

- When you create a virtual private gateway, you can specify the private Autonomous System Number (ASN) for the Amazon side of the gateway. If you don't specify an ASN, the virtual private gateway is created with the default ASN (64512)
-The public IP address value must be static. If your customer gateway is behind a network address translation (NAT) device that's enabled for NAT traversal (NAT-T), use the public IP address of your NAT device, and adjust your firewall rules to unblock UDP port 4500.

Custom VPC:

-enableDnsHostnames :whether instances launched in the VPC receive public DNS hostnames that correspond to their public IP addresses, default false.

- enableDnsSupport:  whether DNS resolution through the Amazon DNS server is supported for the VPC.Default true.

Egress Internet Gateway:

-Only Outgoing, Only for IPV6

-NAT is for IPV4

 

Transit Gateway: (* network)

-To simplify network topology.

-Can peer Transit Gateways across regions.

- connects VPCs and on-premises networks through a central hub. 

-Simplifies your network and puts an end to complex peering relationships. It acts as a cloud router – each new connection is only made once.

Supports IP Multicast

-An Interface endpoint uses AWS PrivateLink and is an elastic network interface (ENI) with a private IP address that serves as an entry point for traffic destined to a supported service. Using PrivateLink you can connect your VPC to supported AWS services, services hosted by other AWS accounts (VPC endpoint services), and supported AWS Marketplace partner services.