VPC-Whitepaper

-In a one single Region/account, you can have multiple VPCs.
-VPC can exist in multiple AZs within the same region.
-By default all subnets within a VPC can communicate with each other.

-One Internet gateway per VPC. 

-First 4 and last 1 IPs are reserved in each CPC .

-Minimum CIDR /28 = 16 ad Max /16 65536 addresses. 

-VPC CIDR should not overlap with other network/corporate network.

-Source destination check must be disabled in NAT instance. 

-One Net gateway per AZ for high availability, 

-In Route table when the destination does not lead anywhere blackhole   is called.

Default VPC
Samples
Sales department: Do not want the servers to be shared across department.
-Create a Sales VPC. For eg. Based on the department or env I can create VPCs
Prod VPC/ Developer VPC.
10.1.0.0/16
All servers will have Network Id= 10.1

Sub network / Subnet:

- Always resides only in one availability zone.

- Every subnet can be associated with only one Route table. But a Route table can be associated with more than one subnets.

- Allowed subnet mask is between /28 to /16. This results 16 to 65,536 possible IP addresses in a VPC.

 - AWS reserves both the first four and the last IP address in each subnet's CIDR block. They're not available for use.

Sample:

Sales Subnet  10.1.1.0/24
I can divide  my servers into multiple subnets
Private IPS 10.1.1.222 : last octate is host id
Public IP is unique  that amzn is providing which would be unique over the internet

Prod VPC: 10.50.0.0/16
Subnet : Use smaller range as possible.
Prod Subnet -1  10.50.1.0/24
Prod Subnet-2 10.50.

Internet Gateway IGW:
- We need to create internet gateway then only we will be able to connect to the VPC
- VPC needs a gateway to connect over internet
- Attach IGW to VPC
- We need to define rule that subnet knows the gateway via creating route table.
- Add Route table entry
- Associate subnet association
- One Internet gateway per VPC
- All instances in the subnet must have either public IP or elastic IP
- Network access control and secuity group must be configured to allow traffic to and from the instances.

NAT Gateway:
- Private subnet gets internet connectivity through NAT network address transalation device.
- NAT: work as a proxy server
- NAT gateway : Chargeable
- Always created in a subnet which has internet access, ie within the public subnet. That will have route to the internate through internet gateway.
- NAT gateway cannot be created without elastic IP address. There is a possibility the EIP added manually is not assigned ot any instance.
- My private network running server not internet access NAT Internet
- It's a managed service and has better availability than NAT instance
-My isolated server
1) create a NAT gateway
2) Create new Route table  from NAT to my isolated server vpc
3) Subnet association. Select subnet prod
Even though I have not given public IP access we are able to connect internet  on the VPC not available over internet.

NAT Instance:
 -Anohter alternative to NAT Instance.
- Runs as an instance in the VPC.
Route Table:
- It knows where the network traffic would be directed by defining a set of rules.
- By deafult subnet has a default round table entry with allowing all local network within the saubnet.
- Each subnet must be assocaited with only one route table.
- public access 0.0.0.0/0 access to all/everything
Elastic IP addresses:
- By default when instance terminates, public IP address is released.
- If you want to retain public IP address even if the intence terminates. Static public IP address is 
 assigned to your account that can be used by the instances.
-This can be asigned to an isntance or network interface.
VPC Peering :
- One to one Peering.
- Applies always within the same Region
- Pairing is one to one relataionship. Transitive relationship is not allowed.
-VPCs with overlapping CIDR/ same IPranges wouldn't be paired

Internet/Private Gateway(VPC)--> Routing Table -->ACL-->Subnet -->Security Group --> Instance

Inter Communications
- Use inter-region VPC peering
- Implement a fully meshed architecture

- Software/Hardware based VPN.// bandwidth and latency problem.

- Direct Connect.

- VPN CloudHub.

VPC Endpoints: Enables private connectivity to services hosted in AWS, from within your VPC without using an an Internet Gateway, VPN, Network Address Translation (NAT) devices, or firewall proxies.
-To push messages to Amazon SNS topics from an Amazon VPC, create an interface VPC endpoint.

Security Group:
- It controls how traffic is allowed for instances basically inbound/outbond traffic.
-Always premissive.
-Are stateful. Response will always flow in regragrdless of inbound security rule.If traffic is allowed into an instance, the response is allowed out regardless of explicit outbound rules.

- Rules can be modified at any time and reflected immediately.
- Resides in a VPC
- IP Range. x.x.x.x MySQL port :1433,  RDP port 3389
- By default security group allow all outbond traffic, deny all inbound traffic. Allow traffic between instances.
-sample names: Webserver sec group, DB server sec group
-Security groups are stateful — if you send a request from your instance, the response traffic for that request is allowed to flow in regardless of inbound security group rules. Responses to allowed inbound traffic are allowed to flow out, regardless of outbound rules.

Network ACL:
- Sits between Route Table and Subnet. Applicable at subnet level.
- Inbound and outbond rule.
- Network ACL Outbond is not open by default.
- Each subnet must be assocaited with one ACL and ACL can be associated with multiple subnets.
- Statelss, it is possible inbound is allowed but not the outbond.
-Rules evaluated in order starting from lowest.

Flow logs:
- Data is stored using Cloudwatch Logs.
- VPC Flow Logs is a feature that enables you to capture information about the IP traffic going to and from network interfaces in your VPC. 
- At 3 different levels, VPC, Subnet and Network Interface levels. Gets stored into s3 and Cloud Watch
- Can be configured to stream to services such as Elasticache, or Lambda
- You cannot enable flow logs for VPC's that are peered with your VPC unless the peer VPC is in your account
- You cannot tag a flow log.

-Athena can be used to analysis and query VPC flow log. 
Best Practices: Use public and private subnets.
Default Limit 
 5 VPC per Region
200 Subnets per VPC
200 RouteTable per VPC
500 sec group per VPC
50 in/out bounrules per VPC
5 IPV4 CIDR blcok
5 Internet gateway per region
5 NAT gatway per AZ
200 Network ACLs per VPC
20 max number of rules per ACL
NateGatwway and VPN conenection is charged on hourly basis.
-Use Sec group for whitelisting traffic ACL for blacklisting traffic
VPC endpoint:
VPC endpoint enables you to privately connect your VPC to supported AWS services and VPC endpoint services powered by AWS PrivateLink without requiring an internet gateway, NAT device, VPN connection, or AWS Direct Connect connection. 
- Instances in your VPC do not require public IP addresses to communicate with resources in the service. Traffic between your VPC and the other service does not leave the Amazon network.

Even if no internet is there/needed, VPC instances would be able to access the AWS services outside VPC.

aws s3 ls --region eu-west-1

More:
https://thetechiehouse.blogspot.com/2020/02/vpc-deep-dive.html
https://aws.amazon.com/vpc/faqs/

Site to Site VPN: Customer Gateway + Virtual Private Gateway.

Virtual Private Gateway:

VPN Gateway on the VPC side.

VPG is created and attached to the VPC. Possibility to customize ASN.

Customer gateway at the Corporate DC

 IP Addresses of the customer gateway :

It's either the static internet-routable
IP address for your customer gateway device.
So it will have a static IP address.
Or, if it's behind a NAT, okay if it's behind a NAT, use the public address of the NAT.

Gateway VPC endpoints only for Amazon S3 Endpoints for Amazon DynamoDB. Rest of the services use Interface VPC endpoints :

Amazon CloudWatch

Monitoring all the components created in AWS.

RDS not out of storage. Check free stprage space in cloudwatch.
YMetrics are at hypervisor level, you cannot see memory metrics.

Status Check

Syatem status check: Physical box(server) where all VMS are hosted

Instance status check: VM

If issues, try rebooting the servers. Log a case with Amazon.

No additional charge for 5 mins interval. Basic monitoring is up tp 5 mins and is free. Granular level less than 5 min (up to 1min) the detailed/enhanced monitoring is chargeable.

Basic monitoring includes CPU load, disk I/O, and network I/O metrics; these are collected at five-minute intervals and stored for two week.

Memory monitoring: Not comes by default. Additional memory agent installation required, Memory monitor agents need to be installed on the instances then the memory can be monitored.

Create Aarm: Send notification to Topic. SNS, Simple notification service.

Amazone SNS: Pub/Sub or push notification service

Publisher -> [SNS Topic ]-Subscribers()

Subscribers// Lamda, SQS, HTTP/S, Email, SMS

EC2 Instances-> Create an Alarm à send notification to SNS topic // 

Trigger condition: Whenever CPU utilization goes up and down to x%

Take Action: Stop/Reboot/Recover the instances…

Dashboard

Metrics graph: select the metrics ->  Per Instance/individual m/c Auto scaling group metric,enter in the instance id.. / Select the metric name

Pricing:

1 Millon mobile push notification/ 100 sms/ 1000 email, 100,000 htts. Lambada/SQS notification free

EBS-Whitepaper

EBS:  (Elastic Block Store)

-Used for Create Partition/ File system, OS installation.

- Installation is possible on block storage.

-Whenever installation comes it has to be in EBS.
- Amazon EBS volumes cannot be shared between instances.
 -EBS Optimization is a network traffic optimization, it does not change the I/O speed of the volume. Istance store is cheaper and faster for performance.
 Instance Store Lifetime

Instance Store data
 - Persists only data in the instance store persis during the lifetime of its associated instance and  instance reboots. 

-Is lost when the instance stops/ terminates/ underlying disk drive fails.

Root primary/bootable disk

/dev/ssda1 :  Hypervisor Kernal is going to refer to

Snapshot contains image:

Magnetic  Old and slow ones

SSD : New generation/ performance guaranteed . Very high performance.

Provisioned  types: is the most suitable for performance

Provisioned IOPs:When Low latency and High throughput workload required. 4GB to 16 TiB volume supported

Throughput related to magnetic volume types

Name of hypervisor is : XEN which is based on Linux

When the server is storage, we are paying for the capacity (EBS). Because space is allocated for this,

All EBS comes from centralize storage system:

Actually the disk/EBS is lying on the central storage.

EBS volume: For any kind of installations.

Box : cpu+ mem+ IO+ disk /// + Local Disk

Local Disk:

Very fast accsss to local disk: Instance Store/Ephemeral Storage. Not persistent. I am goin to lose the data next time restart. They are local persisted disk. . Very high caching performance.

Instance Store added only during server creation.  Don’t use instance store when you have important data and want to preserve it.

We can take a back up of instance store.

Ebs using centralized storage, Ephemeral is using local storage

Snapshot is being created for each of the disk/storage we have in case of multiple volumes.
A snapshot is constrained to the AWS Region where it was created. 

For eg. Box D-drive--à Snapshot Snap-D

We can restore my drive from the snapshot.

Image contains references to multiple snapshot. If the Image is using the snapshots they cannot be deleted.

So first I need to delete the Image/AMI first. Then only I will be able to delete the snapshots. Essentially the actual data is sitting in the snapshot, so we are charged for the snapshot. For eg, how much space it is occupying etc.
 You can back up the data on your Amazon EBS volumes to Amazon S3 by taking point-in-time snapshots.

If you have database on D drive, we can take a snapshot of D drive.

AMI/Image:[snapshot1 + Snapshot2…]

When we create an AMI, there is an underlying snapshot that also get created. So if we deregister an ami, its snapshot should also be deleted  otherwise it will cost.

 Snapshots across Region : Making it possible to use multiple Regions for geographical expansion, data center migration, and disaster recovery.

EC2-I-Whitepaper

Clustered Placement Group: Within the same AG, when low network latency /high throughput required. They are always in the same AZ and region. Only certain instances can be launched in to a clustered placement group. Cannot spread multiple AZs.

Spread Placement Group: Individual critical EC2 instances. Each instances are placed on different underlying hardware. For application that have a small number of instances and that should kept separate from each other to have least impact from failure.They can spam in multiple AZ’s within the same region. Max  7 running instances per AZ. Here the VMs are on different hypervisor , its alowed to use different instance type.
-Max 7 instances per AZ in a group allowed.
-Not supported for dedicated hosts or instances option.

Portioned placement group: Each partition within a placement group has its own set of racks. Each rack has its own network and  power  sources. No two partitions within a placement group share the same racks allowing you to isolate the impact of hardware failure within your application.Every instance has replica.

Multiple EC2 instances HDFS, HBase, Cassandra . They can spam in multiple AZ’s within the same region.

The name specified for placement group must be unique within my aws account.

Only certain types of instances can be launched in a placemen group. (Compute Optimized/ GPU/memory/Storage optimized).

-Recommended homogeneous instances within clustered placement groups.

-Placement groups can’t be merged

-You can’t move an existing instance into a placement group. You can create an AMI from existing instance. And launch a new instance from AMI into a placement group.

EC2:

On Demands

Reserved : 1/3 yrs contract

Spot : Stock Market. If I terminate the instances it will be charged for any hour in which the instatnce Run, amzn does not charge for partial hours for every single instances,

Dedicated Hosts: When regultatory compliance reqiored or server bound licensing neede,

Instance Typs: FIGHT DRMCPX ZAU

If IP address is changed use, Elastic IP address

EBS:

A virtual  hard disk in cloud

Ec2 instances : You must turn on termination protection.

EBS bacled instances. Root EBS volume to be deleted when the instance in terminated. Additionaly attached volume wont be deleted by default.

I can detach additional EBS volume without stopping the instance.

An EBS volume can be attached to only one instance at a time, but multiple volumes can be attached to a single instance

EBS root volume can be encrypted.

https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/ebs-volume-types.html

Root volume persists when the instance stops but deleted when instance terminates.
Data on any other ebs volume persists during termination.

Security Group:

-Stateful.

All inbound traffic is blocked by default. All Outbound traffic is allowed.

Change to security group takes effect immediately

You can any number of Ec2 instances running within a security group.

You can have multiple security group attached to  the Ec2 instances.

Network access control list Block IP addresses

EBS Types:

General purpose to throughput optomised HDD.

Provioson IOPS SSD : data base 4 GB to 16 TB

General purpose SSD: 1 GB to 16 TB

For less frequently workloads: Lowest cost : Cold HDD

Smapshot Exists in S3 and is incremental.

For cosnsistencey stop the instances before taking a snapshots.

AMI can be created from both volume and snapshots.

You can change EBS volume on the fly including size and storage types.

EBS Volume will always be in the same AZ as the EC2.

To migrate EC2 volume from one AZ to another, take a snapshot and create AMI from the snapshot . Use AMI to launch EC2 insances in a new AZ.
Instance Profile:
- An instance profile is a container for an IAM role that you can use to pass role information to an EC2 instance when the instance starts.

- With IAM roles for Amazon ECS tasks, you can specify an IAM role that can be used by the containers in a task.

 - Amazon EC2 instance profiles provide credentials to EC2 instances.

Instance Store volume /Ephermal Storage:

Temporary block level storage for your instace. When we terminate the instance, Instance store is also gets lost. It resides to the diska that are physically located on the same node.
Instance Store volume can be added only to the selected instance. Not available for all types of instances.
Instance store can only be in running or terminated states.
Instance store size depneds on the instance type selected at the time of launc of instance.

Bootsrap Script: Run when Ec2 isntance starts. Useful for installation software and updates.

Cloud HSM : Used for EC2 encryption.

Instance metadata :  Data about your instance that you can use to configure or manage the running instance. Instance metadata is divided into categories, for example, host name, events, and security groups.
User data: You can also use instance metadata to access user data that you specified when launching your instance. 

/latest/meta-data

/latest/user-data

http://169.254.169.254/latest/meta-data

The Instance Metadata Query tool allows you to query the instance metadata without having to type out the full URI or category names.

https://cloud.netapp.com/blog/ebs-volumes-5-lesser-known-functions

- Instances are backed up using AMIs.
ELB:

When there is huge traffic, network load balancer suits best for high performance.

Classic load balancer gives both option https/s and tcp (application and netwok) level balancer   .

EC2 Auto Scaling:
-If connection draining is enabled, Auto Scaling waits for in-flight requests to complete or timeout before terminating instances.
Instance/Terminating Scaling In Logic:
1> If there are instances in multiple availability zones, select the AZa with maximum number if instances. 
2> Then it select the instance with oldest launch confiIguration will be terminated.
3> If there are multiple instances using the oldest launch config, choose the instances which is closest to  next billing hours.
Cooldown period
- The health check grace period allows a period of time for a new instance to warm up before performing a health check (300 seconds by default)
-With a cooldown period in place, the Auto Scaling group launches an instance and then suspends scaling activities due to simple scaling policies or manual scaling until the specified time elapses. (The default is 300 seconds.) 

  

Both the ALB and NLB support WebSockets. However, only the ALB supports HTTP/HTTPS listeners. The NLB only supports TCP, TLS, UDP, TCP_UDP.
- The CLB does not support WebSockets.
- A “Layer-4 Load Balancer” is not suitable, we need a layer 7 load balancer for HTTP/HTTPS.

Life Cycle Hooks: Adding life cycle hooks to Auto scaling group puts the instance into waiting state. During this time you can perform custom activities to retreive critical operational data from a stateful instance.

Instance Launch: Pending -> Pending wait-  pre launch  custom action > In Service.
In Service Launch: Pending -> Pending wait-  pre terminate custom action  > Terminated.