Saturday, November 30, 2019

A Deep Dive into Containerization


Application Containerization:
  • OS-level virtualization , used to deploy and run application without launching entire VM for the respective app.
  • Needs to ensure the application is tested in container before shipping to higher environment. Dependency and security, access might be different in the next envs.
  • Wrap your code to lightweight container and pass it on to Ops team to deploy it to prod. 
  • You can setup your own repository as needed.We can create image using current container as well.
Commands:
Docker pull : download image and put in your local m/c
Docker run:
Docker stop
Docker exec  -it <image_name>  /bin/bash   //Docker enter into the container 


Docker commit <image_name>  repo_name/image_name:some_tag
Docker login  default/<repo>
Docker push >  repo_name/image
From command is used to specific the base image/ reference branch on which we are going to create image. Further layer on top of it.
Sample File:
FROM ubuntu
//FROM  repo_name/<ubuntu_image_name>
// From google? Login to google repo: docker login url
RUN apt-get -y install apache2
Add <source_location> <destination>
Add .  var/www/html
CMD apachectl  -D FOREGROUND
ENTRYPOINT apachectl  -D FOREGROUND
COPY:
CMD is used to run on the start of the container. ‘apachectl  -D FOREGROUND ‘ Used to keep the container up and running.
ENTRYPOINT: can’t be override using argument
For eg. Docker exec -it <image_name> /bin/bash . Here /bin/bas is getting overridden using CMD
Every container will require a single process which should always keep on running at the container start. Apachectl command is going to be executed at the time of container start.

Put a script , ru a while loop using sleep command. Sleep command will make sure that conainer is always running. So pass on any command that keep the container up and running.
Sample2 - v1
Vim Dockerfile // when we run the build, it will always look for Dockerfile
FROM ubuntu:latest
RUN apt-get update
RUN apt-get install apache2
ADD index.html /var/www/html
ENTRYPOINT apachectl  -D FOREGROUND
ENV name DEVOPS
Each line adds a separate layer. Need to achieve the target with minimum number of run command.
Run command will be executed at the time of creating the image


Sample 2 - v2

FROM ubuntu:latest
RUN apt-get update && apt-get install apache2
ADD index.html /var/www/html
ENTRYPOINT apachectl  -D FOREGROUND
ENV name DEVOPS
ENV name =DEVOPS name1=hello-docker

Whenever there is a public stuff, while pulling login is not required, During push it does require.
Docker build -t <repo_name>/<image_name>:version_0.1 // create
Docker push <repo_name>/<image_name>:version_0.1

docker run >-d <image_name>
docker inspect container_id  . It runs on a private IP. Need to expose the container to the host m/c in order to access it.
docker run -p  host_mc__port_81: container_port_80 -d image_name.
docker  run -p 81:80 -p 82:8080
Ultimately we are going to hit host m/c not the container
External_ip:exposed_ip
Netstat  -tunlp
Docker rm image_name
EXPOSE

Saturday, November 2, 2019

Data Safety in RabbitMQ

We will walk-through the data safety mechanism supported in RabbitMQ.
Mnesia is distributed database system RabbitMQ usages to store information related to queue, binding, exchange, etc.

All queues are persisted to DB. Queue marked durable survives node restart, system crash and network failure.

Clustering with Queue mirroring:
-Join multiple nodes to a cluster. Add additional redundancy through mirroring.This will replicate queue across multiple nodes.
Reads and writes will happen only through master node.
Fig 2. A mirrored queue.

Mirroring is achieved by setting ha policy
ha-mode: all/ exactly, ha-params: 2 (one master and one mirror) /nodes, ha-params: rabbit@node1, rabbit@
-Publisher receives confirmation just when the message is written to disk.
-When a broker dies, the cluster looks for the oldest mirror and make it as master node.
-Synchronization happens with the newly labeled master node to other mirrors.

Detecting Network Partitions:
While a network partition is in place, the two (or more!) sides of the cluster can evolve independently, with both sides thinking the other has crashed. This scenario is known as split-brain.

Nodes determine if its peer is down if another node is unable to contact it for a period of time, 60 seconds by default. If two nodes come back into contact, both having thought the other is down, the nodes will determine that a partition has occurred. This will be written to the RabbitMQ log in a form like:

=ERROR REPORT==== 15-Oct-2012::18:02:30 ===
Mnesia(rabbit@hostname): ** ERROR ** mnesia_event got
{inconsistent_database, running_partitioned_network, hare@hostname}

rabbitmqctl cluster_status shows  {partitions, # =>  {partitions,[{rabbit@smacmullen,[hare@smacmullen]},

RabbitMQ also three ways to deal with network partitions automatically: pause-minority mode, pause-if-all-down mode and autoheal mode. The default behaviour is referred to as ignore mode.
 pause-minority mode RabbitMQ will automatically pause cluster nodes which determine themselves to be in a minority.
pause-if-all-down mode, RabbitMQ will automatically pause cluster nodes which cannot reach any of the listed nodes. In other words, all the listed nodes must be down for RabbitMQ to pause a cluster node.
Autoheal mode: Instead, the cluster decides for itself which side of the partition must throw away its data. This mode is nice for availability and low overhead administration, but potentially worse for data loss.