Software Architecture Books Summary And Highlights -- Part 11 Cloud

 

Cloud

Highlights

Container vs VMs:

1. container is smaller so easier to transfer on the internet
2. container layers make it lighter to move from one machine to another by only moving the changed layer
   1. e.g. an app could contain 4 layers and images of a container: pure linux distribution, apache server, db, php server;
   2. if only php server is changed, then only that layer of image need to be moved and others don’t need to be moved which has smaller network throughput
3. one service per container
4. VM can host more different types of software because it doesn’t rely on any OS

SAP ch 16 Virtualization

Autoscaler

Configurations to add and remove an instance

• The new VM image configurations and settings
• The CPU utilization threshold
• The network I/O bandwidth thresholds
• The minimum and maximum number of instances you want in this group

process of autoscaler removing an instance

1. autoscaler notify the load balancer to stop sending request to that instance
2. autoscaler notify the instance to terminate its activity and shutdown in case instance is in the middle of some requests
3. instance enters draining states
4. developer implement interface to receive instruction to terminate and drain an instance of your service

SAP ch 17 Cloud and distributed computation

Pod

A Pod is a group of related containers. In Kubernetes, nodes (hardware or VMs) contain Pods, and Pods contain containers.

The containers in a Pod share an IP address and port space to receive requests from other services.

The purpose of a Pod is to reduce communication costs between closely related containers.

SAP ch 16 Virtualization

Function as a service

since allocation of a container into a container runtime engine is fast, it is not necessary to leave the container running. We can afford to reallocate a new container instance for every request. this is how serverless work.

Cons: cold start latency, limited choice of types of container

SAP ch 16 Virtualization

Level of Abstraction: Serverless

just need to add action or handler. no need to setup a server

pros and cons:

First note that a serverless architecture requires your code to be truly stateless which means even no cache can be setup

In practice, most organizations have needs that cannot be served by truly stateless workloads.

a serverless application can easily scale down to zero; and so the cost of just owning it scales with the traffic while container cannot be scaled to zero

At the very high-traffic end, you will necessarily be limited by the underlying infrastructure no matter you use container or serverless

Finally, adopting a serverless solution implies a certain loss of control over your environment.

Trade off

serverless is good for small org to save container management cost.

SWG Ch 25 compute as a service

Compute as a service Over Time and Scale

Containers as an Abstraction

file system abstraction

A container also provides a simple way to manage named resources on the machine including network ports

Containers and implicit dependencies

container can also have many implicit dependencies which are antipattern and hard to remove.

One Service to Rule Them All

batch job and serving job shares same machine pool so that machine can be handled as cattle.

Serving job usually needs slack resource for spikes and peaks. Batch job can use the slack resource of serving job and can be killed when serving job needs the resource

Serving job and batch job still have different configuration because they have different requirements. e.g. different rescheduling strategy, different warning strategy when killing a job

Submitted Configuration

setup a borg job with standard configuration language instead of rpc or cli call which is easier to manage.

SWG Ch 25 compute as a service

Public Cloud Versus Private Cloud

Pros for using public cloud service: lower management overhead and scale the infrastructure more easily.

Especially for young organizations or products, predicting resource requirements is challenging, and so the advantages of not having to provision resources up front are significant.

One significant concern when choosing a cloud provider is the fear of lock-in—the provider might suddenly increase their prices or maybe just fail.

Solution: make it easy to migrate to other cloud services

1. use a lower-level public cloud solution (like Amazon EC2) and run a higher-level open source solution (like OpenWhisk or KNative) on top of it
2. multicloud: use managed services based on the same open source solutions from two or more different cloud providers
3. a hybrid cloud;
   1. your private infrastructure handles major loads,
   2. public cloud infra handles excessive load for scaling

SWG Ch 25 compute as a service

Related Chapters

SAP ch 16 Virtualization

SAP ch 17 Cloud and distributed computation

SWG Ch 25 compute as a service