[Links of the Day] 09/06/2020 : #WASM on #K8S , Fast Anomaly Detection on Graphs, Linux one ring

• Krustlet : it seems that web assembly is getting more pervasive, we have kernel WASM, WASM for Deep learning, now Krustlet offer WASM for Kubernetes via Kubelet.
• Fast Anomaly Detection in Graphs :  Really cool real-time anomaly detection on dynamic graphs.  the authors claim to be 644 times faster than SOTA with 42-48% higher accuracy. What is even more attractive is the constant memory usage which is fantastic for production deployment. [github] 
• io_uring : this will dominate the future of the Linux interface. It is currently eating up every single IO interface and probably won't stop just there. 

Google Cloud GKE control plane price introduction: the tragedy of the commons or bait and switch?

Recently when GCP announced that On June 6, 2020, Google Kubernetes Engine (GKE) clusters will start accruing a management fee. The fee is $0.10 per cluster per hour which amount to roughly $73/month. Everybody using GKE hit the roof as it was seen as yet another flakiness episode of the chocolate factory. A lot of existing customers are seeing themselves trapped into a bait and switch tactic by Google however the story is a little bit more complex than what it appears.

It seems that Google made a series of mistake in its rush to try to attract enterprise customer with it’s Kubernetes offering.

The first mistake is to have used the free control plane as a "loss leader." GKE provides the manager node, and cluster management so that it’s customers don't have to. And in exchange, you sell more compute, storage, network, and app services. 

The side effect of not charging for the control plane and charging for the control plane leads to two very different Kubernetes architectures. Small, single application clusters are simpler to set up and operated.  With the free control plane, customer embraced this approach as they didn’t need to architect their cloud infrastructure in a multi-tenant fashion.

Moreover, as per google docs, those decisions made at the start are very much set in stone. Customer cannot change their cluster from a regional cluster to a single zone cluster for example. So Google has customers who built their stacks taking into account Google’s free control plane, and GCP is turning the screws in by adding a cost for it — but they cannot change the type of their cluster to optimise their spend, since, per your docs, those decisions are set in stone. Hence the entrapment feeling that a lot of existing clients feel at the moment.

The second key metric they missed when announcing the free control plane is that the majority of Kubernetes deployments tend to have a single application to cluster mapping. So it would have been normal to assume that most of their potential customers would have started with small single app cluster deployment as they didn’t have the natural inertia brought by the cost of running the control plane. 

Third, as per their own metrics, they discovered that customers will use and abuse those free resources. It’s the tragedy of the commons where all those empty clusters cost Google money. 

Obviously, Google hoped that their customers would have applied their best practices and deployed multi-tenant cluster. Multitenant clusters are harder to manage, deploy and maintain.

And no amount of "best practice" documentation will solve this. However, it is not as simple and not every company is a hyper-scale corporation like Netflix and al.  Engineering is about balancing cost and benefit. Often the best practice to have many clusters for a variety of reasons such as: "Create one cluster per project to reduce the risk of project-level configurations". And company are ok with the waste as long as their software & deployment practices can treat any hosted Kubernetes service as essentially the same. Often corporation accepts waste as part of the inherent cost of not rearchitecting their process and culture. It's often more efficient to simplify the infra complexity albeit extra cost than trying to re-architect the company IT structure to embrace the latest best practice. It's a simple cost/risk/ROI analysis. They are even more ok with waste when Google fitted part of the bill with their free control plane. 

In the end, the folks at Google cloud fell between a rock and a hard place. They fell to the trap of the tragedy of the common hoping that all their customer will run their operations like google and now are trying to recoup those extra $$ by introducing a cost for the control plane. By doing so Google did the equivalent of adding a tax to running Kubernetes clusters on GCP. This is perceived as an ex-Oracle way of thinking, "what can we do to meet growth objectives," "how can we tax the people who we own".

To some extent, this is the equivalent by Google of a carbon or petrol tax. Customers need now to rethink their strategy, I.e. adopt public transport ( multi-tenant cluster ) or move to electric (cloud run). Some might move away completely from GCP because of the perceived lack of stability of the offering both in term of services and pricing.

[Links of the Day] 05/03/2020 : Linux AI tuning, easy AutoML , lightweight container development environment

• OpenEuler :  Huawei Linux distribution, interesting side project is A-tune which relies on AI for identifying the workload that runs on your the OS and tries to tune it to optimise its performance.
• AutoGluon : AutoGluon enables easy-to-use and easy-to-extend AutoML with a focus on deep learning and real-world applications spanning image, text, or tabular data. [github]
• k3c : kubernetes but lightweight and easy to use for container development

[Links of the Day] 26/11/2019 : MD + GIT = Powerpoint , K8s minus 5 , Polyglot Notebook

• GitPitch : markdown powered presentation tool. I'm lazy and I feel that powerpoint/google doc does the trick for me. However, if you are willing to go past the learning curve this seems to be a great tool! Hey, I still prefer to write my document in Latex.. So who am I to judge :) [github]
• k3s : Kubernetes but lightweight. 5 times less to be precise ( hence the 3 = 8 - 5 )
• Polynote : polyglot notebook by netflix. I really like the concept of notebook IDE as a service. This is a must for any organisation that heavily rely on data science to optimise their operations.  [github]

[Links of the Day] 17/10/2019 : Kubernetes single node cluster using NIX , Devops Wallboard, Json CLI tool

• kubernix : Single dependency, single node Kubernetes clusters for local testing, experimenting and development. This project uses NIX as it's underlying functional package manager.
• mirrorgate :  this looks like a better version of Hygiea from capital one. From a quick glance at the code, it seems easier to deploy and extend. I think I will give it a spin and see what we can achieve with it. I strongly feel that wallboard application are essential for effective SRE/Devops and provide direct feedback to the teams while contextualizing it with business process output. What's probably missing from this version is the "executive" wallboard that Hygiea offers.
• jtc : select one or multiple elements from a source JSON and apply various actions on the selected elements at once. Another tool for your swiss army knife bash script.

[Links of the Day] 15/10/2019 : k8s python operator, Strangeloop notes, Machine learning UI framework

• Kopf : Kubernetes Operator Pythonic Framework— is a framework and a library to make Kubernetes operators development easier, just in few lines of Python code.
• Strange loop notes : some quick and insightful notes on this year strange loop.
• StreamLit : app framework specifically for Machine Learning and Data Science teams. It helps you build quick UI around your ML models and get the result out quick to the end user. [website]

[Links of the day] 03/10/2019 : Kubernetes policy controller, Bash bible, git blame someone else

• Gatekeeper : a policy controller for Kubernetes. It is based on the openpolicy agent (OPA). It seems that OPA is rapidly becoming the default policy system across a wide array of system. This is encouraging as it is too often dominated by vendor-specific solutions that are hard to port to new solution and greatly limit the range of solution you can embrace.
• Bash Bible : a collection of pure bash alternatives to external processes. Really handy!
• Git Blame-Someone-Else : Introduced a bug in production code, want to hide it? Just blame someone else! This tool allows you to changes not only who authored the commit but the listed committer as well.

[Links of the Day] 05/03/2019 : RDMA for containers, K8s for hobbyist, OpenSource mobile core network

• FreeFlow : virtual RDMA networking purely with a software-based approach using commodity RDMA NICs for containers. 
• K8s clusters for hobbyist : how to setup and operate a fully functional, secure Kubernetes cluster on a cloud provider such as Hetzner Cloud, DigitalOcean or Scaleway.
• Magma : open-source software platform that gives network operators an open, flexible and extendable mobile core network solution.

[Links of the Day] : 29/01/2019 : Agile BS, K8s and Serverless failure stories

• Detecting Agile BS : short sweet and straight to the point. Most companies out there are doing Agile BS. Very few really understand what it takes and even when they do they often assume that carpet bombing the approach across every practice will work... Waterfall model has its place, it all depends on your context and objectives.
• Kubernetes Failure stories : K8s fail and stuff go bad
• Serverless Failure stories : Well serverless fail too and stuff go bad ( and can cost you a lot of money )

[Links of the Day] 04/12/2018 : Matrix cookbook, Serverless Containers, C++ network coroutine lib

• The Matrix Cookbook : nothing about Keanu Reeves in a kitchen apron but a collection of facts about matrices and matters relating to them. A very well documented desktop reference.
• Firecracker : Secure and fast microVMs for serverless computing... This is really another AWS product that makes you ask why do we need Kubernete. To be honest I really see K8s now as another OpenStack. It's at the top of the hype cycle but I don't really see it going anywhere further anytime soon. This type of tech enables serverless containers. Making any of the k8s almost a moot point.
• Coro async : C++ coroutine-based networking library

[Links of the Day] 23/10/2018 : bash history tool, Kubernetes security issue hunter, Kafka recovery toolkit

• hstr : Bash and Zsh shell history suggest box it allows you to easily view, navigate, search and manage your command history.
• Kube Hunter : an open-source tool that seeks out security issues in Kubernetes clusters. The objective is to increase awareness and visibility of the security controls in Kubernetes environments.[github]
• Kafka-Kit : set of tools for Kafka data mapping and recovery. Quite useful when you get into a pickle and you need to fix your Kafka topics.

[Links of the Day] 11/10/2018 : Go powerline, Kubernetes context switcher, Notebook scaling at Netflix

• Powerline-go : a nice low latency PowerShell written in go. Give it a try.
• Kubernetes Context Switcher : another practical tool, allowing you to seamlessly switch between kubernetes context.
• Notebook @ netflix : Notebooks is now the default tool for data scientists. And Netflix shows how they are able to scale this tools to accommodate their ever-increasing data crunching needs. 

[Links of the Day] 13/09/2018 : Kubernetes in Docker, Forensic Diffing AWS image, Consistent File system on top of S3

• kind : Kubernetes-in-Docker - A single node cluster to run your CI tests against that's ready in 30 seconds
• diffy : Diffy allows a forensic investigator to quickly scope a compromise across cloud instances during an incident, and triage those instances for followup actions.
• Snitch: Box created a Virtually Consistent FileSystem build on top of S3. An interesting solution that allows Box to prevent data loss by building a consistent solution using an eventually consistent storage. Sadly not open sourced... 

[Links of the Day] 05/06/2018: All about kubernetes - kops and descheduler

Today is all about k8s

• Kops : Production Grade K8s Installation, Upgrades, and Management
• Kops terraform : HA, Private DNS, Private Topology Kops Cluster all via terraform on AWS VPC
• Descheduler :  this aim at solving the issue of overprovisioning nodes with k8s. This descheduler checks for pods and evicts them based on defined policies. Ideally, these policies aim at maximising resource usage without compromising availability.

[Links of the Day] 29/05/2018 : Tracers performance, Testing Terraform , Virtual-kubelet

• Benchmarking kernel and userspace tracers : a good recap of what tracing toolkit is out there and the performance tradeoff that comes with them
• terratest : this is the thing I was looking for, a way to test and validate my terraform script. This will really help the adoption of Terraform I think as it will significantly increase the confidence in Terraform code before deployment.
• Virtual-Kubelet : that s a really cool concept, and introduce a great dose of flexibility in your Kubernetes cluster deployment. There is already some really exciting solution leveraging it such as the AWS fargate integration. With this, you could implement easily bursting and batching solution or real hybrid k8s solution with virtual kubelet hosted in Azure, Aws and on your private cloud.  

[Links of the Day] 01/05/2018 : CD over K8s , SQL query parser, NLP annotation tool

• Apollo : The logz.io continuous deployment solution over kubernetes
• Query parser : Open Source Tool for Parsing and Analyzing SQL. THis is really interesting tools for a specific use case. How do you map and understand the usage of databases and tables used in an organization that maintains hundred or thousands of systems without central coordination and architecture. 
• Prodigy : An annotation tool powered by active learning. Made by the creator of Spacy. This really cool tool help you streamline the creation of models and quickly test hypothesis. Not free :( but to be honest, if you use spacy ( like I do ) and need to annotate model. The cost is probably worth it. 

[Links of the Day] 07/09/2017 : Platform for Partial Differential Equations solver, High perf serverless event & data processing, K8s serverless framework

• FEniCS Project : a platform for solving partial differential equations. It's really cool as it allows you to rapidly solve and test pde model using a minimum amount of code. 
• nuclio : High-Performance Serverless event and data processing framework
• kubeless : Kubernetes Native Serverless Framework, I have a feeling that Mesos lost the container orchestrator war ...

[Links of the Day] 25/07/2017 : Universal Scalability Law Model , k8s + cloud native apps, python distributed execution engine

• usl4j : Implementation of Universal Scalability Law model, this is really cool because it allows you to model ( and predict ) when your system performance will start to degrade as its scale. It allows you to take real measurements from a live system and continuously build models. [github]
• daft : a tool for developers to create cloud-native applications on Kubernetes.
• ray : distributed execution engine written in python. Useful if you want to execute and schedule task across a cluster of nodes.

[Links of the Day] 27/06/2017 : Blockchain trust & authentication for IoT, K8s patterns, Ripple cryptocurrency Network analysis

• Kubernetes Production Patterns and anti-patterns :  a lot of common sense, actually a lot of the patterns and anti-patterns can be applied to the other environments. But still a good refresher. 
• Blockchain based trust & authentication for decentralized sensor networks : using blockchain to solve the trust issue in a swarm of IoT on a network. The critical bit missing is the power requirement for running all the crypto operations.
• Large-Scale Analysis of the Ripple Cryptocurrency Network : an overview of the paper analysing the Ripple p2p blockchain based money transaction network. Turns out it suffer the same issue that "old school" p2p network. Take out the highly connected nodes and you can bring down / split the network. Nothing new, but still a good read and reminder that small network tends to be resilient to attack. But, if their resiliency diminishes with the increased reliance on a small number of highly connected nodes.

[Links of the day] 10/10/2016 : k8s Stateful container App platform, Modern Bank Backend, Fast websocket & tcp server

• Deepstream.io : fast, secure and scalable websocket & tcp server for mobile, web & iot
• SuperGiant : Application platform specializing in stateful container orchestration, based on Kubernetes
• Building a Modern Bank Backend : Nothing much on the real detail, but it can be summarized: We took "classic" banking features and used a new stack to deliver it. The HN discussion is also worth checkking out. What is interesting is the discussion on Industrial vs open source software when it come to audit and security as well as reactivity. However what the authors of the discussion is that it often boil down to the blame game and emergency patch by industrial are NOT always thoroughly tested / audited / secured but you benefit from the insurance / SLA in case something goes wrong. [HN discussion]