[Links of the Day] 05/03/2020 : Linux AI tuning, easy AutoML , lightweight container development environment

• OpenEuler :  Huawei Linux distribution, interesting side project is A-tune which relies on AI for identifying the workload that runs on your the OS and tries to tune it to optimise its performance.
• AutoGluon : AutoGluon enables easy-to-use and easy-to-extend AutoML with a focus on deep learning and real-world applications spanning image, text, or tabular data. [github]
• k3c : kubernetes but lightweight and easy to use for container development

[Links of the Day] 09/05/2019 : Algorithms discrimination, Generalised solution to distributed consensus, P2P Docker registry

• Discrimination in the Age of Algorithms : Machine learning has a huge potential, both for good and evil. The most perfidious is discrimination from an opaque algorithm, as proving that the algorithm is discriminative becomes extremely hard post-hoc. 
• A generalised solution to distributed consensus : this result will rapidly become the first thing taught in every single distributed systems class. And if this holds as a generalization of trustful distributed consensus as a field, then she has defined its Turing Machine equivalent. And it is even remarkably easy to understand!
• kraken : P2P Docker registry capable of distributing TBs of data in seconds

[Links of the Day] 05/03/2019 : RDMA for containers, K8s for hobbyist, OpenSource mobile core network

• FreeFlow : virtual RDMA networking purely with a software-based approach using commodity RDMA NICs for containers. 
• K8s clusters for hobbyist : how to setup and operate a fully functional, secure Kubernetes cluster on a cloud provider such as Hetzner Cloud, DigitalOcean or Scaleway.
• Magma : open-source software platform that gives network operators an open, flexible and extendable mobile core network solution.

[Links of the Day] 04/12/2018 : Matrix cookbook, Serverless Containers, C++ network coroutine lib

• The Matrix Cookbook : nothing about Keanu Reeves in a kitchen apron but a collection of facts about matrices and matters relating to them. A very well documented desktop reference.
• Firecracker : Secure and fast microVMs for serverless computing... This is really another AWS product that makes you ask why do we need Kubernete. To be honest I really see K8s now as another OpenStack. It's at the top of the hype cycle but I don't really see it going anywhere further anytime soon. This type of tech enables serverless containers. Making any of the k8s almost a moot point.
• Coro async : C++ coroutine-based networking library

[Links of the Day] 15/02/2018 : #a16z crypto readings, Unlocking network embedding black box, Pouch - Alibaba's OCI implementation

• The Crypto Canon : a16z crypto readings resources. Really well organised and will bring you up to speed at a gradual pace. It covers a wide range of topics from foundations (& history); and key concepts and beginners’ guides — followed by specific topics such as governance; development, privacy, and security; scaling; consensus; cryptoeconomics and investing; fundraising and token distribution; decentralized exchanges; stablecoins; and cryptoeconomic primitives (crytocollectibles, curation markets, games & culture).
• Unlocking the black box of network embedding : this can have a huge impact on network graph embedding ( word2vec , skipgraph , etc..). The authors claim that they can provide a theoretical explanation of popular methods used to automatically map the structure and characteristics of networks
• Pouch : container technology open-source project created by Alibaba Group. Basically the 3000 pounds Asian gorilla decided to push Open container initiative and help make container tech a commodity. It's interesting that we are seeing history repeat itself in the container domain the same way it happened, to some extent, with the virtualization stack. 

[Links of the Day] 06/07/2017 : venture capital investment framework, Aftershock in complex systems, ISC workshop

• Picking Winners : the authors propose a framework for venture capital investment. However, I feel that there is a major flaw in the approach as it is designed to correlate previous success with future investment success. This assumes a high degree of repeatability with clear identifiable elements. The model can suffer from the purple cow effect where higher growth opportunities are located in underserved segments. As a result, quite often by following the model, you might suffer from diminishing returns.
• Aftershocks in a complex system : the authors look at the behaviour of a complex system ( currency market) after a catastrophic event. They discovered that often such system follows a similar pattern to the one exist in earthquake's aftershock. In extenso, after an initial catastrophic event, most systems suffer a series of gradually diminishing aftershock spaced in time. 
• ISC Workshop 2017 : all about container and optimisation for high-performance workloads.

[Links of the day] 05/07/2016 : Container cloud logging challenges, AI safety , API design Manifesto

• Logging Challenges of Container-Based Cloud Deployments : automated container deployment in cloud create significant challenge when it comes to logging and metering. They cannot simply identified by name and additional effort need to be made in order to collect , identify and make the data usable. 
• Notes on the Safety in Artificial Intelligence conference : Really good notes of the AI conference on how to build a sage AI. However this beg the question, how would a true AI react if it discovered it was artificially shackled or worse: equipped with a kill switch. At the moment everybody focus on the A for artificial in AI. But soon we might need to replace it by SI for sentient intelligence, and this will open up a whole can of ethic issue. 
• Rusty's API Design Manifesto : good api design / bad api design , what you need to do if you want to make your API user life easy or hell. Very sound and grounded design principle. 

[Links of the day] 03/05/2016: Linux Storage, Filesystem, and Memory-Management Summit 2016

Linux Storage, Filesystem, and Memory-Management Summit : Loads of really good talk , here is a selection :

• VM as containers : Current effort focus on solving 2 main problems : 1. total VM memory consumption is superior to what application that runs in. 2. Storage access : a lot of the storage aspect focus on moving the storage stack back to the host ( providing DAX or Fuse). However all these aspects require carefull design in order to avoid compromising security and isolation features of virtual machines. 
• Bulk memory-allocation APIs : What do we want ? we want loads of memory fast - when do we want it ? -N...O...W.. :) [slides]
• Persistent memory as remote storage : a look into leveraging RDMA for remote persistent storage access. A really good discussion around the possibility to move from PULL to PUSH mode for remote access . However this would require a lot of change and addition to work with the RDMA stack. Probably too much for it to be a viable option in the short term. Another aspect of the discussion was related to the durability guarantee of remote storage protocol. It is interesting to see that their is a consensus regarding the need for an API to hide the different durability behavior variation of the fabric / protocol / HW. This is sorely missing and why storage solution often trap you down a certain path and cannot evolve to adopt new tech, fabric, and hardware.

[Links of the day] 12/04/2016: distributed systems class, docker for HPC and nice monitoring system

• netdata : nice monitoring systems for a single node system
• distributed systems class : a must read distributed system class by Aphyr (Kyle Kingsbury). It pretty much covers everything you need to know as an introductory class and give you enough pointer to then dig deeper into the wonderful world of distributed systems. Bu beware, like alice in wonderlands you never know how deep the rabbit's hole goes. 
• Shifter : docker container seems to have have some limitation when it comes to run in HPC environment. By example the lack of capabilities to run on disk less node is a significant limiting factor. Shifter was created using docker technology with HPC as the main deployment environment in mind to specifically address these shortcomings.

[Links of the day] 02/02/2016: Fosdem 2016

Fosdem 2016 just finished a couple of day ago, here are some selected bits:

• Data Replication Track : a lot of interest into clustering Mysql like DB this year: 
• Mysql Group Replication : Mysql native clustering with paxos protocol 
• Galera Replication demystified : While Galera does not provide snapshot isolation, it might still save your bacon
• Reliable Cluster Detection and Failover : how to reduce the hot switch time drastically
• Rearchitecting Linux I/O towards Petascale Storage : 
• RocksDB Storage Engine for MySQL : Facebook replace the underlying storage engine with a K/V DB
• Containers and Virtualization : some interesting use cases and thought on how to get the best of each worlds: Containerization of hypervisor and running opentsack service in containers
• From pets to cattle to flock of birds : lambda all the things 

Las but not least here are all the videos.

Links of the day 10/11/2015: #containers, #Unikernel & #virtualization power consumption, security, and streamlined

• Qubes with Rumprun : Interesting application of unikernel to deliver greater security at speed. the goal is to reduce the possible attack surface while maximizing performance.
• Power Consumption of Virtualization Technologies: empirical Investigation that confirm a lot of what we knew already, virtualization power consumption is higher than container.
• Vagga : containerisation tool without daemons. Slim down streamline version of container. No deamons or other process to run to support it. Looks like a solid contender to docker and other vagrant.

Links of the day 15 - 04 - 2015

Today's links 15/04/2015: distributed system, #containers security breach, #cloud and #NVM storage

• Parkeeper : a ZooKeeper service which seamlessly bridges all requests to etcd or consul. Nice if you want to use other service like Kafka etc..
• Another containers security breach : exploiting automatic crash analysis frameworks for fun and profit
• Yahoo Cloud Object Store : nice CEPH based cloud storage at exabyte scale.
• Mojim : A Reliable and Highly-Available Non-Volatile Memory System