Thursday, November 08, 2018

[Links of the Day] 08/11/2018 : large scale study of datacenter network reliability, What to measure in production, Failure Mode effect Analysis

  • A Large Scale Study of Data Center Network Reliability : the authors study reliability within and between Facebook datacenters. One of the key findings is the growth in complexity, heterogeneity and interconnectedness of datacenter increase the rate of occurrence of unwanted behaviours. Moreover, this seems to be also a key potential limiting factor for world scale spanning infrastructure undergoing rapid organic growth.
  • Understanding Production: What can you measure? : what do you need to monitor and measure in production. Very good summary of many blog post out there.
  • Failure Mode Effects Analysis (FMEA) : once you start reaching a certain production scale and more stringent requirement kicks in ( unless you were unlucky enough to have them at the get-go). You might want to run a failure modes and effects analysis (FMEA) is a step-by-step approach for identifying all possible failures in a design, a manufacturing or assembly process, or a product or service. While it was mainly designed to address shortcomings in the manufacturing industry, it is still extremely useful for IT system analysis, especially when you want to prepare yourself pre-rollout of a chaos monkey like system.


Tuesday, November 06, 2018

[Links of the Day] 06/11/2018 : Intro to probabilistic programming, Unit tests for data, Ali Wong stand-up routing analysis


  • An Introduction to Probabilistic Programming: a first-year graduate-level introduction to probabilistic programming. It not only provides a thorough background for anyone wishing to use a probabilistic programming system but also introduces the techniques needed to design and build these systems.
  • deequ : library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
  • Ali Wong structure of stand up comedy : fantastic and beautiful designed article analysing Ali wong stand up routine and how it is closer to a tv/movie script than a slapstick one-line joke comedy.


Thursday, November 01, 2018

[Links of the Day] 01/11/2018 : multi-platform, multi-architecture CPU emulator framework , Periodic Table of Data Structures, Model-Based Machine learning book

  • Unicorn : a qemu offshoot offering a lightweight multi-platform, multi-architecture CPU emulator framework. 
  • Periodic Table of Data Structures : the author try to express the different data structure used in computer science using a universal model. Once completed, formalized and validated this could be invaluable help for accelerating decision making processing for software architecture design. Now if somebody could do the same for software project management...
  • Model-Based Machine learning book : this book is trying to teach you how to think like a statistician to solve some nontrivial analytical problems. Not many new books cover this skillset, and just for this alone, this book is a must-read for anybody using machine learning.


Tuesday, October 30, 2018

[Links of the Day] 30/10/2018 : Python Object CLI generator, Alibaba Distributed File System, Microsoft API Design guideline

  • Python-fire : library for automatically generating command line interfaces (CLIs) from absolutely any Python object
  • PolarFS : Alibaba n Ultra-low Latency and Failure Resilient Distributed File System for Shared Storage Cloud Database. Keep an eye on this one, as the authors are planning to deliver a TLA+ proof soon. Moreover, I hope that they also run a benchmark against GPFS or Luster rather than Ceph. Ceph is not really competing in the same league.
  • API design : pretty much the gold standard in API design. A must-read for anybody designing or using API.

Thursday, October 25, 2018

[Links of the Day] 25/10/2018 : Distributed AI framework, Reverse proxy API gateway, AirBnB Change data capture service

  • Ray : A Distributed Framework for Emerging AI Applications [Github]
  • annon.api : Configurable API gateway that acts as a reverse proxy with a plugin system.
  • SpinnalTap : Change Data Capture (CDC) service capable of detecting data mutations with low-latency across different data sources, and propagating them as standardized events to downstream consumers.

Tuesday, October 23, 2018

[Links of the Day] 23/10/2018 : bash history tool, Kubernetes security issue hunter, Kafka recovery toolkit

  • hstr : Bash and Zsh shell history suggest box it allows you to easily view, navigate, search and manage your command history.
  • Kube Hunter : an open-source tool that seeks out security issues in Kubernetes clusters. The objective is to increase awareness and visibility of the security controls in Kubernetes environments.[github]
  • Kafka-Kit : set of tools for Kafka data mapping and recovery. Quite useful when you get into a pickle and you need to fix your Kafka topics.

Thursday, October 18, 2018

[Links of the Day] 18/10/2018 : Rust conf 2018 videos, Financial modeling for startups, Baidu RPC framework


  • bprc : Baidu enterprise-grade RPC. If you look at the performance, this looks really impressive.  But, sadly most of the docs are in Chinese. I don't really know how committed to opensource and if Baidu is culturally attuned to run opensource community. But I think we need to keep a close look at all the recent opensource announcement from Chinese companies and see how they withstand the test of time. Last but not least, you might note that Chinese tends to prefer QQ over slack for discussion medium. 
  • Financial Modeling for Startups : This is an interesting laying out the core element of financial startup modelling. This can be of great help for entrepreneurs out there that want to wrap their head around the difficult financials aspect of creating and running a company. 
  • RustConf 2018 : video from Rust conf 2018