olric : Distributed cache and in-memory key/value data store. This can be embedded as a go library.
Pilsung Kang : a lot of really cool git repository for machine learning and datascience lecture , notes, code etc.. by Pilsing Kang of the School of Industrial Management Engineering Korea University.
Developer Roadmaps : Step by step guides and paths to learn different tools or technologies, checkout the devops one .. you probably need two lifetime to cover everything.
iter : Go implementation of C++ STL iterators and algorithms. I really need to look into C++ again. Been a really long time since I touched it.
HotChips 2019 : All videos from talks at this year HotChips conference ( 31st of its kind!)
Granne : Graph-based approximate nearest neighbour search in Rust, I really like Annoy from Spotify but this Rust implementation seems attractive enough that I will give it a spin.
NiftyNet : TensorFlow based opensource CCNs platform aiming at accelerating research in medical image analysis.
Bigmachine : like coroutine? like golang? like serverless? well seek no further, bigmachine is for you. It's a really cool library for self-managing serverless computing. You basically write your service in a single monolithic go and bigmachine will distribute it for you. It feels like openMP but for serverless. [github]
Why do people neglect maintenance: well sadly it seems that often maintenance cost get compounded into technical debt. As the authors describe in this article, there is a lot of reason why maintenance often take a back seat when it comes to prioritisation of the next dev cycle. I also feel that the authors missed an important aspect: employee churn. New employees coming to a company usually don’t want to “maintain and fix” the code of others. Moreover, HR and any interviewer rarely advertise a position for: we need to make sure our services keep ticking so we need somebody to maintain our codebase.
Continuous Compliance : an interesting post about continuous compliance and how to integrate it as a standard practice in your CICD. However, more often than not, the main issue on the human side and not the technical side. Sadly, companies tend to avoid the introduction of continuous compliance. They this technology as an increased risk because the constant validation can pickup problem that are not detectable with episodic checks. Which ultimately translates into $$ cost, as if a compliance issue is detected, it has to be fixed. That's why your best chance to introduce continuous compliance practice within your SDLC would be in the initial requirements collection phase of a Greenfield project.
Lazy-Docker : great UI terminal for managing docker and docker-compose. Bonus : written in GO.
Startup Checklist : a succinct checklist that can quickly help you develop ideas and filter them. However, like the author mention, be careful to not over filter ideas as the good one can be too easily dismissed.
NewDNS : Want to build a DNS in GO, this is for you.
OctoSQL : query tool that allows you to join, analyse and transform data from multiple databases and file formats using SQL.
kdevops : devops framework for Linux kernel development that relies on ansible, vagrant and terraform, ansible roles through the Ansible Galaxy, and terraform modules. It aims at making setting up and testing the Linux kernel for any project as easy as possible. I wish I had that a couple of years ago.
Solana : Proof of History based blockchain system. [website]
Vulcanizer : Go library for interacting with an Elasticsearch cluster. It is not meant to be a full-fledged Elasticsearch client. Its goal is to provide a high-level API to help with common tasks that are associated with operating an Elasticsearch cluster such as querying health status of the cluster, migrating data off of nodes, updating cluster settings, and more. [github]
Grobid : machine learning software for extracting information from scholarly documents.
Trill: high-performance one-pass in-memory streaming analytics engine. This seems like a highly versatile and performant streaming engine. The team behind it is making some bold claims regarding its capability ( see table below). I think that this architecture is promising, however, the .net language might put some people off.
Powerline-go : a nice low latency PowerShell written in go. Give it a try.
Kubernetes Context Switcher : another practical tool, allowing you to seamlessly switch between kubernetes context.
Notebook @ netflix : Notebooks is now the default tool for data scientists. And Netflix shows how they are able to scale this tools to accommodate their ever-increasing data crunching needs.
A Survey of Techniques for Dynamic Branch Prediction : with all the spectre and meltdown attack, this paper is a good refresher on what is dynamic branch prediction, how it works and why we need these techniques.
Cgo and Python : When you want to run python in your go, and you realise that the python threading model is still a pain in the $"!$*(^!& .
Wallaroo : this is a really impressive alternative to spark/storm. The creators really focused on ease of use and deployment. Which can be a huge barrier to entry with the spark, storm, Hadoop stacks. [github]
Unified Clock : Must read blog post, on how Riot games use a unified clock mechanism to synchronise player in the league of legends multiplayer games.
Riot : golang text search engine with some really good perf - 1M blog 500M data 28 seconds index finished, 1.65 ms search response time, 19K search QPS
Evolution and challenges of Tofu Interconnect : Deep learning and heterogeneous hardware is putting a strain on HPC interconnect. They need to adapt to new application communication model as well as hardware while retaining best of breed capability for "traditional" HPC application.
Satellite-to-ground quantum key distribution : this is ground breaking from Chinese scientists. Where they demonstrate they are able to solve the main hurdle behind planet-wide quantum communication by leveraging satellite to satellite quantum crypto key distribution. And they are deploying a proof of concept!
Badger : nice key/value store written in go. It's based on Wiskey paper and is heavily optimised for SSD. It's 3.5 faster than rocksDB .[github]
Earth Computing Network Fabric : event based protocol for datacenter that target specifically datacenter as it eliminate the need for heartbeats and timeouts. The protocol relies on recoverable atomic token to deliver deterministic in order communication. To some extend they are proposing to move back to latices system where each server are a node within the network and act as router for messages. This eliminate switch requirement and looks really neat. However adoption might be difficult due to the ubiquitous Ethernet hardware and also the need to change the underlying communication protocol. Last but not least I do not really know how to efficiently secure and trust messages on such network. [slides]
Computer Science video courses : Extensive collection of links to CS courses ranging from introductory to expert in pretty much the full scope of CS subject (DB, distributed systems, etc..)
Appdash : Application tracing system for Go, based on Google's Dapper.
ZeroCash : Trustless Bitcoin Tumbling, the authors proposed a pooled approach to anonymise transaction in Bitcoin. However the authors go a step further than just pooling. They popose a system where participant can anonymously check in and out resources from a global pool. Effectively creating an anonymous cooperative resource sharing infrastructure. [github] [Paper]
Zap : Fast, structured, leveled logging in Go. When you start to reach Uber or other hyperscale microservice architecture. Every aspect counts, and logs are everywhere. This library provide a high performance structure log for go.
Scalable software controller : This controller basically allow to allocate on the fly hardware ressource, compute, memory storage, network based ont the demand of the deployment tool (openstack, k8, mesos, etc..) .
Intel Drop PCOMMIT : Intel decided to simplify its persistent memory specific instruction set. The logic behind this change is that Asynchronous DRAM Refresh (ADR) is now a requirement for persistent memory support. As a result there is no need for PCOMMIT anymore because of the guarantee of the Write Pending Queues flush on power loss.
Sigcomm 2016 conference : schedule, paper and slides. I already mentioned Microsoft paper yesterday on RDMA at scale over Ethernet fabric. Here is a few more selected paper:
Real-time Distributed MIMO Systems : MIT achieved the feat to create a distributed MIMO system operating across devices with independent clocks. This allow system to clearly maximize bandwidth usage while maintaining fairness.
Globally Synchronized Time via Datacenter Networks : Authors proposed to use a hardware solution leveraging low level network fabric for maintaining synchronous clock across data-center. Basically they embed within the fabric extra information for maintaining coherent time. It allow a 200ns clock skew within a 6 hops data-center. Which is quite impressive even if it comes at the cost of requiring HW modification.
Numa-aware scheduler for go : as machines get bigger, Numa optimization start to creep up as a requirement. Especially with language using garbage collection and automatic scheduling like go. This proposal show a possible approach for Golang to handle Numa setup.
Kubernetes The Hard Way : The famed Keysley Higtower explain how to deploy and run a kubernetes system on google cloud the hard way ( no automatic installation cheat)
Wicked Fast Storage and Beyond : Intel IDF 2016 talk on the future of storage , looking at 3dxpoint, and the Optane SSD . What is really exceiting is the upcoming Intel DIMM using 3DXpoint tech. This provide native PMEM capability , 2x storage vs RAM (but slower yes). I can seriously see future in memory database ( SAP HANA ... ) bypassing storage together and just stacking NVM DIMM. Using slower storage form ( over NVMe by example) for backup / snapshot, etc..
BlockStack : a distributed storage system that uses blockchains on Bitcoin to securely define a global name space. The name’s are all tied to values which representing URI(URL)s to storage systems like AWS S3 but could be any cloud storage service.
Use Case track : Videos from the Use Case track at @DockerCon 2016
Atmanos : this is really cool, this project enable you to compile go code and run it as a unikernel on top of Xen
