Links of the day 26 - 03 - 2015

Today's links 26/03/2015: #HPC Network and #Kafka REST proxy

• Receiving Packets at the Speed of Light : Data Plane Acceleration for modern network when the amount of time to process packets decreases to as little as 6.7ns per packet at 100Gb/s. A memory barrier can cause stalls of 7ns or more strategies must be devised to mitigate in order to avoid wasting bandwidth. 
• Multi-Host Technology in OCP-Server Track : Mellanox ConnectX-4 (100GbE)+Multiple Heterogeneous Hosts (x86, POWER, ARM, GPU) [slides]
• slimFly : high-performance cost-effective network topology that approaches the theoretically optimal network diameter.
• Kafka REST proxy : making it easy to connect to a Kafka cluster 

Links of the day 25 - 03 - 2015

Today's links 25/03/2015: #RDMA, #NVM, Google Build tools, OFS dev workshop

• Bazel : Google build tool that builds code quickly and reliably, a lot of nice feature in there, automated testing, rule based integration , caching ,parallelism, language and platform diversity.
• A Comparison Of Log-Structured Merge (LSM) And Fractal Tree Indexing : Trees everywhere!!
• Prototyping Byte-Addressable NVM Access :   fine granularity doesn't always make things easier to use
• RDMA and NVM Programming Model : I predict strange behavior with RDMA unless all vendor agree onto a single common API for atomic persitency.
• 2015 International OFS Developer’s Workshop Video Gallery

Links of the day 23 - 03 - 2015

Today's links 23/03/2015:  Full stack Lttng profiling, #RDMA + #HADOOP , VRAM data leakage, difference between DL vs ML vs PR in AI (acronym overload)

• Full stack system call latency profiling : demo of Lttng based profiling similar to the linux perf tools. It allows the use the static kernel tracepoints to dynamically decide what and when to record relevant information for our use-case and trying to find the needle in the haystack. 
• HiBD : RDMA-based Apache Hadoop 2.x solution. Feels like Big Data is just going through the same evolution curve as HPC
• Is your VirtualBox reading your E-Mail? : Collecting data from VRAM. I wonder when we will see the first proof of concept attack of cross VM information leakage over (GP)GPU 
• Deep Learning vs Machine Learning vs Pattern Recognition : if you ever wonder what the hype is really about.

Links of the day 19 - 03 - 2015

Today's links 19/03/2015:#python framework, #linux perf tools, #bitcoin API, Memory-centric distributed  #storage

• SuPPort : in-development distillation of PayPal Python Infrastructure. SuPPort is an event-driven server framework built on top of several open-source technologies designed for building scalable and maintainable services and clients.
• Linux Performance Analysis: New Tools and Old Secrets : Brendan Gregg's excellent talk on new Linux performance tools: perf-tools collection. These use existing kernel frameworks, ftrace and perf_events, which are built in to most Linux kernel distributions by default, including the Linux cloud instances he analyze at Netflix.
• Coinkite : Bitcoin API provides simple and powerful REST integration for adding bitcoin functions into application.
• Tachyon: A memory-centric storage system enabling reliable file sharing at memory-speed across cluster frameworks, such as Spark and MapReduce. [SOCC 13] [Github]

Links of the day 18 - 03 - 2015

Today's links 18/03/2015: NVDIMM , Dynamic CPU Cache allocation, NAS + Bigdata Analytic, DPDK packet generator

• Atomic Sector Updates inSoftware for Persistent Memory : Intel proposal to adress atomic sector update as byte addressable persistent storage is great but limited when you want to do large write.
• Priority-Based Cache Allocation in Throughput Processors : interesting approach that propose to dynamically vary the amount of cache allocated to processors based on the pollution rate. Allowing cache friendly process to benefit from greater cache real estate [slides]
• Qumulo : start up proposing a file system with Analytic capability directly embedded within the NAS [spec]
• MoonGen : fully scriptable high-speed packet generator build on DPDK and LuaJIT. It can saturate a 10 GbE connection with 64 byte packets on a single CPU core while executing user-provided Lua scripts for each packet. It also features precise and accurate timestamping and rate control.

• 

Links of the day 17 - 03 - 2015

Today's links 17/03/2015: hash table datastructure and hashing Algorithm, #ARM virtualization, Hyper-converged Solution

• A Fast, Minimal Memory, Consistent Hash Algorithm : interesting and minimalist (in a good way) hashing algorithm allowing simple variable partitioning of the address space.
• Fast Bounded-Concurrency Hash Tables : very nice lock free data structure with bounded guarantee. This is an impressive piece of work.
• ARM virtualization extensions : in depth look into the virtualization features and extension of the ARM platform.
• Datacore & Huawei : Huawei FusionServer servers running DataCore SANsymphony-V software with Huawei Oceanstor direct-attached storage made into a virtual SAN across 2 to 64 servers.

from Adexchanger comic strip

Link of the day 16 - 03 - 2015

Today's links 16/03/2015: #devops, time service, simultaneity in distributed system, value of data

• DIY Punk Rock DevOps Playbook : punk rock view of devops from Joyent CTO Bryan Cantrill 
• Introduction to Time Service : after reading this one would really wonder why we still stick to NTP and not use GPS based clock system that provide multiple order of magnitude higher precision within data centers.
• There is No Now : a look into problems with simultaneity in distributed systems
• The Value of Data : 3-part series about building companies that leverage the value of data , part [1] [2] [3]

Links of the day 13 - 03 - 2015

Today's links 13/03/2015: machine learning , Time-Series and Geospatial database, Write acceleration for virtual environment

• Clustered Fault Tolerant Write Acceleration in a Virtualized Environment : PernixData present a fault tolerant host-side flash write acceleration layer that seamlessly integrates with the virtualized environment while preserving dynamic resource management and high availability
• Machine Learning for Programming : Peter Norvig keynotes on using machine learning techniques to solve more general software problems, helping both the advanced programmer and the novice one.
• Time-Series Database Requirements : the author present its own ideas about what constitutes a good time-series database
• Why Are Geospatial Databases So Hard To Build? : the root of the problem is that the internals of a database engine designed for traditional “text and numbers” data models, which is virtually all of them, has very little in common with a database engine designed for real-time spatial data models.

Links of the day 12 - 03 - 2015

Today's links 12/03/2015: cryptocurrency landscape, vsphere memory state, hyperscale datacenter system, rowhammer bit flip

• Bitcoin Landscape : database and categorization of crypto based company [ database here ]
• Vsphere memory state : What technique vmware use to increase memory density depending of the system memory state . Bonus : memory management performance in vsphere 5.0
• HDS 8000 : Ericsson HDS 8000 (Hyperscale Datacenter System), built on Intel® Rack Scale Architecture. It uses optical interconnect. Combining a disaggregated hardware architecture with optical interconnect removes the traditional distance and capacity limitations of electrical connections.
• Project Zero - row hammer : how flipping bit in memory can allow you to gain root access to a physical machine. Basically by aggressively flipping 2 bits you can trigger the third one to switch. ECC can help prevent that to a certain extent but is not completely immune.

Links of the day 11 - 03 - 2015

Today's links 11/03/2015: category theory, linux profiling, CRDT, Log structured merge trees

• Category Theory : notes and list of article, books on category theory.
• Linux Profiling at Netflix : perf and flame-graph, should be part of every body standard toolbox. 
• Consistency without Consensus: CRDTs in Production at SoundCloud
• Log Structured Merge Trees : Instead of having one big index structure (which will either scatter-gun the file system or add significant write amplification) batches of writes are saved, sequentially, to a set of smaller index files.

Links of the day 10 - 03 - 2015

Today's links 10/03/2015: architecture modeling, why engineers build crappy products, lock free DS, growth hacking

• Structurizr : software architecture models and diagrams as code based upon the C4 software architecture model.
• Why Engineers build crappy products : short version - we like to be able to be able to do everything even if it is unusable.
• Lock-Free Data Structures : some introduction and overview of lock free algorithm and datastructures
• A Developer’s Guide to Growth Hacking : explains what Growth Hacking is and describes the processes key for it to be effective – from setting goals, to working through an experimentation cycle and how it works in practice.

Links of the day 09 - 03 - 2015

After a brief hiatus ( still enjoying the snow in Chamonix until the 15th ) here is Todays' links 09/03/2015: real time stream engines, high performance networking framework, RLE lib

• GearPump: Real-time Streaming Engine Using Akka [github] 
• Seastar : Really interesting framework for high-performance server applications on modern hardware. It leverage DPDK and single threaded model to enable fast zero copy communication, however like any higly specific framework model there is a certain learning curve. Applications using Seastar can run on Linux or OSv. [github] 
• TurboRLE : an Efficient and fastest Run Length Encoding library, nice real time stream engine using SQL semantics.
• Pulsar : Ebay start moving more aggressively into the opensource ecosystem game [ source ]