Friday, August 28, 2015

Links of the day 28/08/2015 : libfabric, IO visor and demoscene

  • IO visor : it seems that the effort from Plumgrid around eBPF are picking up speed and an official linux foundation project is been setup. However one must wonder how such solution will compete against the pure user space solution relying on DPDK and consort? You can find a more in depth slide deck of the concept [here].
  • Libfabric : Intel is announcing in grand pomp the "Open source" library supporting its Omni path fabric. However it is not other that the fantastic Lib fabric. This library offer a set of next-generation, community-driven, ultra-low latency networking APIs. The APIs are not tied to any particular networking hardware model, it support Infiniband / Iwarp, usNic from Cisco, OPA from intel What is interesting is that it goes one step further than the RDMA library while maintaining a good balance between low level tuning and high level program-ability. While the learning curve might be a little bit more steep compared to Accelio from Mellanox it delivers ( I think ) greater advantage and flexibility.
  • Winning 1kb intro : released at Assembly 2015, prepare to be amazed


Thursday, August 27, 2015

Links of the day 27/08/2015 : PCIe fabric, One Time SSH key and China ARM processor

  • Dolphin PCIe fabric : seems that the PCIe fabric is getting a lot of news recently, nice to see that we are reaching the 300 nanoseconds latency. While greater bandwidth and lower latency is really great this bring a certain amount of challenge as DRAM access time is still around 10-30 nanoseconds. Implying that if we the CPU needs to do anything with this data it only has ~250 nanoseconds to do so before moving to the next one. I wonder when we wills start to see event based GPGPU or FPGA approach in CPU in order to handle such performance. Maybe its time to move toward stream oriented CPU architecture ? 
  • One-Time SSH Keys : I like this concept, it allow the use of single-use key ready for connecting to untrusted system. With single-use keys, even if the key is compromised, it will have already been used, and would be of little use.
  • China ARM : Homegrown ARM v8 based cpu chips. Seems that the Chinese are getting serious in leveraging homegrown solution in the CPU field and especially HPC. From this expertise they hope to spin off in the scale out computing (high density, low power). Maybe the ARM server breakthrough will come from the Asian market? 


Wednesday, August 26, 2015

Links of the day 26/08/2015 : 3Dx point & SAP HANA , PCIe fabric, Hashmap for NVM

  • PCIe switch : after buying PLX avago is trying to ramp up the PCIe fabric business. The interesting bit will come up when it will start to compete against Intel omni path fabric. While PCIe fabric provide the advantage of direct device access and MR-IOV is really attractive a less HW and more Network oriented fabric might have certain advantage. Maybe we will start to see true heterogeneous Clos topology fabric in the future. PS: bonus patent for a PCIe fabric using non transparent bridge.
  • 3Dx point and SAP HANA : while the post in itself is not that interesting ( enable bigger better in memory system). The responses from Hasso Plattner to it show a rather puzzling perspective, it seems that there is a disconnection between the "marketing" and actual real use case. HANA can "theoretically" be deployed on massive machine but in reality you only need a third of that. Basically very , I repeat, VERY few (to no) companies need that type of specs. Maybe, if SAP started to offer modular HANA solution over their clouds, customers will finally avoid the massive headache that is sizing (and adopt the product more easily). By modular I mean at providing Memory re-sizing at runtime with pay for what memory you consume model. 
  • NVC-Hashmap : A Persistent and Concurrent Hashmap For Non-Volatile Memories

Tuesday, August 25, 2015

Links of the day 25/08/2015 : #container conference ( #docker ), Hardware management interface standard , Computer System design

  • Redfish : Intel and other main player data center and systems management systems specification that aims to supply improved performance, functionality, scalability and security. Basically they are trying to sort out the IPMI and other ILO interface mess in order to offer a simple clean interface. Let's hope they achieve it. 
  • Hints for Computer System Design : nice presentation and discution on how how Lampson's hints can still be used, some thirty years later, to implement functionally accurate, high-performing fault-tolerant software systems.
  • Workshop on Containers 2015 (WoC) : some interesting bits in that conference, Obviously the performance comparison ( linked yesterday) but the lesson learned slides show a very interesting bit regarding containers. It seems that the cost of security group is quite heavy as you scale the number of VM / containers.

Monday, August 24, 2015

Links of the day 24/08/2015 : #Docker vs #kvm vs #Osv performance, #Linux NVM programming tutorial and Datacenter SLA ontology


Friday, August 21, 2015

Links of the day 21/08/2015: #kvmforum , Two laws, and NVMe transforming the Datacenter


  • KVM forum : slide deck of the 2015 KVM forum, some gems in there especially related to the real time efforts.
  • A tale of two laws : As Amdahl’s law and Moore’s law reach their 50th anniversaries, this paper review the roles they have played in shaping both perception and reality in high-performance computing. Along the way, it also attempt to clarify some misconceptions that have surrounded both of these highly influential but not always fully appreciated ‘‘laws.’’
  • NVM Express - The Data Center and Client Storage Transformation: Loads of good stuff from Intel IDF 2015, i ll be distilling it over the next week. But this one is particularly interesting talk by Intel and DSSD folks ( acquired by EMC recently). It show how the new tech is trying to cut the CPU out of the loop as much as possible and storage start to embrace Memory type transaction model ( RDMA ). To a certain extend storage is just being absorbed back into the overall memory model. The interesting bit is that because of the current performance limitation of flash couple with the fabric capabilities there is a very small overhead in using remote storage vs local one.

Thursday, August 20, 2015

Links of the day 20/08/2015 : #Facebook DC network, #cloud at #Berkley go BOOM and Eve

  • Inside the Social Network’s (Datacenter) Network : A very insight full and deep look into Facebook data center networking. What's interesting to see is that there is a non negligible inter data center communication going on there. 
  • BOOM : Berkeley Orders Of Magnitude project, Enabling the construction of data-rich systems at unprecedented scales using a minimum of software.
  • Eve : version 0 : a relational database, a new programming language, an IDE, and a UI editor, all built from scratch to fit our goals for a better programming foundation. Ambitious project, lets see how it develop.

Tuesday, August 18, 2015

Links of the day 18/08/2015 : #RDMA fabric, #ReRAM NVDIMM, #Micron & 64bit dev problem

  • Congestion Control for Large-Scale RDMA Deployments : Microsoft Research paper demonstrating how they look at flow control in mixed use environment with RoCE. What's interesting is that it show that Microsoft start to heavily leverage RDMA in its data-center in order to deliver fast guaranteed performance especially in the storage domain.
  • ReRAM Storage Class Memory NVDIMM : Viking technology already had a Flash NVDIMM offer, but it seems they are expanding with the faster ReRAM type of tech. Probably in direct competition with 3DxPoint from Intel. I need to dig into the spec to see what are the trade-off between the different tech in term of cost/GB/Perf.
  • 64 bit program development forgotten problems : me bug you long long time.
  • Micron analyst presentation : interesting to see that the NB of datacenter customer represent is roughly equal to the number of enterprise one. Obviously we are at the tipping point where datacenter will clearly outtake enterprise in IT HW consumption following the natural shift from product to utility. Does that means that traditional storage vendor will be relegated to niche market in the future? 






Monday, August 17, 2015

Links of the day 17/08/2015 : #NVMe & #RDMA , #Strategy , Cryptography in hostile environment


  • NVMe over RDMA fabric : interesting bit PMC Sierra and Mellanox unveiled NVMe ove RDMA fabric as well as peer direct technology for NVM storage. This open up a certain world of possibility where you could combine without CPU involvement GPU - NVM(e) - RDMA. Literally offloading all the storage operations.
  • Strategy Scenario and the use of mapping : excellent series of posts by Simon Wardley showing how leveraging his mapping technique allow CEO - CIO to navigate the tortuous strategic decision. The Analysis of the scenario can be found here 
  • The network is hostile : TL;DR: we don't encrypt enough and early enough




Friday, August 14, 2015

Links of the day 14/08/2015 - a lot of Kalman Filter and some Flabbergast


Thursday, August 13, 2015

Links of the day 13/08/2015 - Flash failure @ #facebook , tech behind #google voice and its datacenter network provisioning, distributed system #cloud challenges


Wednesday, August 12, 2015

Links of the day 12/08/2015 : #Google #Datacenter network, #Intel Omni-path Fabric and hardware x86 cpu security flaw

  • Memory sink hole : when x86 processors ancient (unused by now) feature bites back : it allows to move the memory window of the APIC registers to an arbitrary address. By moving this window, it is possible to circumvent built-in hardware security checks and access the highly privileged System Management Mode (Ring -2) from Ring 0 Code.
  • Google Datacenter Network : very good overview of the ONS keynote [video
  • Intel Omni-Path Architecture Webinar Series: very good overview of intel next gen fabric. Its really interesting how they place it directly in competition with infiniband. It seems that the PCI fabric fell completely off the radar and might stay stuck to niche solution.

Monday, August 10, 2015

Links of the day 10/08/2015 : Cryptocurrency architecture, Distributed data stream processing and Rump kernels

  • Architecture Of A Cryptocurrency : good overview of what make a cryptocurrency and the five pillars of its architecture : Network consensus, transaction Protocol and internal state
  • High-throughput, low-latency, and exactly-once stream processing with Apache Flink : on the evolution of fault-tolerant streaming architectures and their performance
  • On rump kernels and the Rumprun unikernel : a very good overview of the rump kernel . As stated, One of the key advantage of this type of unikernel is that it can run unmodified "legacy" applications. By that I means you can use your tested, running application that wasn't design for unikernel and slap it on top of rump. To a certain extend it allow a easy and smooth transition without the pain of re-coding everything from scratch. 

Thursday, August 06, 2015

Links of the day 06/08/2015: #Microservice 7 deadly sins, NVM memory stick, Fat vs skinny node

Diablo Tech Memory 1 $ / GB


Wednesday, August 05, 2015

Links of the day 05 - 08 - 2015 : Homer Explain #SAP, awesomeness, cryptii


  • Homer explain SAP : very good description of SAP and its evolution. As an ex-employee of the mastodon i would have loved to have this given to me on my first day with them.
  • Get awesomeness : To retrieve all amazing awesomeness from Github
  • Cryptii : convert, encode and decode content between different format systems. This happens fully in your browser using JavaScript, no content will be sent to any kind of server.


Tuesday, August 04, 2015

Links of the day 04 - 08 - 2015 : #AWS vs #AZURE , #Trends of Memory and #Datacenter technology



Monday, August 03, 2015

Links of the day 03 - 08 - 2015

Today's links  03/08/2015 : API cheat sheet, NVMe, Git for #Bigdata , Huffman compression explained
  • Over API : cheat sheet for many programming language python , Javascript, NodeJS, ruby, Css, and many more
  • NVMe Integrator's List : Full list of NVMe devices straight form UNH IOL
  • DAT : US gov data collaboration tool , basically a git for data.
  • Huffman - part1 [part2] - [part3] : nice series of posts on Huffman encoding