HPC ecosystem - SC20

 This article is a quick summary of SC20 trends and the current state of the HPC ecosystem from a tech and market perspective.

Technology-wise there are three main competing HPC architectures: 

• Commodity (e.g. Intel)
• Commodity + accelerator (e.g. GPUs)
• Lightweight cores (e.g. IBM BG, Xeon Phi, TaihuLight, ARM )

Commodity systems represent the bulk of the systems out there. However, commodity + accelerator are ramping up their presence aggressively. Nvidia dominates this market segment with 142 systems out of 149. With Intel scooping 4 with it's Phi solution. Lightweight cores systems are a minority with only four systems. But with the new A64FX and a renewed appetite for custom chips, this might change rapidly.

Intel is still dominating the ecosystem, with 92% of the shares, followed by AMD with 4%. However, this might change rapidly with AMD EHP technology ramping up. Another aspect is that AMD technology tends to be more open-source friendly, which can make it more attractive long term. Not to mention that their GPU also start to become highly competitive in the AI space.

From a market size, the HPC market was $39.0 billion in 2019, up 8.2% from $36.1 billion in 2018. Predictions show growth to $55.0 billion in 2024. Most of the growth was led by government spending after six years of growth led by industry. The number of system in industry vs public is not equally divided with ~50% each.

One notable change is double-digit growth of cloud HPC related market. Cloud grew 17.8% to $1.4 billion; however, this might only be the tip of the iceberg as many companies might be using HPC like system in the cloud without labelling it as HPC. Cloud solutions are heavily displacing low-end HPC segment. Entry and mid-range level server classes have the slowest growth in years as consumers prefer to buy HPC as a service solution and reduce their CAPEX. 

AI is still heavily influencing the HPC infrastructure market as it represents a considerable opportunity for HPC solution vendors. HyperscaleAI infrastructure by itself is about $8 billion. It seems that for the moment, AI and HPC future are closely intertwined.

Sources: Intersect360 research - Pre-SC20 Market Update & Jack Dongarra - An overview of HPC

ARM ecosystem disintegration and the rise of RISC-V

#ARM acquisition by #Nvidia is making people uneasy. 

And the early sign of the unravelling of the #ARM ecosystem start to appear: ThunderX3 general-purpose ARM CPU has been cancelled.

One would ask why spending $$ to build a better product and increase its number of consumers if, for that, it will have to use the Nvidia IP and compete directly against the IP owner.
If you combine this with the difficult viability of putting together a general-purpose #ARM alternative to #Intel / #AMD as #ARM vendors are effectively competing on cost with much lower volumes.

We start to understand why Marvell decided to shift toward the much more trendy IPU/PDU/Smartnic market.

On the other hand, I think we will see an acceleration of RISC-V adoption. Eating away at the traditional #ARM market share. This will be driven by the large scale edge deployment of #riscv sees chips with a RISC-V core and an #NPU (neural processing unit). These chips can be churned out at incredibly cheap cost, less than $10, and these will become ubiquitous really rapidly.

It might take 10-15 years but ultimately this will seal the fate of the ARM franchise.

[Links of the Day] 05/12/19 : Tensor Processing Units for Machine learning, Short history of Intel CPU registers, Chrome as a service

• Tensor Processing Units architectures : David Patterson describes the evolution of the TPUs unit over three generations.
• Short history of the Intel CPU's register : i4004, 1971 => i8008, 1972 => i8080, 1974 => Z80, 1976 ...
• Chrome as a service :  chrome in docker that you can run in the cloud easily, google cloud run is probably your best friend in this case. [git]

[Links of the Day] 19/06/2018 : Facebook network balancer, Open policy agent, Intel NLP libs

• OPA : an open source policy agent that decouple policy from actual code logic. This is essential to provide great flexibility with fine-grained control of resources. These kinds of features are a key building block for secure and robust API based solution. [github]
• Katran : facebook scalable network load balancer. It relies on eBPF and XDP from the Linux kernel to deliver impressive performance at low-cost thanks to its capability to run on off the shelf hardware. [github]
• NLP Architect : Intel NLP library and solution. Sometimes I feel that Intel has some great hardware and software but the release cycle is rather decoupled. Which often leave the user in an odd situation, where the hardware is out but the software is not there yet. And sometimes it's the opposite. I really feel that Intel should work on this. Maybe externalise the software to a separate entity as the hardware culture might be impeding the software side of the company.

[Links of the Day] - 23/05/2017 : Intel Manufacturing Forecast, Topological Quantum Computing and Serverless Conf Video

• Intel Manufacturing Conf : sets of slide deck giving a peek into Intel manufacturing process and the upcoming wave of 10 nm chips. It seems Intel is currently keeping up with Moore's law, not by reducing the transistor size but by also increasing transistor density.
• Introduction to Topological Quantum Computation : introduce the concept of quantum computing with anyons which allow a more resilient quantum computing system. 
• Serverless Conf video :  almost all video are now available. Check out the Serverless at Nordstrom video. This is, in my view, the best of the bunch. It's an actual practical talk by the dev who implemented it and without marketing spin. 

[Links of the Day] 08/03/2016 : Intel blockchain, Fast17 conference and papers, AWS cloud formation devops tool

After a small hiatus, here is the return of the links of the day.

• Sawtooth Lake: Intel distributed ledger system. It uses an interesting security mechanism to deliver secure consensus. Sadly it relies on Intel proprietary hardware encryption modules to deliver this feature.
• Fast17: File and Storage technology Usenix conference happened last month. There were a couple of interesting papers but one picked my interest: Redundancy Does Not Imply Fault Tolerance:Analysis of Distributed Storage Reactions toSingle Errors and Corruptions. The authors look at single file system fault impact on Redis, ZooKeeper, Cassandra, Kafka, RethinkDB, MongoDB, LogCabin, and CockroachDB. Turns out most systems are not able to handle these type of faults very well. It seems that a single node persistency layer error can have an adversarial ripple effect as distributed system seems to have put way to much trust in the reliability of this layer. Sadly they lack tools for recovering from errors or corruption emerging from file systems.
• Stacker : remind 101 tools for creating and updating AWS formation stacks. Looks like an interesting alternative to terraform. 

[Links of the Day] 24/01/2017 : Numpy cheat sheet, Innovations patterns, Persistent memory summit

• Numpy Cheat Sheet: all you need for data analysis in python with NumPy 
• Mathematical Model of innovation patterns: Vittorio Loreto at the Sapienza University of Rome in Italy and al, created the first mathematical model that accurately reproduces the patterns that innovations follow. 
• Persistent Memory Summit: SNIA NVM summit 2017. Finally, with the introduction of Intel 3dXpoint, we start to see more HW NVM solution out there. And with that software that uses it. Some really interesting talks: 
• Nova file system demonstrates the benefit of NVM optimises storage solution. 
• SAP Hana on NVM: interesting to see that they still require redundant copies as they fear data corruption on NvDimms. I wonder when we will start to see ECC NvDimms on the market? 
• New interconnect: lots of hot new interconnects battling for the heterogeneous compute ecosystem domination. And this pass by offering persistent memory specific solution:  Gen-z provide PM pooling, Open CAPI accelerate PM access and CCIX share PM 

[Links of the Day] 23/11/2016 : AMD Exascale vision, Hardware Resiliency myths and truths, MIT EmTech

• Resiliency for Reliability– Myths and Truths : this slide deck provide an overview of the resiliency issue and how Intel tackle those for  hardware fault. From fans down to soft errors ( ex: neutron beam ... yes this can £%£ your system). The authors present the two type of approach , reactive and proactive handling of errors.
• AMD's Exascale computing vision : Its all about 3d stacked chip with future interconnect. The interesting bit is the ROCM platform and the P2P multiGPU and P2P with RDMA. Slowly we are removing the need to have a full server to deploy GPU, one step closer to fully modular system with each resourced pooled and optimized in their own enclosure. Its a lot easier to design power supply, cooling system, etc.. When you do not have to deal with heterogeneous hardware with different power, and cooling profile ( cpu, memory , disk etc.. in the same enclosure).
• MIT EmTech 16 : This year MIT EmTech is all about AI & machine learning ... reaching maximum hype in the domain

[Links of the Day] 18/11/2016 : Extreme Scale OS, GPU Stream Benchmark, Neural Net that produce neural net

• Neural Architecture Search with Reinforcement Learning : Neural net that produce neural net. Cool thing is that the authors are able to beat human generated model for text processing and deliver equivalent performance for image processing model. Who needs human anymore.... 
• Extreme-Scale Operating Systems : multi-OS research project at Intel aiming to be the node OS for HPC machine. Intel is trying to deliver a polymorphic OS that can quickly adapt to new software and hardware without the need for specialized solution like it exist commonly on high end HPC systems. To some extend it looks like the Jailhouse system. Where the HW is physically partitioned. A few core are dedicated for management, while the rest are partitioned and are running lightweight kernel (LWK) + application. Note that I really resent Intel for always trying to rename things that are commonly used. LWK are Unikernel dammit.. Anyway its jailhouse + unikernel for HPC. 
• GPU-STREAM : Stream benchmark for GPU, much needed benchmark to understand and quantify memory transfer rate to from global memory device on GPUs.

[Links of the Day] 11/11/2016 : Anonymous Trustless Bitcoin, Zap golang log lib, Intel RSA controller

• ZeroCash : Trustless Bitcoin Tumbling, the authors proposed a pooled approach to anonymise transaction in Bitcoin. However the authors go a step further than just pooling. They popose a system where participant can anonymously check in and out resources from a global pool. Effectively creating an anonymous cooperative resource sharing infrastructure. [github] [Paper]
• Zap : Fast, structured, leveled logging in Go. When you start to reach Uber or other hyperscale microservice architecture. Every aspect counts, and logs are everywhere. This library provide a high performance structure log for go. 
• Scalable software controller : This controller basically allow to allocate on the fly hardware ressource, compute, memory storage, network based ont the demand of the deployment tool (openstack, k8, mesos, etc..) . 

[Links of the day] 19/10/2016 : #AI hard problems, Dark Silicon & Reliability , Transport Layer Dev Kit

• Applied AI hard problems : current and future AI hard problem, the interesting bit is the "emergent" behavior aspect that computer scientist are trying to achieve. Where AI is not tailored for a specific problem by adapt to the environment it encounter. 
• Dark silicon & Hardware Reliability : the authors look at the impact of the dark silicon approach ( when not all component are turned on when the system is up) and how to leverage the "dark" ratio to maximise lifespan of hardware. [slides]
• TLDK : project lead by Intel within the fd.io framework. It is trying to adresse the lack of high level ( as in layer 4 ) packet processing capabilities. The project aim at delivering UDP/TCP etc.. packet processing on top of vector packet processing of FD.io (which can works on top of DPDK). By doing so Intel will be able to finally have a comprehensive framework which will enable DPDK based solution to flourish beyond the pure networking stack (NFV) solution.

[Links of the day] 23/09/2016 : Intel's 3dxpoint vanishing performance, VLDB16, Core to Core HW queue engine

• 3dxpoint performance evaporate : seems that Intel is heavily scaling back its xpoint NVM performance claim. From 1000x to 10x ( still good but a far cry from what was promised). It seems that Intel had to push the technology early in order to counter a potential acquisition of its partner, Micron, by a competitor. Announcing the technology surely propped the share price making an acquisition difficult. 
• VLDB : very large databases 2016 proceedings are out. Sadly its one big zip file and didn't have time to go through it.
• CAF : the authors propose a hardware core to core communication offloading engine. Providing an efficient queuing mechanism for transferring data between cores. I am not sure 100% of the value but the concept is interesting, let see if it catch on and if it can plays well in heterogeneous environment of today's datacenter. As core to core is slowly replaced with cored to GPU or core to FPGA or core to NVM.

[Links of the day] 22/08/2016 : Kubernetes the hard way , GopherCon 2016, 3dxpoint DIMM

• Kubernetes The Hard Way : The famed Keysley Higtower explain how to deploy and run a kubernetes system on google cloud the hard way ( no automatic installation cheat)
• GopherCon 2016 : Videos of this year Gopher academy
• Wicked Fast Storage and Beyond : Intel IDF 2016 talk on the future of storage , looking at 3dxpoint, and the Optane SSD . What is really exceiting is the upcoming Intel DIMM using 3DXpoint tech. This provide native PMEM capability , 2x storage vs RAM (but slower yes). I can seriously see future in memory database ( SAP HANA ... ) bypassing storage together and just stacking NVM DIMM. Using slower storage form ( over NVMe by example) for backup / snapshot, etc..

[Links of the day] 12/05/2016: Lustre + Omnipath in Bridges Supercomputer & Storage Media Evolution

• Lustre + Omnipath : HPC filesystem of choice meet Intel Omnipath fabric. Intel was poised to release such crossover as it continue to push in the HPC domain and rack infrastructure domination . Remember that  Intel acquired Whamcloud (Lustre) a while back.
• Storage Media Overview: Historic Perspectives of storage solution. Interesting snippet of information all storage media revenue decreased from 2014 to 2015 except for NAND. However, NAND revenue increased by 30% in 2014 but only 3% in 2015. Hinting a plateau of the tech and entering a commoditization phase with lower margin. [Video]
• Bridges :supercomputer being built at the Pittsburgh Supercomputing Center (PSC), they have a really cool Virtual Tour .

[Links of the day] 02/05/2016 : All about storage @ Intel IDF 16 + no more secret

• Storage Transition : NVM Express and PCI Express in the Client and Data Center
• Intel Non-Volatile Memory Inside : A look into 3d NAND and next gen SSD
• Modern Storage Architectures : The implication of new fabric and NVM tech for modern storage architectures
• No more secrets : recreate the famous "decrypting text" effect as seen in the 1992 movie Sneakers

Links of the day 11/12/2015: Devops Summit, Intel's HPC framework, Cybercrime fail

• DOES15 : DevOps Enterprise Summit 2015 video
• Intel’s HPC Scalable System Framework Plans : delves into aspects of the framework and highlight the relationship and benefits to application development and execution.
• Cyber crime: Preventing young people from getting involved : this is just begging for a parody anti piracy video style

 

Links of the day 09/12/2015 : Intel telemetry framework, log search system and DIGITAL!

• Snap : Telemetry framework by Intel. While this is yet another monitoring system it is a little bit different from what is out there. It offer a full secure clustering solution design to be not only scalable but also has a integrated flexible life-cycle management of its component. This might looks like a great framework for future telemetry systems and I can see it being adopted very quickly across the industry.
• Designing a search system for log data : 3 part post on how to design from the ground a system for analyzing  logs at large scale.
• Digital : because digital!

Links of the day 04/12/2015 : Sub-NUMA patent, Fabric Emulation for HP Machine, Performance monitoring toolkit

• Sub-NUMA : Numa is already hard to deploy and use efficiently, lets start bringing sub numa clustering. Sadly one of the consequence of the ever increasing core count. Rather than selling micro chip single core highly optimize CPU. Intel prefer to cram more feature into a larger package even if 50%+ of them are unused. Sometimes I really hope that ARM server will finally come to fruition and shake the whole damn market.
• Fabric attached memory : want to play with HP "the machine" and its new shiny fabric but can't get early access to it or you don't have the finance. Fear not ! here is the emulation toolkit of Fabric-Attached Memory for The Machine.
• LIKWID : Performance monitoring on Intel Haswell platforms easy to use but yet powerful performance tools for the GNU Linux operating system. Really cool set of tools for topology discovery , numa pinning, power monitoring etc... [github]

Links of the day 19/11/2015: #ARM rack scale infrastructure fabric, Canonical ByoHW #Cloud software and Next Gen #Bigdata Storage

• X-Tend : Applied micro get into the rack scale computing market with its ARM server and now the fabric to enable and support dis-aggregation of the resources. What is interesting there is since the ARM processor are more nimble in a certain sens it might make them more suitable for this paradigm than the big Intel processor. And if you throw into the mix hybrid and heterogeneous processors might end up having a better fit as you can more easily tailor the resource to your need as well as the power consumption coupled with it.
• Autopilot : canonical private ByoHW cloud solution (more about that in another post).
• Pyro: A Spatial-Temporal Big-Data Storage System for high resolution geometry queries and dynamic hotspots.

Links of the day 06/11/2015: More Intel rack scale resource

A lot of new stuff released by Intel around their rack scale effort:

• Ressource : Probably you main point of entry for the RSA project. Documentation, API and reference architecture. 
• Github : everything you need to get started (as long as you have Intel HW) 
• How it works :  very good and short overview on how Intel intend to Orchestrate the bare metal hardware in order to offer dis-aggregated pools of resources. The objective is to have these transparently and just in time procured for cloud or any workload. Ultimately you will be able to avoid the age long issue of server sizing and simply manage each type of resource independently within pools. Adding and removing them, upgrading them without the need to re-architect your whole data-center.