HPC ecosystem - SC20

 This article is a quick summary of SC20 trends and the current state of the HPC ecosystem from a tech and market perspective.

Technology-wise there are three main competing HPC architectures: 

• Commodity (e.g. Intel)
• Commodity + accelerator (e.g. GPUs)
• Lightweight cores (e.g. IBM BG, Xeon Phi, TaihuLight, ARM )

Commodity systems represent the bulk of the systems out there. However, commodity + accelerator are ramping up their presence aggressively. Nvidia dominates this market segment with 142 systems out of 149. With Intel scooping 4 with it's Phi solution. Lightweight cores systems are a minority with only four systems. But with the new A64FX and a renewed appetite for custom chips, this might change rapidly.

Intel is still dominating the ecosystem, with 92% of the shares, followed by AMD with 4%. However, this might change rapidly with AMD EHP technology ramping up. Another aspect is that AMD technology tends to be more open-source friendly, which can make it more attractive long term. Not to mention that their GPU also start to become highly competitive in the AI space.

From a market size, the HPC market was $39.0 billion in 2019, up 8.2% from $36.1 billion in 2018. Predictions show growth to $55.0 billion in 2024. Most of the growth was led by government spending after six years of growth led by industry. The number of system in industry vs public is not equally divided with ~50% each.

One notable change is double-digit growth of cloud HPC related market. Cloud grew 17.8% to $1.4 billion; however, this might only be the tip of the iceberg as many companies might be using HPC like system in the cloud without labelling it as HPC. Cloud solutions are heavily displacing low-end HPC segment. Entry and mid-range level server classes have the slowest growth in years as consumers prefer to buy HPC as a service solution and reduce their CAPEX. 

AI is still heavily influencing the HPC infrastructure market as it represents a considerable opportunity for HPC solution vendors. HyperscaleAI infrastructure by itself is about $8 billion. It seems that for the moment, AI and HPC future are closely intertwined.

Sources: Intersect360 research - Pre-SC20 Market Update & Jack Dongarra - An overview of HPC

The real motivation behind the Matrix engine (GPU/TPU/...) adoption in HPC

 There is current backlash in the HPC community against GPU/TPU/... aka matrix engine accelerator adoption. Most of the arguments are performance, efficiency and real HPC workload driven. 

Like in a recent paper by Jens Domke et Al., colleagues of Dr Matsuoka at RIKEN and AIST in Japan, explore if the inclusion of specialized matrix engines in general-purpose processors are genuinely motivated and merited, or is the silicon better invested in other parts.

I wrote before in that a lot of new HPC systems overuse matrix engine hardware in their architecture. In this paper, the authors looked at the broad usefulness of matrix engines. They found that there is only a small fraction of real-world application that use and benefit from accelerated dense matrix multiplications operations. Moreover, when combining HPC and ML or when you try to accelerate traditional HPC applications, the inference computation is very lightweight compared to the heavyweight HPC compute. 

While I agree with the argument put forward some other aspects that go beyond HPC need to be taken into consideration as to why there is such a push for matrix engine adoption. And these aspects are mainly market-driven. If you compare markets, there is significantly more money in the "hyped" years old AI market (training + inference) vs the 30 years old "mature" HPC market. 

In raw numbers, the HPC market is worth $39 Billion. In comparison, the AI market is worth $256 Billions in hardware along. If you focus on AI semiconductor only it is still $32 Billion alone! And the growth projections are not in favour of HPC. 

If you then look into the N^4 computing complexity for AI vs at best N^3 for HPC. Or look where (institutions, companies, systems/individuals such as in cars, wearables, medical appliances, etc.) those AI systems are going vs HPC systems. You quickly understand the significant difference and potential between the two markets.

If you take the ROI of AI-related business into consideration, it now makes more sense why HPC institutes are investing in such type of hardware. Such investment will allow them to tap into a promising and fast-expanding market. The matrix engine movement is simply a market-driven investment to ensure the best ROI for HPC centres.

Amazon ads stuffing and upcoming “China Crisis.”

This article on #amazon stuffing its search results with ads reminded me of the tipping point when I did the switch from #altavista to #google search.

When the results are optimised to sell you something vs tuned to answer your search, the user starts to lose confidence in the service.

As David J. Carr suggested:

This behaviour is probably rooted in the tension between Bezo’s “I’m sceptical of any mission that has advertisers at its centrepiece” and Olsavsky’s “Our strategy is to make the customer experience additive by the ad process”.

Moreover, it indicates a switch from quantity over quality as they are probably willingly accepting that they can’t guarantee the quality & the authenticity of the products sold by the swarm of third-party vendors under the Amazon brand ( as in via Amazon website).

Last but not least, Brands and Sellers are increasingly competing with 3-week product lifecycle of pseudo-brands. Chinese producer can get to market a smartwatch lookalike within five days and using these self-serve ad platforms can outspend x100 at the bottom of the funnel.

As it becomes hard to compete with those Amazon is naturally increasingly banking on its logistics and distribution platform advantage. When the pseudo-brand life-cycle shorten the only common part of the product experience that will remain constant will be the delivery experience.

Intelligence cannot be commoditized

In the first posting, we saw how the enterprise world needs to be realistic regarding its expectations of data science tools. In this second posting, we will be looking at why it is still essential to embrace them sooner rather than later or else they run the risk of suffering dire consequences.

Intelligence cannot be commoditized

Companies must understand that they need to rapidly embrace data analytic methods as they are essentially the next stage of evolution in the enterprise toolkit for understanding and leveraging information to gain competitive advantages. While enterprises should not just jump and sprinkle “bigdata” everywhere, but they also should not wait too long to embrace these technologies as you cannot commoditize Intelligence.

Almost everything else is commoditizing over time, albeit at different pace but nonetheless it is evolving from custom solution to product and finally to utility. However, for data science tools, the opposite actually happens. Data and knowledge improve by use and accumulation of information and one cannot hope that it will be commoditized for the particular enterprise at some point.

In short, “thee” who has the best data (and uses it) wins. If you wait too much to collect and learn from it you are giving an advantage to your competitors that you may never recover from. By adopting such techniques early on, an enterprise will gain an edge over the rest of the actors within its ecosystem as they will now have to catch up. Moreover, very quickly, big-data - machine learning tools - analytic models will become widely available, as well as affordable - and these same tools will rapidly devolve to become another weapon in the market Red Queen’s race.

As you can see it can be rather dangerous to delay the adoptions of these novel methods within a given enterprise because of its intrinsic nature. In the following posting, I will look into the double edge sword that is the data itself.