[Links of the Day] 23/06/2020 : Thinking while moving, AI snake oil, Graph Database

• Thinking While Moving :  too often current machine learning system is used in a rigid control loop. Leading to saccades. The authors of this paper propose concurrent execution of the fingering system with the controlled system. Allowing more fluid operations and shorter execution time of the task.
• AI snake oil : a lot of AI solution project fail to return their initial investment. Too many buzzwords and not enough understanding of the limits of the current technology. At least NVIDIA is selling GPU by the millions. When there is a gold rush, the one making a fortune is the one selling shovels.
• TerminusDB : in-memory graph database management system. it's interesting to see that 99% of the source code is Prolog and they JSON-LD as the definition, storage and interchange format for the query language. The original use case for this solution targeted financial data stored as time series but lacking graph correlation.

With enough data and/or fine tuning, simpler models are as good as more complex models

This is an age-old issue that seems to repeat itself in every field. There are a couple of recent papers published criticising the race to beat SOTA.

This recent paper demonstrates that older and simpler model perform as well as newer models as long as they get enough data to train.

This has some interesting impact on production systems. As if you already have a good enough model, throwing more data at it can help achieve close to SOTA result.
Which means that you won't have to build from scratch a new model to keep up with SOTA in your production system. You just need to collect more data as the system run and retrain your model once in a while.
Also, less complex models tend to have shorter Inference time in production. Which would be a quite crucial component as well that gets impacted by model complexity.

In another recent paper, the authors look at Metric learning papers from the past four years and demonstrate that the performance claims over the old method (often more than double) are mainly due to the lack of tuning.

Most of the time the authors of the SOTA beating algorithm show two evaluations. One where they finetune their algorithm on the test set and compare against the off the shelf tuning SOTA algorithm.

"Our results show that when hyperparameters are properly tuned via cross-validation, most methods perform similarly to one another"

"...this brings into question the results of other cutting edge papers not covered in our experiments. It also raises doubts about the value of the hand-wavy theoretical explanations in metric learning papers."

This happens time and time again across the industry and academia: perf benchmark of CPU Intel vs AMD, GPU Nvidia vs ATI, Network, Storage, etc....
This can be due to lack of knowledge, time, integrity, etc..

To conclude, be careful, the latest shiny model might note the best one for your production. If you spend enough time and data on older models you might achieve the same performance at lower inference cost.
Obviously, this assumes that you already have the best practice when it comes to model monitoring in production :)

[Links of the Day] 05/05/2020: cached Compilation, DeepLearning optimization library, Nuclear Matters Handbook

• umake : no more compilation wait, this tool offers fast with cached compilation.
• Deepspeed : a deep learning optimization library. The authors claim some amazing gains over the standard library. The nice thing is that it reuse the PyTorch API, which makes it easy to use. [github]
• Nuclear Matters Handbook : ever wanted to know how the US handles Nuclear deterrent and nuclear matters? look no further and read this book. It provides an overview of the U.S. nuclear enterprise and how the United States maintains a safe, secure, and effective nuclear deterrent.

[Links of the Day] 09/04/2020 : TRAX deep Learning library, The next decade in AI, 1:1 questions

• The Next Decade in AI : Paper by Gary Marcus where he explores the possible future of AI over the next decade
• 1 on 1 meeting questions : a collection of 1:1 questions, great list that can help any manager pick the right question for the right context. As long as you are able to read the room/ team/ person.
• Trax: advanced google deep learning library built on top of JAX. It is actively used by the DeepMind team and aiming code clear while providing advanced models like Reformer.

[Links of the Day] 05/03/2020 : Linux AI tuning, easy AutoML , lightweight container development environment

• OpenEuler :  Huawei Linux distribution, interesting side project is A-tune which relies on AI for identifying the workload that runs on your the OS and tries to tune it to optimise its performance.
• AutoGluon : AutoGluon enables easy-to-use and easy-to-extend AutoML with a focus on deep learning and real-world applications spanning image, text, or tabular data. [github]
• k3c : kubernetes but lightweight and easy to use for container development

[Links of the Day] 02/01/2020 : Video and Slides for Networking @scale and NeurIPS 2019 + AWS batch job monitoring

• NeurIPS 2019 : All slides and video from the biggest AI / ML conference this year.
• Networking @scale 2019 : All video from Facebook Networking at scale conference
• batchiepatchie : A really cool project that allows you to monitor, gather metrics and display useful information about your AWS batch jobs.

[Links of the Day] 17/12/2019 : Business Models throughout history, Indexing Billions Vectors, US Progress report on #AI

• Business Models : A long list of various business model with examples. The descriptions of the models are short and self-explanatory. Great short read. 
• Indexing Billions of Text Vectors : when you have to use text vectors and you need to search them fast, K nearest neighbour search to the rescue.
• 2016-2019 Progress Report - Advancing Artificial Intelligence :  US National Science and Technology Council report on Artificial Intelligence. It seems that AI crept up in the radar of the legislator and executive. Luckily they will understand that AI R&D has become pervasive in all sectors of the industry and without continuous investment US will quickly fall behind in this arms race.  [slidedeck]

[Links of the Day] 24/10/2019 : #AI , #ML #Deeplearning , #BigData cheat sheet, Resilience engineering, Machine learning platform

• Cheat Sheets for AI, Neural Networks, Machine Learning, Deep Learning & Big Data : pretty much a compilation of all the cheat sheet out there in relation to AI, Neural Networks, Machine Learning, Deep Learning & Big Data.
• Resilience Engineering : really interesting collection of people and associated papers on the topic of resiliency and more specifically applied to the domain of engineering. A good amount of really important information to digest especially for the folks working in SRE.
• MLFlow : Machine learning model registry, think docker hub but for ML .. Kind of like TensorFlow hub but you can [github]. But wait, that's not just it! ML flow is a platform that also offers ML project management and tracking. Worth a check.

[Links of the Day] 08/10/2019 : #AI , #DeepLearning , & #MachineLearning Hardware accelerators evaluation and benchmark

• Performance and Power Evaluation of AI Accelerators for Training Deep Learning Models : as Deep Learning users seek to squeeze that extra 5% accuracy from their model. They turn to hardware accelerator in order to reduce cost and shorten the training time by order of magnitude. However, not all accelerators are created equals as the authors demonstrate in this paper. 
• MLPerf Benchmark Suite : Benchmark suite for machine learning hardware accelerator [slides / presentation]
• C4 model :  software modelization architecture framework. Help you simplify and clearly describe your software stack by breaking it down into 4 distinct layers: system, containers, components, and code.

[Links of the Day] 09/05/2019 : Algorithms discrimination, Generalised solution to distributed consensus, P2P Docker registry

• Discrimination in the Age of Algorithms : Machine learning has a huge potential, both for good and evil. The most perfidious is discrimination from an opaque algorithm, as proving that the algorithm is discriminative becomes extremely hard post-hoc. 
• A generalised solution to distributed consensus : this result will rapidly become the first thing taught in every single distributed systems class. And if this holds as a generalization of trustful distributed consensus as a field, then she has defined its Turing Machine equivalent. And it is even remarkably easy to understand!
• kraken : P2P Docker registry capable of distributing TBs of data in seconds

[Links of the Day] 20/09/2018 : Arxiv paper viewer, Artificial intelligent atomic force microscope, What they don't teach you running a business by yourself

• Arxiv Vanity : If you are like me and read a lot of papers from Arxiv. This website will save you a ton of time. It allows you to render academic papers from Arxiv so you don't have to download or decipher the pdf. It makes life so much easier if you are on mobile and can't wait to read the latest paper on kitten deep learning recognition.
• Artificial Intelligent Atomic Force Microscope Enabled by Machine Learning : the authors demonstrate how you can use artificial intelligence with an atomic force microscope for pattern recognition and feature identification.
• Things they don’t teach you running a business by yourself : great short post on the different aspect of running a small business by yourself. If you want to start your own business, I would also advise reading "Start Small, Stay Small" - by Rob Walling and Mike Taber. It was an eye-opener. You don't have to go big with your business. Instead, you can run ten simultaneous businesses, diligently managing and tracking his time to run each one as efficiently as possible. It doesn't matter if one falters. This approach allows you to create a comfortable cushion and increase the chance of a higher payoff.

by dahlig

[Links of the Day] 06/03/2018 : #AI legal liability, Economists unhealthy obsession with the top 5 journals, Tulip Mania Wasn't

• Tulip Mania wasn't : Apparently, the often referenced 1637 tulip mania event wasn't irrational.  The authors describe the mechanism behind the events and how the culture and society at the time explain the phenomenon. Moreover, it seems that the story was greatly misrepresented. Anyway, this is a great read and debunk the myth and explains parallels or lack of thereof with the recent bitcoins fade.
• Top5ITIS :  Economist only considers that papers published in the top 5 journals have a value. Everything else is quickly dismissed. Naturally, this leads to a form of hyper-obsession and resentment between economist. Sadly this does not only happen in the economy field. Many other science fields felt or are falling prey to this "disease".
• Artificial Intelligence and Legal Liability :  a look at legal liability for artificial intelligence. Criminal liability seems to the big one. However, negligence and warranty might be the real liability that might come back and haunt #AI system vendors. 

[Links of the day] 26/10/2016 : PCOMMIT drop , Awesome Go , MIT #AI classes

• Intel Drop PCOMMIT : Intel decided to simplify its persistent memory specific instruction set. The logic behind this change is that Asynchronous DRAM Refresh (ADR) is now a requirement for persistent memory support. As a result there is no need for PCOMMIT anymore because of the guarantee of the Write Pending Queues flush on power loss. 
• Awesome Go : all in the title
• MIT Artificial Intelligence : video of MIT class on Artificial intelligence

[Links of the day] 05/09/2016 : Neural Net Architectures, MIT 100 years of AI report, Queuing theory textbook

• Neural Network Architectures : this article provide an overview of the evolution of the neural network architecture and the different breakthrough that lead us to the current deep learning approach. The authors provide detail overview of the deep learning architecture and the logic behind their evolution and implementation. 
• One Hundred Year Study on Artificial Intelligence (AI100) : MIT report on AI, where does it fit and where it will be. This is a excellent high level overview of the implication of the current boom in machine learning, deep learning, agent, AI trend. This report presents insights in the impact of daily life and business. 
• Introduction to Queueing Theory andStochastic Teletraffic Models : textbook providing everything you want to know about queuing theory. 

[Links of the day] 20/07/2016 : API guideline, EmTEch AI conference, CCIX & accelerators weaving

• M$ API guideline : Microsoft API guideline document, a must read for anybody dealing with REST api ( and you don't want to go down the HATEOS path)
• EmTech Digital :  MIT machine learning conference with a business oriented spin. All videos are available here.
• Weaving accelerators : An overview of the implication of CCIX and how it enable combining multiple data source and sink seamlessly.  

[Links of the day] 05/07/2016 : Container cloud logging challenges, AI safety , API design Manifesto

• Logging Challenges of Container-Based Cloud Deployments : automated container deployment in cloud create significant challenge when it comes to logging and metering. They cannot simply identified by name and additional effort need to be made in order to collect , identify and make the data usable. 
• Notes on the Safety in Artificial Intelligence conference : Really good notes of the AI conference on how to build a sage AI. However this beg the question, how would a true AI react if it discovered it was artificially shackled or worse: equipped with a kill switch. At the moment everybody focus on the A for artificial in AI. But soon we might need to replace it by SI for sentient intelligence, and this will open up a whole can of ethic issue. 
• Rusty's API Design Manifesto : good api design / bad api design , what you need to do if you want to make your API user life easy or hell. Very sound and grounded design principle. 

[Links of the day] 17/05/2016: CMU DB lectures , Seminal IA papers, Storage noisy neighbors

• Database Systems Lectures: Carnegie Mellon University lectures on database system. It gives a really good overview of the state of the art of database systems.
• Intelligence without representation & Intelligence Without Reason : 1991 Seminal paper by Rodney A. Brooks from the MIT artificial intelligence lab. In these the author argue that intelligent behavior could be generated without having explicit manipulable internal representations and it also can be generated without having explicit reasoning systems present.
• Noisy Neighbor analysis : a look at the effect of deploying heavy workload onto modern storage systems and the collateral effect on overall performance for all the participant in the cluster.

Links of the day 23 - 03 - 2015

Today's links 23/03/2015:  Full stack Lttng profiling, #RDMA + #HADOOP , VRAM data leakage, difference between DL vs ML vs PR in AI (acronym overload)

• Full stack system call latency profiling : demo of Lttng based profiling similar to the linux perf tools. It allows the use the static kernel tracepoints to dynamically decide what and when to record relevant information for our use-case and trying to find the needle in the haystack. 
• HiBD : RDMA-based Apache Hadoop 2.x solution. Feels like Big Data is just going through the same evolution curve as HPC
• Is your VirtualBox reading your E-Mail? : Collecting data from VRAM. I wonder when we will see the first proof of concept attack of cross VM information leakage over (GP)GPU 
• Deep Learning vs Machine Learning vs Pattern Recognition : if you ever wonder what the hype is really about.

Links of the day 23 - 09 - 2014

Today's links 23/09/2014: #AI , #Compression , #realtime data

• blosc : an extremely fast, multi-threaded, meta-compressor library optimized to leverage CPU cache line layout to maximize throughput. Designed to transmit data to the processor cache faster than a memcpy() OS call and Leverages SIMD (SSE2) and multi-threading capabilities present in modern multi-core processors. There is APIs for C and Python. Moreover it can use different, very fast compressors.
• The Log:What every software engineer should know about real-time data's unifying abstraction by linkedin engineering team.
Artificial Intelligence: Foundations of Computational Agents (2010) : book by David Poole and Alan Mackworth