Data is the new oil fueling Machine learning adoption but Businesses are discovering #AI is no silver bullet

Data is the new oil. However, unlike oil, as data scarcity is becoming less of a problem, processing costs are skyrocketing. The business world is waking up to the fact that while the cost of computing keeps getting cheaper all the time. The cost of training machine learning models is outpacing the compute cost drop.

Moreover,  business are finding challenging to adopt #ai, and the economist report numbers are showing how often #machinelearning projects in the real business world fail :

• Seven out of ten said their #ai projects had generated little impact so far.
• Two-fifths of those with “significant investments” in ai had yet to report any benefits at all.

Companies are finding that #machinelearning is not the promised silver bullet. The non-tech company are discovering what tech companies had to learn the hard way: that they are no Google, Facebook, ...

To successfully deploy an AI/ML/DL project you need: a vast amount of data, skilled employee, solid engineering practice, access to infrastructure and last but not least, a clear understanding of the business problem.

I have a false hope that corporation will abandon the silver bullet thinking, but I would settle for avoiding another #ai winter cycle.

[Links of the Day] 30/06/2020 : Homomorphic encryption for Machine Learning, Neural Network on Silicon, Python Graph visualization library

• PySyft : Python framework for homomorphic encryption for Machine learning. It allows you to train model on encrypted data without the need to decrypt it. It's 40x slower than normal method but you this means you don't have to deal with the new EU regulation on AI. 
• Neural Networks on Silicon : a collection of papers and works on Neural Networks on Silicontopic
• Pygraphistry :  Python visual graph analytics library to extract, transform, and load big graphs in Graphistry 

[Links of the Day] 23/06/2020 : Thinking while moving, AI snake oil, Graph Database

• Thinking While Moving :  too often current machine learning system is used in a rigid control loop. Leading to saccades. The authors of this paper propose concurrent execution of the fingering system with the controlled system. Allowing more fluid operations and shorter execution time of the task.
• AI snake oil : a lot of AI solution project fail to return their initial investment. Too many buzzwords and not enough understanding of the limits of the current technology. At least NVIDIA is selling GPU by the millions. When there is a gold rush, the one making a fortune is the one selling shovels.
• TerminusDB : in-memory graph database management system. it's interesting to see that 99% of the source code is Prolog and they JSON-LD as the definition, storage and interchange format for the query language. The original use case for this solution targeted financial data stored as time series but lacking graph correlation.

With enough data and/or fine tuning, simpler models are as good as more complex models

This is an age-old issue that seems to repeat itself in every field. There are a couple of recent papers published criticising the race to beat SOTA.

This recent paper demonstrates that older and simpler model perform as well as newer models as long as they get enough data to train.

This has some interesting impact on production systems. As if you already have a good enough model, throwing more data at it can help achieve close to SOTA result.
Which means that you won't have to build from scratch a new model to keep up with SOTA in your production system. You just need to collect more data as the system run and retrain your model once in a while.
Also, less complex models tend to have shorter Inference time in production. Which would be a quite crucial component as well that gets impacted by model complexity.

In another recent paper, the authors look at Metric learning papers from the past four years and demonstrate that the performance claims over the old method (often more than double) are mainly due to the lack of tuning.

Most of the time the authors of the SOTA beating algorithm show two evaluations. One where they finetune their algorithm on the test set and compare against the off the shelf tuning SOTA algorithm.

"Our results show that when hyperparameters are properly tuned via cross-validation, most methods perform similarly to one another"

"...this brings into question the results of other cutting edge papers not covered in our experiments. It also raises doubts about the value of the hand-wavy theoretical explanations in metric learning papers."

This happens time and time again across the industry and academia: perf benchmark of CPU Intel vs AMD, GPU Nvidia vs ATI, Network, Storage, etc....
This can be due to lack of knowledge, time, integrity, etc..

To conclude, be careful, the latest shiny model might note the best one for your production. If you spend enough time and data on older models you might achieve the same performance at lower inference cost.
Obviously, this assumes that you already have the best practice when it comes to model monitoring in production :)

[Links of the Day] 18/06/2020 : 3d sensing for pixel 4, Transformer for long document , OCI artworks repo

• uDepth : explain the tech behind real-time 3d sensing for pixel 4
• Longformer : fed up with the 512 token limits of transformers?  here comes long former for long documents. 
• OCI artworks and logo : All OCI artwork and logo under apache 2 license[github]

[Links of the Day] 16/06/2020 : Time Series Forecasting best practices, AWS account controller, Causality Arxiv papers

• Time Series Forecasting : Microsoft Time Series Forecasting Best Practices & Examples [github]
• Aws Account Controller : AWS Self-service creation and deletion of sandbox-style accounts. [github]
• Arxausality : a collection of every causality + machine learning paper submitted to arXiv in the recent past.

[Links of the Day] 02/06/2020 : Real time network topology, Detecting node failure using graph entropy, Monitoring machine learning in production

• Skydive : open source real-time network topology and protocols analyzer providing a comprehensive way of understanding what is happening in your network infrastructure.
• Vertex : the authors propose to use vertex entropy for detecting and understanding node failures in a network. By understanding the entropy in a graph they are able to circumvent the lack of locality in the information available and pinpoint critical nodes. 
• Monitoring Machine Learning Models in Production :  Once you have deployed your machine learning model to production it rapidly becomes apparent that the work is not over.

[Links of the Day] 28/05/2020 : Reverse Oauth proxy , Prometheus timeseries backend, Google use machine learning to improve audio chat

• Oauth Proxy : A reverse proxy that provides authentication with Google, Github or other providers.
• Zebrium : Prometheus backend project, not sure why you don't want to just export the data from Prometheus into a distributed column-store data warehouse like Clickhouse, MemSQL, Vertica. This gives you fast SQL analysis across massive datasets, real-time updates regardless of order, and unlimited metadata, cardinality and overall flexibility. Maybe because they want to focus on the monitoring / reactive aspect and less on the analytics.
• Improving Audio Quality with WaveNetEQ : Google uses machine learning to deal with packet loss, jitter, and delays. An interesting bit of info: " 99% of Google Duo calls need to deal with packet losses, excessive jitter or network delays. Of those calls, 20% lose more than 3% of the total audio duration due to network issues, and 10% of calls lose more than 8%."

[Links of the Day] 27/05/2020 : Combining Knowledge graphs, Real World storage resource management, Jespen black box transactional safety checker

• Combining knowledge graphs : The authors describe a new entity alignment technique that factors in information about the graph in the vicinity of the entity name. It provides a 10% higher accuracy while reducing computational cost for model generation.
• Wizard : project looking into real-world storage reliability for cost-effective data and storage resource management system for reliability enhancement.
• Elle : Jespen black-box transactional safety checker based on cycle detection. You can find more in the Arxiv Paper by Kyle Kingsbury and Peter Alvaro : "Elle: Inferring Isolation Anomalies from Experimental Observations"

[Links of the Day] 21/05/2020 : Aging Bloom Filters, Awesome List of Machine learning Production Library, Large scale data quality management platform

• Age partitioned Bloom Filters : Age-Partitioned Blocked Bloom Filter variant
• Open source libraries to deploy, monitor, version and scale your machine learning : A curated list of open source libraries to deploy, monitor, version and scale your machine learning
• Data Sentinel : Linkedin platform for automatically validating the quality of large-scale data in production environments

[Links of the Day] 19/05/2020 : Fat Tails, AutoML Zero , Version Control for GIS

• Statistical Consequences of Fat Tails : Nassim Nicholas Taleb book investigates the misapplication of conventional statistical techniques to fat-tailed distributions and looks for remedies
• AutoML Zero : aims to automatically discover computer programs that can solve machine learning tasks, starting from empty or random programs and using only basic math operations. The goal is to simultaneously search for all aspects of an ML algorithm—including the model structure and the learning strategy—while employing minimal human bias.
• Sno : Distributed version-control for geospatial and tabular data

[Links of the Day] 14/05/2020 : Wasm In Linux Kernel, Knowledge Graphs, Contrastive Machine learning Model for Software Performance Regressions

• Kernel Wasm : Looks like people want to run WASM everywhere. This time the authors propose to run wasm program in the kernel. In this case, I just wonder if it would not be more judicious to try run WASM in EBPF. From the GitHub repo it seems that they might actually try to do the opposite. [github]
• Knowledge Graphs : A comprehensive introduction to knowledge graphs. If you want to learn more about the knowledge graph I would recommend reading the following paper before reading the arxiv one.
• A Zero-Positive Learning Approach for Diagnosing Software Performance Regressions : really innovative approach by Intel folks there. I have started to see interesting trends in machine learning where instead of trying to train the ML model using a dataset that contains the whole spectrum of possibility. The authors start to use contrastive methods instead. In this case, the ML model is trained on a non-abnormal dataset in order to identify abnormal behaviour. It is much easier in performance evaluation to obtain ideal, or standard metrics rather than abnormal scenario. In this case, the author uses the ideal hardware performance counter to train their model in order to identify abnormal behaviour. [poster]

[Links of the Day] 12/05/2020 : Learning From Unlabeled Data, Fast Dataset Classifier, Azure Bad Rollout guardian

• Learning From Unlabeled Data : Slidedeck of a talk by Thang Luong of Google research. Thang present a novel method for learning from unlabeled data and more specifically semi-supervised learning methods. These methods were used to generate Google Meena Chatbot model.
• Flying Squid : Looks like a super-fast Snorkel with even better performance. Like Snorkel this is used to quickly building classifiers of datasets that would be otherwise extremely time-consuming (and expensive) to label by hand for training purposes.
• Gandalf : Azure machine learning system trained to catch bad rollout deployment. The aims of this system is to catch bad deployment before they can have ripple effects across the whole system.

[Links of the Day] 07/05/2020 : Startup tactical manuals, AutoML pipeline, Thread Caching Malloc

• Tactical manuals and guides for startups : an awesome collection of strategic posts, essays or documents for startups. While these are great resources, it doesn't replace experience.
• AutoML Pipeline : The power of Juila meet Machine learning. However, beware as just feeding data into a system and hoping to get the best result coming out without any effort is doomed to deliver sub-optimal results. Often you end up with an ok-ish solution that blows up in production down the line.
• Tcmalloc : Google Thread Caching Malloc

[Links of the Day] 30/04/2020 : Time Series on MxNet, Intuitive Linear Algebra, NLP attacks

• Gluon Times series : Probabilistic time series modelling in Python using AWS MxNet.
• Intuitive Guide to linear algebra : Nice and simple introduction to linear algebra.
• TexFooler :A Model for Natural Language Attack on Text Classification and Inference. Repo contains example of BERT and ESIM attack. 

[Links of the Day] 23/04/2020 : Machine Learning technical debt, Python and Bayesian Deep Learning perspective of generalization

• Bayesian Deep Learning and a Probabilistic Perspective of Generalization : the authors propose and demonstrate that ensembles provide an effective mechanism for approximate Bayesian marginalization, and propose a related approach that further improves the predictive distribution by marginalizing within basins of attraction, without significant overhead.
• The High Interest Credit Card of Technical Debt for Machine learning : machine learning solutions are not simple, and technical debt is expensive to pay back.
• Machine Learning in Python : Main developments and technology trends in data science, machine learning, and artificial intelligence centred around python.

[Links of the Day] 21/04/2020 : Machine Learning for Relational Query processing, Augmenting Language model with latent knowledge retriever, Computer Vision Recipies

• Extending Relational Query Processing with ML Inference : the authors present advanced cross-optimizations between ML and DB operators in Raven DB. The authors demonstrate significant performance improvement, up to 5.5x from the native integration of ML in SQL Server, and up to 24x from cross-optimizations.
• REALM: Retrieval-Augmented Language Model Pre-Training : the authors propose to leverage Retrieval-Augmented Language Model pre-training for the challenging task of Open-domain Question Answering.By using augmenting their model with latent knowledge retriever they are able to beat current SOTA models while limiting the model growth size.
• Computer Vision Recipies : Microsoft is releasing a lot of really good content, this time it's for computer vision. In this repository, you will find best practices, code samples, and documentation for Computer Vision.

[Links of the Day] The future of Machine Learning is DBMS, Fun Exploring Explanations, High performance Regex

• Hyperscan : high-performance multiple regex matching library
• Cloudy with a chance of DBMS : A. Colier reviews the 10 year ML prediction paper. the TL;DR: Model, model everywhere in enterprise databases. You have already seen of a glimpse of what that means with Big-query ML
• Explorables : awesome website explaining a lot of concepts through play. Lot of computer science stuff in there. 

[Links of the Day] 14/04/2020 : Time series dynamical attractors Autoencoder , Binarized Neural Network framework, Machine learning and Databases

• Deep learning of dynamical attractors from time series measurements : the authors propose a general embedding technique for time series, consisting of an autoencoder trained with a novel latent-space loss function. Worth giving it a look if you deal with time series.
• larq : open-source Python library for training neural networks with extremely low-precision weights and activations, such as Binarized Neural Networks. Basically, this framework is aiming at embedded / FPGA / ASIC machine learning models deployment. A fantastic resource and great model zoo on top of that.
• Cloudy with a chance of DBMS : Databases are going to embedded more and more machine learning solution. Big query from Google already does that. But it's just a question of time for most mainstream DB to offer ML service.

[Links of the Day] 02/04/2020 : Grep all, HealthCare mobile data collection for machine learning, FastAI framework

• ripgrep : grep search in PDFs, E-Books, Office documents, zip, tar.gz, etc.
• pymedserver : a server framework for mobile data collection and machine learning in healthcare
• fastai : fantastic machine learning library trying to abstract away a lot of PyTorch into simple API and building blocks. Sometimes it attracts a bit too much, especially with you want to get murky with some details. But all in all, fastAI is really a framework you want to look at if you are doing machine learning.