event-reduce : accelerate query result after write. Basically if cache part of the write and recalculate the new query result using past query result and the recent write event. The authors observe an up to 12 times faster displaying of new query results after a write occurred.
The Next Decade in AI : Paper by Gary Marcus where he explores the possible future of AI over the next decade
1 on 1 meeting questions : a collection of 1:1 questions, great list that can help any manager pick the right question for the right context. As long as you are able to read the room/ team/ person.
Trax: advanced google deep learning library built on top of JAX. It is actively used by the DeepMind team and aiming code clear while providing advanced models like Reformer.
On Drafting an Engineering Strategy : short post describing how to draft an engineering strategy when you start in a new role and new need to layout the next 10-18 month roadmap
Contrastive Self-Supervised Learning : Contrastive self-supervised learning techniques is an interesting class of methods that build representations by learning to encode what makes two things similar or different.
TensorFlow Deployment : A collection of tensorflow use case and infrastructure associated deployment patterns. Each example comes with code and often a ready to run docker image.
Aethos : an interesting project where the authors designed a library/platform to automates data science and analytical tasks at any stage in the pipeline. They offer a uniform API for automating analytical techniques from various libraries such as pandas, sci-kit learn, gensim, etc. It's still work in progress but has some potential.
A Failed SaaS Postmortem : an interesting postmortem of a failed SaaS. The TL;DR: too much focus on tech, not enough on customers.
MIT Deep Learning : 2019 lecture on deep learning, started this January.
KafkaHQ : Nice Kafka GUI for topics, topics data, consumers group, schema registry, connect and more.. If you want an alternative to it you can also check out Pulsar [pulsar] [pulsar dashboard]
Resilience Engineering : really interesting collection of people and associated papers on the topic of resiliency and more specifically applied to the domain of engineering. A good amount of really important information to digest especially for the folks working in SRE.
MLFlow : Machine learning model registry, think docker hub but for ML .. Kind of like TensorFlow hub but you can [github]. But wait, that's not just it! ML flow is a platform that also offers ML project management and tracking. Worth a check.
Performance and Power Evaluation of AI Accelerators for Training Deep Learning Models : as Deep Learning users seek to squeeze that extra 5% accuracy from their model. They turn to hardware accelerator in order to reduce cost and shorten the training time by order of magnitude. However, not all accelerators are created equals as the authors demonstrate in this paper.
C4 model : software modelization architecture framework. Help you simplify and clearly describe your software stack by breaking it down into 4 distinct layers: system, containers, components, and code.
The Dynamics of Venture Capital Relationships : Study of VC network of relationship. The authors found that having a deeper relationship leads to fewer, not more future co-investments. Moreover, deeper relationships lead to lower exit performance, even after controlling for endogeneity. Interestingly, deeper relationships first lead to lower performance and subsequently lead to a slowdown in the relationship intensity. Relationship effects are more negative for VC firms with less central network positions, and for deals made in “hot” investment markets.
Cerebras : a new technique for normalizing hidden activations on neural networks. This allows researchers to greatly accelerated their training sequence without the need to be a Google or AWS with dedicated accelerators. [presentation] [git]
Bit-swap : lossless data compression technique using deep learning.
Scalable mobility support in future internet architectures : MIT Master Thesis by Xavier K. Mwangi where he argues for a move away from host centric (IP) toward a data-centric approach where the naming and routing scheme revolves around object name resolution architecture. The argument behind this is to eliminate the tight coupling between location and data and allow a more fluid interaction, especially in an ever increasing mobile world.
Deep Learning: A Critical Appraisal : Critical analysis of the recent deep learning revival and argumentation by the authors that deep learning will be soon replaced by other technique as we want to progress toward artificial intelligence.
Delayed impact of fair machine learning : A paper that tries to answer the central question of fairness of machine learning algorithm. I.e. how to ensure fair treatment across demographic groups in a population when we let a machine learning system decide who gets an opportunity (e.g. is offered a loan) and who doesn’t.
EdgeML : Microsoft research demonstrate how to push machine learning at the edge and run KB models. We could quickly see machine learning enabled IoT device popping around us. [slides] [github]
Efficient Methods and Hardware for Deep Learning : Quest for speed never stop, and often that means getting read of those pesky indirection layers that make your software architecture so flexible :)
Pure Operation-Based Replicated Data Types : CRDT for operations rather than just value. But the core concepts are a little bit tricky and there is some potential pitfall in the approach. Such as the performance limitation and the reliance on causal stability ( which is really hard to obtain in pure decentralised systems)
A Survey on Homomorphic Encryption Schemes : this paper presents a thorough survey of the state of homomorphic encryption schemes. Homomorphic encryption allows manipulation of the encrypted data without the need to decrypt it. This will allow when hardware will be fast enough to deal with the complexity of the operations, to have a true secure distributed multitenant database. As no operation on the hosting side will require clear text decryption of the data and everything can be done securely on the client side.
Efficient Methods and Hardware for Deep Learning : Standford lecture where guest lecturer Song Han present algorithms and specialized hardware (FPGA, GPU, ASIC, etc..) that can be used to accelerate training and inference of deep learning workloads. [video]
Toward Automatic Turing Test : when software is used to detect if its a software or a real person talking, feel like it should be submitted to totally, not robot subreddit. The problem is way more complex and useful than it seems. By automating the procedure you could do fast prototyping and testing of models with limit human input. Accelerating the research and reducing costs.
Survey of Deep Reinforcement Learning : l cover central algorithms in deep reinforcement learning, including the deep Q-network, trust region policy optimisation, and asynchronous advantage actor-critic.
Unix - History and Timeline : The history of Linux's grandfather OS , surprisingly enough the latest version 3 spec and ISO/IEC spec came out in 2003. Which is only 14 years ago ( I feel old now.... )
One Model To Learn Them All : the authors propose a model that is good enough for most needs, kind of a jack of all trades/master of none model for deep learning. This can be really practical for experimenting and probing dataset for potential use. Or if you do not have the time to spend to create the ideal model. However, there is always the risk to end up having a sub-performing solution.
Pay-with-a-Selfie : interesting payment model where the authors propose to use group selfie to extend the split bank note metaphor for executing financial transactions.
Pedis : Redis Compatible NoSQL datastore using the Seastar Framework. It's interesting to see that on the single thread benchmark Redis and Pedis are on par while it Redis gets smoked on 8 thread benchmark. However on a side note, the author should probably have chosen another name for project.
serverless : Serverless Framework with serverless architectures using AWS Lambda, Azure Functions, Google CloudFunctions [github]
Microsoft's Production Configurable Cloud : built in custom nic + fpga for highly configurable and dynamic network stack in Microsoft DC. The work is really impressive. It demonstrate how pervasive FPGA and customization hardware will be in future datacenter.
Open Pilot : open source driving agent providing Adaptive Cruise Control (ACC) and Lane Keeping Assist System (LKAS) for Hondas and Acuras. This is a really interesting solution and I wonder how fast other company will start to leverage or opensource their own solution in order to accelerate adoption. However without extremely strong verification and proof system ( formal method ) it will be extremely hard ( and illegal probably ) to deploy such software at this stage.
Nuts and Bolts of Building Deep Learning : Andrew Ng reiterated at NIPS2016 that there is no secret AI equation that will let you escape your machine learning woes. All you need is some rigor. [video]
Continuous Delivery Maturity Model : look at the different level of maturity for continuous integration-delivery-build-.... in software development
LipNet : deep learning for full sentence lip reading. One step closer to a fully fledged HAL.
Dmake : tool to manage micro-service based applications. It allows to easily build, run, test and deploy an entire application or one of its micro-services.
Assessing Threat of Adversarial Examples on Deep Neural Networks : machine learning is the next frontier for hacker. And because of its inherent opacity it requires special capabilities to secure system that relies on this underlying technology. This paper show that for text driven classification, adversarial exemple are more an academic curiosity than a real threat. However, we need to see if this can be applied to other type of classification.
Lesson learns about scaling Uber : Many talk are about scaling, however most company and startup would love to have those problems. Often its not about scaling, its about having the right product market fit. Then you can enjoy the roller coaster of scaling problems.
Quiet : TCP over sound . This is really cool, it allows to pass data through speakers on android devices.
Amazon Web Services Open Guide : This is THE practical guide for anybody that use AWS. Really well constructed and easy to use guide for the majority of AWS services out there.
Uncertainty in Deep Learning : Thesis looking into the probabilistic aspect of Bayesian network and how not everything is black and white in deep learning land. It seems that we need to see an emergence of probabilistic programmation + deep learning hybrid in order to handle the new world of uncertainty that is opening up with the progress of AI research. [PhD Thesis]
Google Interview University : program to help you beat the google technical interview process. While this is excellent as a refresh course in basic computer science I would also recommend MIT course : hacking a google interview. When interview by google i almost got a carbon copy of the question in the MIT course.. Sadly the google interview process is, lets say, "abysmal" for senior people. They want you to go through this marathon of interview without providing you an idea of what you are applying or other informations..
OpenBR : open source bio-metric framework, I can't wait for the first community driven mass recognition system to come out. No more secrets...
Inside the Magic Pocket : really good case study and architecture behind the storage system design to replace AWS S3 after Dropbox moved out of AWS [HN discussion]
Comparative Study of Deep Learning Software Frameworks : version 3 of the extensive study of deep learning framework. What is interesting is while tensor flow is deemed extremely versatile it seriously lag behind the other framework performance wise.
