Reflections Of The Void: netflix

Showing posts with label netflix. Show all posts

Thursday, November 28, 2019

Mantis : netflix metric stream processing framework. This seems to be a really powerful platform for realtime monitoring and especially contextual alerting and alerting on logs. What's impressive is the throughput it can churn through events... [github]
SOPS'19 : SOPS papers are in free open access, a lot of papers on security and formal verification of distributed this year. [Murat review Day0, Day1, Day2]
Tikv : Distributed transactional key-value database. [Github]

GitPitch : markdown powered presentation tool. I'm lazy and I feel that powerpoint/google doc does the trick for me. However, if you are willing to go past the learning curve this seems to be a great tool! Hey, I still prefer to write my document in Latex.. So who am I to judge :) [github]
k3s : Kubernetes but lightweight. 5 times less to be precise ( hence the 3 = 8 - 5 )
Polynote : polyglot notebook by netflix. I really like the concept of notebook IDE as a service. This is a must for any organisation that heavily rely on data science to optimise their operations. [github]

Powerline-go : a nice low latency PowerShell written in go. Give it a try.
Kubernetes Context Switcher : another practical tool, allowing you to seamlessly switch between kubernetes context.
Notebook @ netflix : Notebooks is now the default tool for data scientists. And Netflix shows how they are able to scale this tools to accommodate their ever-increasing data crunching needs.

Build Fault Tolerant Cross-Region AWS VPC : how Rackspace deploy fault tolerant solution on top of AWS multi region using VPC
10 Lessons from 10 Years of AWS : words of wisdom from Werner Vogel , CTO of AWS
#NetflixEverywhere Global Architecture : Qcon presentation from Netflix director of Operations Josh Evans, interesting bit is the focus on data replication cross data center ( or availability zones in this case). It seems pretty obvious that Netflix went the right way with dealing with scaling the resiliency of their product : start with the primitive then the data not the other way around. If the data is not available or consistent their is always a chance to fallback at a cost. While if the services are down, having the data available won't help.

1/3 of internet Traffic : How Netflix Direct architecture was designed to route a third of the world internet traffic.
How to tell if a FLOSS project is doomed to FAIL : quick check list to get an indication if an opensource project is doomed on not.
Iterating Grace : satire about a Silicon Valley programmer named Koons Crooks, who has a spiritual awakening that leads him on a pilgrimage to a volcano in Bolivia. Living in a yurt, he spends his days pondering the transcendent wisdom of tweets by top venture capitalists before apparently being trampled to death by vicuñas. Just read it.

360 view of Netflix architecture : most of it i already published, however it is really nice to have all the different part in one place and more importantly the links to all the open source goodness they leverage or contribute.
Tensor flow explained : Jeff Dean, Google Large-Scale Deep Learning for Intelligent Computer Systems explain tensor flow (just open sourced)
Backblaze Storage pods : what hardware it takes to deliver $0.005/GB/month for a S3 like service.

Today's links 05/05/2015: First Aid Git,Intel cache allocation tech, FIDO automatic security analysis.

First Aid Git : searchable collection of the most frequently asked git questions.
Intel Cache Allocation Technology :provides a way for the Software (OS/VMM) to restrict cache allocation to a defined 'subset' of cache which may be overlapping with other 'subsets'. When multi-threaded applications run concurrently, they compete for shared resources including L3 cache. At times, this L3 cache resource contention may result in inefficient space utilization. For example a higher priority thread may end up with lesser L3 cache resource or a cache sensitive app may not get optimal cache occupancy thereby degrading the performance. CAT kernel patch helps provides a framework for sharing L3 cache so that users can allocate the resource according to set requirements.
Fido : system for automatically analyzing security events and responding to security incidents.

Today's links 8/11/2014: stream processing, documentation, #machinelearning , #bigdata

Write the docs : a place where the art and science of documentation can be practiced and appreciated
Mantis : Netflix's Event Stream Processing System
ACID Stream Processing : transactional stream processing system that supports full ACID properties without compromising scalability and high throughput
Machine learning :

when machine learning goes bad : when using ML to do preemptive maintenance end up being counter productive and really expensive..
large scale deep learning : lecture deck from Google fame Jeff Dean

Today's links 29/09/2014 : #mesos , cloud, performance , optimization , virtual cpu, root cause analysis

Mesoscon 2014 : Video from mesosphere conference 2014
From Clouds to Roots: Performance Analysis at Netflix : how Neflix does root cause performance analysis [slides]
Virtual CPU : How to write your own Virtual CPU in C++
Bytes tricks : cool optimization trick for counting bytes