- 100 open source Big Data architecture papers : all in the title
- Two very good talk from the Paper we love meetup :
- Raft : introduces the Raft consensus algorithm, which is an attempt to make a consensus algorithm that is easy to understand and easy to implement.
- Making Lockless Synchronization Fast : overview of common safe memory reclamation mechanisms and then explores their performance implications.
A blog about life, Engineering, Business, Research, and everything else (especially everything else)
Friday, June 26, 2015
Links of the day 26 - 06 - 2015
Today's links 26/06/2015: Lockless synchronization, #bigdata architecture papers, Raft consensus
Labels:
architecture
,
bigdata
,
consensus
,
links of the day
,
raft
Thursday, June 25, 2015
Links of the day 25 - 06 - 2015
Today's links 25/06/2015: Java 8 Micro service actors, Device testing framework, #Docker security cheat sheet
- NTS: cloud-based automation framework that lets you remote control most Netflix Ready Devices.
- Kontraktor : (Micro-)Service oriented, distributed Actors designed for Java 8
- Docker Security Cheat Sheet : container are as secure as the duct tape around it :)
Labels:
docker
,
famrework
,
java 8
,
links of the day
,
security
Monday, June 22, 2015
Links of the day 22 - 05 - 2015
Today's links 22/05/2015: Steal key via radio, Turing x86 mov, Beyond #Nash equilibrium, #Ethernet jungle
- Steal PC info with radio : Tempest on steroid and on the cheap!
- x86 Mov is turing complete: but page fault are too! Just not that efficient.
- Going beyond nash equilibrium in online auction : New way to approach auction system and why nash equilibrium doesn't always apply in the ads auction ecosystem.
- The jungle of fast ethernet : many protocols cohabiting under the same umbrella. Its bound to get crowded and awkward at some point.
Labels:
ethernet
,
Nash equilibrium
,
security
,
turing
,
x86
Friday, June 19, 2015
Links of the day 19 - 06 - 2015
Today's links 19/06/2015: Google network, Replication with RDMA, Triton #container
- Google Network : TL:DR version without the "OMG we are so great tone" SDN relying on an underlying clos switch topology. [Video]
- DRBD9 : now with RDMA added .. Next DPDK? We are moving away slowly from the all kernel solution for low latency and a userspace oriented ecosystem.
- Triton : Joyent triton container tech. Interestingly enough they offer CPU bursting and RAM pooling for extra oomph when needed ( and if available)
Labels:
container
,
drbd
,
google
,
joyent
,
links of the day
,
network
,
rdma
,
replication
Thursday, June 18, 2015
Links of the day 18 - 05 - 2015
Links of the day 18/05/2015: NVM biblio , Author of Pegasus and Heracles PhD dissertation, Container for #apple #OSX
- Pegasus and Heracles : PhD defense of David Lo presenting his result in data-center orchestration system. He developed the Pegasus and Heracles platform for Google which resulted in significant power consumption reduction as well as resources usage efficiency increase. My guess is that Google is probably running between 13x and 15x more efficient than classic data-center and the gap is growing. I m not really sure that the whole container craze is actually improving the efficiency side as fragmentation often mean more complex and difficult orchestration. Ha almost forgot the PhD defense slides are here
- NVMDB : comprehensive survey of the (at last count) 340 non-volatile memory technology papers published between 2000 and 2014 in International Solid-State Circuits Conference (ISSCC), Symposia on VLSI Technology and Circuits (VLSI Technology, VLSI Circuits), and International Electron Devices Meeting (IEDM). The resulting data set provides a clear picture of how these memory technologies have evolved over time. [ Online Biblio ]
- xhyve : Mac fanatic rejoice, you thought you avoided the container craze so far. But fear no more here comes the lightweight virtualization solution for OS X in all its glory. CoreOS is already talking about supporting it
Labels:
apple
,
container
,
datacenter
,
google
,
links of the day
,
load balancing
,
nvm
,
orchestration
Tuesday, June 16, 2015
Links of the day 16 - 06 - 2015
Today's links 16/06/2015: memory addressable #NVM software, failure at large scale
- FlashMap : Unified Address Translation for Memory-Mapped SSDs
- Memkind : library is a user extensible heap manager built on top of jemalloc which enables control of memory characteristics and a partitioning of the heap between kinds of memory.
- Memory errors at scale revisited : we do not live in a Bayesian world, with scale rare small problem become common enough that they need to be dealt with however sometimes we cannot even predict them before its too late.
Monday, June 15, 2015
Links of the day 15 - 06 - 2015
Today's links 15/06/2015: #NVMe history, #Docker Load balancer, real time distributed OLAP
- NVM Express : History and driver evolution [ Slide deck ]
- Flaxton : Load Balance Docker containers and child servers using Linux ipTables, without dealing with container ip
- Pinot : Linkedin realtime distributed OLAP datastore deliver scalable real time analytics with low latency. It can ingest data from offline data sources (such as Hadoop and flat files) as well as online sources (such as Kafka).
Friday, June 12, 2015
Links of the day 12 - 06 - 2015
Today's links 12/06/2015: #google Heracles, #Nginx architecture, #Qemu security breach, sins of product development
- Heracles : how Google achieve its SLO while improving TCO by 15 to 300%
- NGINX architecture: very good overview of what make nginx run fast
- Qemu heap overflow : if you use PCnet device with qemu your guest can escape and play around with the host.
- 11 deadly sins of product development : could easily add a few more.. But it could be resumed in : lost track of customer, lost track of budget, lost track of ego.
Labels:
architecture
,
development
,
Distributed systems
,
google
,
links of the day
,
nginx
,
product
,
security
,
sins
Wednesday, June 10, 2015
Links of the day 10 - 06 - 2015
Today's links 10/06/2015: Open source routing, OSDC conference and fujistu tech forum, Chrome HTTP server
- Open source routing : nice comparison of existing open source routing solution.
- OSDC 2015 : Slides and video of the 2015 open source datacenter conference.
- Fujitsu North America Technology Forum 2015 : IoT everywhere :)
- Chrome Http Server : http server on google chrome.
Labels:
chrome
,
conference
,
http
,
links of the day
,
networking
Tuesday, June 09, 2015
Links of the day 09 - 06 - 2015
Today's links 09/06/2015 : Local algorithm for highly scalable architecture, Artic time series Database on top of #MongoDB, KVM Forum
- Local algorithms : Very good survey of algorithms with the very interesting property that they runs in constant time, independently of the size of the network. Which makes then extremely attractive for large scale distributed systems.
- Artic : a high performance datastore for numeric data on top of mongoDB. It supports Pandas, numpy arrays and pickled objects out-of-the-box, with plug-gable support for other data types and optional versioning. They have wild claim in regard performance and scalability but until we get a proper time series benchmark out there it still hard to compare time series solutions.
- KVM forum 2015 : Schedule of this year KVM forum. Interesting Bits: Real time stuff , some live migration with DPDK and running 1k guests per host.
Labels:
database
,
kvm
,
links of the day
,
local algorithms
,
qemu
,
time series
Monday, June 08, 2015
Links of the day 08 - 06 - 2015
Today's links 08/06/2015: #SSD no panacea for VMs, Hardware X86 #security breach, #Bigdata stream engine
- SSD no panacea with VM : because of is operational model SSD tend to have poor performance in virtual environment when shared across multiple VM. The authors suggest to isolate and dedicate SSD for each VM in order to benefit from the speed benefit of SSD. Micro storage solution anyone ?
- X86 design flaw : Apparently there is a way to escalate from ring 0 to the high privileged state. This can be really interesting if this would enable VM or container to escape and gain access to the bare-metal system. : :
- Heron : the descendant of Storm by Twitter. The architecture seems nice and try to address some of Storm shortcoming.
Labels:
bigdata
,
distributed system
,
links of the day
,
security
,
SSD
,
storage
,
Stream Processing
,
virtualization
,
vm
,
x86
Subscribe to:
Posts
(
Atom
)