- Introduction to Apache Kafka : If you lived under a rock and don't know about kafka.
- Kafka at Scale : Interesting overview of running Kafka at Linkedin : 1100+ Kafka brokers - 675 Billion messages per day - 150 Terabytes In - 580 Terabytes Out - Peak Load – 10.5 Million messages/sec – 18.5 Gigabits/sec Inbound – 70.5 Gigabits/sec Outbound
- Fluent : videos of the 2015 Fluent Conference
- Vault : tool for securely managing secrets and encrypting data in-transit by HashiCorp
A blog about life, Engineering, Business, Research, and everything else (especially everything else)
Thursday, April 30, 2015
Links of the day 30 - 04 - 2015
Today's links 30/04/2015: Apache conf - kafka - more kafka - Fluent Conference,Vault keeping secret in the cloud
Labels:
cloud
,
conference
,
kafka
,
links of the day
,
secret
Wednesday, April 29, 2015
Links of the day 29 - 04 - 2015
Today's links 29/04/2015 : Concurrency vs parallelism, Reenix #Rust OS, Pulsar Real Time Analytics, Business Town
- Concurrency is not parallelism : very good talk by Rob Pike on the misconception that often occur when people hear the word concurrency they often think of parallelism, a related but quite distinct concept. In programming, concurrency is the composition of independently executing processes, while parallelism is the simultaneous execution of (possibly related) computations. Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once.
- Reenix : unix-like operating system kernel in rust. I like seeing this type of Rust project, it show the vitality of the language and its versatility. [github]
- Pulsar : Ebay Open-source real-time analytics platform and stream processing framework. Some nice characteristic for real time large scale CEP solution [github]
- Business Town : when real life cliche meets cartoonist .. long live captain moonshot !
Labels:
analytic
,
bigdata
,
cartoon
,
concurrency
,
ebay
,
links of the day
,
os
,
parallelism
,
rust
Tuesday, April 28, 2015
Links of the day 28 - 04 - 2015
Today's links 28/04/2015: #Rump Kernel Stack, Disque Distributed In Memory MQ, #FusionIO new PCIe product, Power level estimation of VM systems
- Ramp Stack : Nginx, MySQL, and PHP built on Rump Kernels without rearchitecting the application. Most of the work requires the app to be cross compile correctly (Nginx & MySQL). This implies that Unikernel-compatible unmodified POSIX C and C++ applications “just work” on top of Rump Kernels, provided that they can be cross- compiled.
- Disque :a distributed, in memory, message broker by Redis folk. Not production ready but a promising start.
- Pcie Flash : fusion IO is still kicking and deliver an interesting solution: up to 350,000 I/O operations per second (IOPS) on random reads and 385,000 IOPS on random writes (on the 3.2 TB model) with a 15k nanosecond write latency and 2.8 GB/sec of read bandwidth. .. However I still don't get why they don't want to use NVMe tech
- Process-level Power Estimation in VM-based Systems : the authors describe a fine-grained monitoring middleware providing real-time and accurate power estimation of software processes running at any level of virtualization in a system.
Labels:
distributed system
,
flash
,
fusionio
,
links of the day
,
message queue
,
pcie
,
power
,
rump kernel
,
virtualization
Monday, April 27, 2015
Links of the day 27 - 04 - 2015
Today's links 27/04/2015: #Postgresql on #FPGA and with #Kafka , #NVM impact on storage and Database
- Bottled Watter : Real time integration of Postgresql and Kafka.
- How Will Emerging Memory Technologies Impact Computing and Storage Architectures : looks at existing architectures and how they might adapt to the unique qualities of emerging memory technologies.
- Let’s Talk About Storage & Recovery Methods forNon-Volatile Memory Database Systems : look at how many "legacy" DBMS components are unnecessary and will degrade the performance of data intensive applications with the advance of NVM technology use.
- PostgreSQL on FPGA : interesting project trying to deliver basic infrastructure in order to help PostgreSQL developers have their "own" FPGA hacking projects.
Labels:
database
,
kafka
,
links of the day
,
nvm
,
storage
Thursday, April 23, 2015
Links of the day 24 - 04 - 2015
Today's links 24/04/2015: iOS fail, Memory Snooping Protection in HW, NVMe/PCIe fabric for storage, High performance server application framework
- Back to the Future With C++ and Seastar : Meetup notes on Seastar, open source server application framework written in C++ that presents a future/promise based API with 7 million requests per second served on a single machine. [slides]
- No iOS zone: PoC of attack : within WiFi hotspot range iOS devices are rendered unstable / unusable by constant reboots.
- Cloud security reaches silicon : Hardware implementation of method for thwarting memory snooping or inference across VM attacks by disguising memory-access patterns
- NVMe/PCIe as a Storage Interface : an very good overview of the market, future solution and form factors.
Labels:
apple
,
framework
,
High Performance
,
ios
,
links of the day
,
memory
,
nvme
,
pcie
,
security
Links of the day 23 - 04 - 2015
Today's links 23/04/2015: HPC MQ, Ethernet Connected Drives, MongoDB consistency issue, Memory Errors
- CoralMQ : full-fledged, ultra-low-latency, high-reliability, software-based middle ware for the development of distributed systems based on asynchronous messages. Developed for high speed trading and FX applications. Its always interesting to see that a lot of high frequency trading apps are written in Java while the (often flawed) perception is that it should be avoid for HPC solution.
- Ethernet Connected Drives : I really think that this type of storage is going to be the next generation of jack of all trade cloud storage solution. Massive potential scale out potential with better simplicity and maintainability ( not to mention lower TCO) [slides][Q&A]
- MongoDB stale reads : well not all good news if you really want to have consistent transaction.. MongoDB can return old / stall read value and stripe heavily relies on it is even more puzzling. This all scream run to me..
- Memory Errors in Modern Systems : with more memory in system ( and exascale HPC) more errors will occur and with the current state of things probably be silently ignored.. The risk is to what extend these error will damage the system and how to detect and mitigate them.
Labels:
error
,
ethernet
,
links of the day
,
memory
,
message queue
,
storage
Wednesday, April 22, 2015
Links of the day 22 - 04 - 2015
Today's links 22/04/2015: #bitcoin alternative, Nested OS , #Intel MXP, State of Probabilistic Programming
- Stellar : Stanford professor David Mazières thinks he has a faster, more flexible, and more secure alternative to bitcoin. The new cryptocurrency protocol, called SCP, is being adopted by a nonprofit called Stellar to replace a Bitcoin-inspired system meant to make financial services cheaper and more widely accessible. [white paper]
- Nested Kernel Operating System Architecture : old idea coming back ( ring protection) [paper][presentation]
- Intel MXP : Intel's MPX extension offers much of the old segment-level protection, but is opt-in for software and needs tooling support (compiler, linker, loader, etc). This is being worked on, e.g., Gcc started adding support for it.
- The State of Probabilistic Programming : good overview of probabilistic programming and where its at ( if you can see past the Deep learning craze)
Labels:
bitcoin
,
intel
,
links of the day
,
os
,
probabilistic
,
security
Subscribe to:
Posts
(
Atom
)



.jpg)

