Reflections Of The Void: cluster

Showing posts with label cluster. Show all posts

Thursday, October 17, 2019

kubernix : Single dependency, single node Kubernetes clusters for local testing, experimenting and development. This project uses NIX as it's underlying functional package manager.
mirrorgate : this looks like a better version of Hygiea from capital one. From a quick glance at the code, it seems easier to deploy and extend. I think I will give it a spin and see what we can achieve with it. I strongly feel that wallboard application are essential for effective SRE/Devops and provide direct feedback to the teams while contextualizing it with business process output. What's probably missing from this version is the "executive" wallboard that Hygiea offers.
jtc : select one or multiple elements from a source JSON and apply various actions on the selected elements at once. Another tool for your swiss army knife bash script.

Efficient Queue Management for Cluster Scheduling : MS researcher look into introducing queue management techniques, such as appropriate queue sizing, prioritization of task execution via queue reordering, starvation freedom, and careful placement of tasks to queues for big data task cluster scheduling.
Keystone : a indie gogo project to refactor LLVM to build a multi-architecture, multi-platform, open source assembler framework.
Erasure Coding in Windows Azure Storage : MS azure use Local Reconstruction Codes (LRC) for its storage. LRC greatly reduces the number of erasure coding fragments needed for reconstruction in case of failure/ offline data. The key benefit is a drastic reduction in I/O and bandwidth requirement for repairs while maintaining the storage overhead low.

Disks for Data Centers : google research paper on the place of "legacy" storage system in current and future datacenter. Its all about the use case and RoI.
How To Kill A Supercomputer : Dirty Power, Cosmic Rays, and Bad Solder
BeeGFS : parallel cluster file system formerly known as FhGFS is going full open source. An alternative to Lustre , however I wonder how it will really fair with Ceph improvement

Today's links 21/04/2015: #NVM architecture, #HPC storage fabric, Cluster sqlite, Just in time #Unikernel Quic protocol

Architecture-level Designs using Emerging Non-volatile Memories : describe architecture techniques necessary for building efficient and reliable systems with NVMs.
Jitsu : Just-In-Time Summoning of Unikernels using a Xen toolstack that satisfies the demands of secure multi-tenant isolation on resource-constrained embedded ARM devices.
Fortissimo : A3CUBE Fortissimo Foundation, computing/storage converged solution and data engine, provides a very fast and scalable platform for storing and retrieving data items with 100s of gigabytes/s of throughput and Mega IOPS. But as always hte catch is that it requires a proprietary fabric to do so which limits its adoption [Architecture] [Benchmark]
rqlite : Cluster of sqlite using Raft protocol, really cool implementation. I wonder if it could be used as a for of fast ACID compliant distributed cache for SQL DB [github]
quic : Google pushing for their udp based protocol as IETF standard

Today's links 10/04/2015: out of core #Bitcoin transaction, cluster mgmt @google, DC memory & storage

Lightning Networks : Summary of the 4 part blog post by Rusty Russell. The concept is really interesting however it also require a "soft fork" of bitcoin. And that won't be easy.
Cluster Management at Google : John Wilkes shares lessons learned managing clusters at the scale of Google.
Datacenter memory and storage: very good overview of server primary system memory and secondary storage technologies spanning the past, present, and future.