- FastHTTP : like net/http but up to 10x faster . Zero memory allocation on a fast path. When you really really need to go fast and the standard golang lib is not enough.
- Virtio-fs : shared file system build on top of the virtio framework. Finally a good way of sharing files and data between the host and the guest VM or between guests [git].
- Asylo : I recently came across the concept of enclave applications. An enclave is a special execution context where code can run protected from even the OS kernel, with the guarantee that even a user running with root privileges cannot extract the enclave’s secrets or compromise its integrity. It relies on Intel SGX or ARM trustzone security hardware feature to protect the runtime operation of the process. This might be the next security level for highly regulated industries that want to deploy in a public cloud environment which will completely eliminate a lot of the conundrum that they currently face.
A blog about life, Engineering, Business, Research, and everything else (especially everything else)
Showing posts with label file system. Show all posts
Showing posts with label file system. Show all posts
Thursday, February 14, 2019
[Links of the Day] 14/02/2019: golang fast HTTP , virtio shared file-system, enclave application
Labels:
file system
,
kvm
,
links of the day
,
security
,
virtio
Friday, March 10, 2017
[Links of the Day] 10/03/2017 : User-space SysFS, Key Value consensus Algo, Cost efficient Big Data Serverless Framework
- ProcStat : Userspace equivalent of kernel SysFS. Really cool project by my friend Sasha. makes it really easy to expose internal counter and state of a process via FUSE
- Bizur : Key-value Consensus Algorithm using a nice solution where consensus is achieved on the key themselves rather than relying on a globally distributed log. The great aspect is that recovery and failure management is greatly simplified and streamlined. However, it implies that the progress and consensus on each key are independent of each other. As a result, you cannot rely on serialisation of state between key. Which can be limiting if you expect the state of Key A to be changed after the State of Key B by example.
- PyWren : Framework that let you use serverless functions for cheap large-scale data analysis. [github]
Labels:
analytic
,
bigdata
,
consensus
,
file system
,
key/value store
,
links of the day
,
serverless
Tuesday, June 14, 2016
[Links of the day] 14/06/2016: Freeze Frame File System, Strata+Hadoop16 Keynotes, Linky French Smart meter
- Freeze Frame File System : file system specifically designed to handle real time data stream processing applications. Basically it is able to handle streams of updates while supporting "temporal reads" on demand. To do so the system keep all update history in a memory-mapped log, cache recently retrieved data for repeat reads, and use a hybrid of a real-time and a logical clock to respond to read requests in a manner that is both temporally precise and causally consistent.
- Strata + Hadoop 2016 : All keynote like if you where there.
- Linky : This is a fantastic analysis (in french) of the electric LINKY smart meter that is being deployed across France. It dispel many of the FUD surrounding smart meter but also ask interesting question regarding upgradability , security and who really benefit from these devices. It seems that the customers won't be able to interact or benefit form the data generated by these device, which partially defeat the purpose of installing one. But it's not like they have a choice as the deployment of smart meters is written in french law.
Labels:
conference
,
file system
,
keynotes
,
links of the day
,
smart meter
Friday, February 26, 2016
[Links of the day] 26/02/2016 : Usenix Fast 16 , FPGA liberouter and Event delivery at spotify
- Fast 2016: all of Usenix Fast 2016 goodness available in one place.Interesting to see that we start to see the emergence of storage systems optimized for time series [BTrDB] . Also note the always interesting report on failure rate (this time for flash)
- Liberouter : really cool project using FPGA to deliver hardware acceleration of network security and monitoring tools.
- Event Delivery at spotify : part 1 of a series of blog post on event monitoring and management system used at spotify.
Labels:
conference
,
file system
,
flash
,
fpga
,
links of the day
,
network
,
storage
,
time series
Wednesday, February 24, 2016
[Links of they day] 24/02/2016: cluster file system, Disks, death and datacenters
- Disks for Data Centers : google research paper on the place of "legacy" storage system in current and future datacenter. Its all about the use case and RoI.
- How To Kill A Supercomputer : Dirty Power, Cosmic Rays, and Bad Solder
- BeeGFS : parallel cluster file system formerly known as FhGFS is going full open source. An alternative to Lustre , however I wonder how it will really fair with Ceph improvement
Labels:
cluster
,
data center
,
disk
,
failure
,
file system
,
links of the day
,
storage
Thursday, February 04, 2016
[Links of the day] 04/02/2016 : Hierarchy are bad for human cooperation, Software defined memory and D&D alignement
- Hierarchy is Detrimental for Human Cooperation : really good experimental study demonstrating something that a lot of people intuitively know. However, I have a slight concern regarding the interpretation of their finding. While they pretty much nailed it (see below), I suspect that a more accurate title would have been inequalitarian hierarchy is detrimental. If an organisation is able to deliver similar payoff throughout the layer of the hierarchy the detriment will inherently disappear. However, I have yet to see any organisation achieving this feat!
"""We have shown that achieving cooperation among humans is more difficult when there is an underlying hierarchical structure producing different ranks between people and therefore unequal payoffs for the participants. This result is driven by insufficient contributions from lower ranked individuals who cannot be confident that they will benefit from cooperating."""
- Plexistor SDM : Plexistor Software-Defined Memory (please tell my why every single product out there has to be has to use a hype compliant description....) concept is a generic filesystem for storage, from NVDIMM to NVMe to SSD. They aim to offer a new POSIX filesystem that any application can use rather than trying to modify applications. In short they offer an under the hood tiering mechanism between the different underlying persistent layer.
- Chaotic or Evil? : applying D&D characters alignement system to your project, team and teammate. Next hiring interview: ask all past characters sheet & compare with team (ps: never trust a rogue)
Labels:
dungeons & dragons
,
file system
,
hierarchy
,
links of the day
,
memory
,
nvm
,
organisation
,
posix
Subscribe to:
Posts
(
Atom
)