[Links of the Day] 27/02/2020 : Finance Datasets, OS build using WASM and IPFS, Team Wiki

• Quandl : financial, economic, and alternative datasets for financial professionals. A treasure trove of information, sadly not all free.
• RedShirt : Operating system build using WASM that you can run off straight from IPFS ... Next thing you know we will have a blockchain-based package management system for an end to end trustworthy OS... Well as long as you have access to the internet.
• Outline : wiki and knowledgebase for growing teams using the markdown language. You can run it straight of docker.

[Links of the Day] 16/09/2019 : Proof of History Blockchain, ElasticSearch cluster operation Library, using ML to extract scholar document info

• Solana : Proof of History based blockchain system.  [website]
• Vulcanizer : Go library for interacting with an Elasticsearch cluster. It is not meant to be a full-fledged Elasticsearch client. Its goal is to provide a high-level API to help with common tasks that are associated with operating an Elasticsearch cluster such as querying health status of the cluster, migrating data off of nodes, updating cluster settings, and more. [github]
• Grobid : machine learning software for extracting information from scholarly documents.

[Links of the Day] 16/04/2019 : L3 VPN gateway, NSA reverse engineering toolkit, Proof of History based block-chain

• Vita : high-performance L3 VPN gateway you can use to interconnect your networks.
• Ghidra : the first real competitor to IDA Pro, the extremely expensive and often pirated state-of-the-art software for reverse engineering.
• Solana : bold claim from the authors as they claim to support 710,000 tx/s with off-the-shelf hardware and no sharding. This seems to be enabled by leveraging proof od history mechanism rather than the traditional proof of work that bitcoin and al. use.

[Links of the Day] 03/05/2018 : Fundamental Values of Cryptocurrencies, Kafka SQL streaming engine, SaaS pricing

• Fundamental Values of Cryptocurrencies and Blockchain Technology : paper looking at the fundamental values of cryptocurrency. The authors propose fundamental models and framework to study the price of digital assets. It's an interesting approach, however, they limited their analysis to only two cryptocurrencies: Bitcoin and Ethereum. While there are loads of other cryptocurrencies out there that either derives from these two or are independent. By solely focusing on these two, there is a feeling that the authors are missing the bigger picture and especially the transient aspect of the cryptocurrency market. 
• KSQL : this project offers a streaming SQL engine for Kafka. It basically allows you to use the Kafka stream engine with the common SQL model.
• SaaS Pricing : a real-life example of why the right SaaS pricing model is SOOO.. important. It can make or break your business. And you rarely have more than one shot at it. 

[Links of the Day] 16/11/2017 : Sparse and dense array database, Rythm of memory, Routing over blockchain

• TileDB : manages massive dense and sparse multi-dimensional array data simply. This is a really good project as often there is no real support in existing database. 
• Rhythm of memory : the brain is a complex organ. And we just barely scratched the surface. Scientist discovered that part of the memory processing in the brain is segregated in different subcomponent that process information in parallel and at different speeds. This gives a glimpse of how the brain works and how it is able to store and access so much data at the various level of granularity fast.
• IPvPub : I really think there is something behind this concept. While I tend to be wary of the current trend of sprinkling blockchain everywhere. Using this technology for large-scale address resolution and routing can solve so many problems... Reduce reliance on DNS system in the age of lambda. What I really want to see is this integrated with a Lambda framework for simple exposure of service endpoints.

[Links of the Day] 27/06/2017 : Blockchain trust & authentication for IoT, K8s patterns, Ripple cryptocurrency Network analysis

• Kubernetes Production Patterns and anti-patterns :  a lot of common sense, actually a lot of the patterns and anti-patterns can be applied to the other environments. But still a good refresher. 
• Blockchain based trust & authentication for decentralized sensor networks : using blockchain to solve the trust issue in a swarm of IoT on a network. The critical bit missing is the power requirement for running all the crypto operations.
• Large-Scale Analysis of the Ripple Cryptocurrency Network : an overview of the paper analysing the Ripple p2p blockchain based money transaction network. Turns out it suffer the same issue that "old school" p2p network. Take out the highly connected nodes and you can bring down / split the network. Nothing new, but still a good read and reminder that small network tends to be resilient to attack. But, if their resiliency diminishes with the increased reliance on a small number of highly connected nodes.

[Links of the Day] 06/06/2017 : Secure Machine Learning, Quantum secured blockchain and Survey of Machine Learning in Hardware

• DeepSecure : a framework that enables scalable execution of the state-of-the-art Deep Learning models in a privacy-preserving setting. The authors propose a system that enables data owner and model owner to maintain segregation of information while allowing them to work together without data leak between the two parties. 
• Quantum-secured blockchain : The authors propose in this paper a quantum blockchain architecture specifically designed to solve the post-quantum computer cryptographic weakness of currently used crypto algorithms in Bitcoin and other blockchain frameworks. However, it seems that they conveniently ignore newer cryptographic solutions that are "quantum resistant".
• Survey of Neuromorphic Computing and Neural Networks in Hardware : heterogeneous hardware solutions are becoming the norm as classic CPU are not able to handle the bandwidth and processing power. Seriously, how a Intel or AMD CPU can process 1 Tb/S of bandwidth ... Anyway, as machine learning is reaching peak hype, the hardware that comes to accelerate it is getting more mainstream and diverse. This paper provides a good overview of the various technic and hardware used in the field. Moreover, it references an exhaustive collection of papers of the field.

[Links of the Day] 26/04/2017 : Aphyr Scala Day, Sia blockchain file storage , How brains are built

• Aphyr Scala Day 17 : Aphyr breaks database for a living and then talks about it :) 
• Sia : a Blockchain-based marketplace for file storage, the really attractive thing is the cost comparison of SIA vs public cloud system. Which is between a tenth to a hundredth time cheaper than S3 or other similar solution. I would be curious to see the performance thought.
• How brains are built: High-level overview of principles of computational neuroscience.

[Links of the Day] 10/04/2017 : Loopy , Distributed execution engine and Yet another distributed ledger algo

• Loopy : Fantastic tools for explaining and describing complex system interaction. It's easy to use and even easier to get the message across [github]
• Ray : experimental distributed execution engine replicating code across multiple workers. Written in python it leverages object store and distributed task execution to achieve parallelism. I really wonder if it wouldn't have been better to code Ray using AWS lambda and S3. 
• Algorand : yet another distributed ledger. The approach proposes to eliminate the segregation of actors in the ledger system. No more miners and users, everybody is an equal participant. Moreover, it relies on a new for of Byzantine agreement ( need a TLA+ proof to really feel comfortable with that) and cryptographic selection algorithm for selecting leader (verifier) of the ledger process execution. This is a rather interesting paper and I will try to produce a short summary of it if I find the time.

[Links of the Day] 07/04/2017 : TensorFlow Example, Systems in the microseconds era and blockchain distributed direct democracy

• Naked Tensor : Google tensor flow bare-bone example. 
• Attack of the Killer Microseconds : Hardware ( especially storage ) is entering the micro or sub-micro second era. This has far and wide ranging implication. And system designer needs to rethink the existing stack that was designed for the millisecond era. It looks like we are entering an era where the software stack is not the bottlenneck.
• Cicada : Distributed secure proof of work blockchain combined with a privacy guaranteed ID system. The creator of cicada aim at enabling distributed direct democracy and decentralised application platform. This is worthy goals, however, the creator forgot that direct democracy tend to fail as the majority of the population is not interested or knowledgeable enough in the problem they will be asked to vote on. 

[Links of the Day] 08/03/2016 : Intel blockchain, Fast17 conference and papers, AWS cloud formation devops tool

After a small hiatus, here is the return of the links of the day.

• Sawtooth Lake: Intel distributed ledger system. It uses an interesting security mechanism to deliver secure consensus. Sadly it relies on Intel proprietary hardware encryption modules to deliver this feature.
• Fast17: File and Storage technology Usenix conference happened last month. There were a couple of interesting papers but one picked my interest: Redundancy Does Not Imply Fault Tolerance:Analysis of Distributed Storage Reactions toSingle Errors and Corruptions. The authors look at single file system fault impact on Redis, ZooKeeper, Cassandra, Kafka, RethinkDB, MongoDB, LogCabin, and CockroachDB. Turns out most systems are not able to handle these type of faults very well. It seems that a single node persistency layer error can have an adversarial ripple effect as distributed system seems to have put way to much trust in the reliability of this layer. Sadly they lack tools for recovering from errors or corruption emerging from file systems.
• Stacker : remind 101 tools for creating and updating AWS formation stacks. Looks like an interesting alternative to terraform. 

[Links of the day] 22/07/2016 : Blockchain distributed storage, Docker use cases, Run Go in unikernel

• BlockStack : a distributed storage system that uses blockchains on Bitcoin to securely define a global name space. The name’s are all tied to values which representing URI(URL)s to storage systems like AWS S3 but could be any cloud storage service.
• Use Case track : Videos from the Use Case track at @DockerCon 2016
• Atmanos : this is really cool, this project enable you to compile go code and run it as a unikernel on top of Xen

Links of the day 06/01/2016: art of ware, data analytic platform, #blockchain in 2015 #fintech

• Qminer : data analytics platform for processing large-scale real-time streams containing structured and unstructured data. Really cool if you want to provide real time classification or sentiment analysis solution within your product. [github]
• The art of ware : great reinterpretation of Sun Tzu classic for creating and marketing IT products. A classic and must read.
• Blockchain 2015: Slidedeck presenting and anlyse of the blockchain in the Finabcial service landscape

Bitcoin: the blockchain ecosystem lubricant

As mentioned in a previous post, Bitcoin is a rather attractive solution for self contained market ecosystem. What is really attractive and useful is the underlying blockchain technology. Recently, banking institution and other financial institutions have started looking into the possibility to deploy blockchain technology to implement a distributed ledger system. However, these institutions might want to create a fully private self contained blockchain system, probably without any virtual currency functionality.

Private blockchain

Financials institution have an history of seeking privacy and control over the tools and mechanism they rely on. As a result, they will have the natural tendency to try to deploy their own private blockchain platform. Essentially, instead of having a fully public and uncontrolled network and state machine secured by cryptoeconomics (eg. proof of work, proof of stake), they seek the creation of a system where access permissions are more tightly controlled, with rights to modify or even read the blockchain state restricted to a few users, while still maintaining many kinds of partial guarantees of authenticity and decentralization that blockchains provide. The obvious objective for a consortium or company running a private blockchain is to be able to change the rules of a blockchain, revert transactions, modify balances, etc.

Private does not implies trust

The parties to a transaction have to be 100% confident that nobody can change the transaction. However, since you can change the rules at will there's a risk of manipulation of the system. The potential collusion, and lack of transparency for a system that requires an arbitration system to guarantee fair execution should be concerning for potential customer. Naturally, there is many incentive to have a private systems such that the need to protect the secrecy of transaction. But, you can still deliver such feature by relying on hybrid system where you have dark transaction pool sanctioned by the public infrastructure. Moreover, without a virtual currency to provide an incentive for uninterested third party to participate in the system, accountability remains difficult to ascertain.

Blockchain system lubricant : Bitcoin 

Public blockchain system is tied to a virtual currency, which reinforce its trust-ability while allowing an incentive in selfish participation. The principle is that blockchain consumer recompense participant in a form of tit for tat approach, allowing a more heterogeneous ecosystem preventing the dominance of a single type of participant as well as peer pressure to keep the system trust-able. Without it, the value of the currency will simply vanish and as a result remuneration from participation.

This heterogeneity prevent collusion (think LIBOR scandal) and make the system more robust and accountable. This is where bitcoin is really needed and shine, not for common everyday operations.

Links of the day 21/10/2015: #blockchain consensus protocol, Awesome #aws resources, Killing patents

• Awesome AWS : A curated list of awesome AWS libraries, open source repos, guides, blogs, and other resources with a popularity rating system built in :) 
• Blockchain Consensus Protocols : a little bit of blockchain beyond fintech (but still a lot of fintech in this slidedeck). 
• So You Want To Murder a Software Patent: talk by Jason Scott on how to deal with annoying patents

Links of the day 23 - 01 - 2015

Today's links 23/01/2015: #cloud storage - Open vStorage, #bitcoin #blockchain scalability, shortest path in corporate communication

• Open vStorage :  its roadmap and some interesting bits on its architecture.
• Blockchain scalability : A look at the stumbling blocks to blockchain scalability and some high-level technical solutions.
• Surprising Facts About Shortest Paths : In a corporate communication network  the shortest path is not the fastest ( Social Network Analysis ). In other words, don’t let your train pass through the central hub for a shortcut, ’cause it’s going to stay there for a long long time.

Links of the day 16 - 01 - 2015

Today's links 16/01/2015: TCP variants, Probability books, #blockchain technology

• Newer TCP Implementations : a good overview of the various TCP implementation , their tradeoff and usage. 
• 15 books to learn Probability : there is a non zero chance that you will learn probability stuff with these books.
• Understanding the blockchain : a very good overview of the reach and impact of the blockchain technology.

Links of the day 18 - 12 - 2014

Today's links 18/12/2014: #devops infra provisioning model, #blockchain Distributed apps stack, anonymous group communication

• 7 Layer DIP : 7 layer osi like model that decompose the DevOps Infrastructure Provisioning environment. 
• Eris : blockchain based Distributed Application Software Stack that aim at enabling design, test, and deployment of distributed applications which are as flexible, user-friendly, and legally compliant as they are secure. 
• Dissent : accountable anonymous group communication delivering prractical anonymous group communication system offering strong, provable security guarantees with reasonable efficiency

Links of the day 10 - 11 - 2014

Today's links 10/11/2014: cluster CI , DHT, Multiqueues, blockchain ssh key pki

• Cluster Runner: fast easy test feedback for your continuous integration system. Its always a challenge to retain speed and agility as your test and team growth. Cluster Runner helps solve that. 
• DHT routing table maintenance: performance improvements of the DHT by the bitorrent crowd
• MultiQueues : multiple sequential priority queues that outperform previous more complicated data structures
• emcssh : blockchain based secure, decentralized management of PKI.

Links of the day 23 - 10 - 2014

Today's links 23/10/2014: fast TCP socket, HFT , cuckoo filter and  sidechain for Bitcoin blockchain

• Fastsocket : highly scalable socket and underlying networking implementation of Linux kernel rewrite. Nice performance and results, however its going to be hard to push within the upstream kernel. Might take a while if it ever happen.
• 5th Annual Modeling High Frequency Data in Finance Conference : last year HFT conference slides
• Cuckoo Filter - Practically Better Than Bloom : Performance wise, at the same level of space efficiency, insert speeds are better than a standard Bloom filter when the hash is mostly empty (low load), but significantly worse than a Bloom filter at high load when the hash gets filled and many 'evictions' are necessary for each insert. But if a tradeoff for space is acceptable (to stay at ~50% load), the Cuckoo Hash inserts much faster than its Bloom counterpart. Lookup speeds for existing keys are 2-3x the speed of a standard Bloom filter, and about 1.5x than the "Blocked" Bloom filter variant. Negative lookups (key not present) are about 2x faster than Bloom at high load, but about 3/4 the speed at low load. ( full summary on HN ) 
• Pegged Sidechains : Sidechains in theory could address lots of the scalability issues of Bitcoin and could accelerate the roll out of new features