Monday, November 07, 2016

[Links of the Day] 07/11/2016 : Baidu Open Source Repo(s), Wan Replicated DB

  • Baidu : Baidu open source code on Github. It looks like it replicate a lot of service / feature that other hyperscale system use. Raft seems to be the default underlying consensus protocol for all applications. A lot of nice goodies in there, especially: 
    • BFS : Baidu file system that provide the underlying persistence for Baidu real time application. Its a distributed multi datacenter using raft for metadata coherence and use a shared nothing approach for linear scalability. 
    • Tera : Distributed database 
    • Galaxy : mesos / kubernetes equivalent. 
    • Paddle : Distributed machine learning 
    • iNexus : Distributed K/V store . Looks similar to consul and it also use raft as the underlying consensus protocol
  • Bedrock : Wan replicated distributed data (base). Designed to use SSD and other nice features.