[Links of the Day] 08/03/2016 : Intel blockchain, Fast17 conference and papers, AWS cloud formation devops tool

After a small hiatus, here is the return of the links of the day.

• Sawtooth Lake: Intel distributed ledger system. It uses an interesting security mechanism to deliver secure consensus. Sadly it relies on Intel proprietary hardware encryption modules to deliver this feature.
• Fast17: File and Storage technology Usenix conference happened last month. There were a couple of interesting papers but one picked my interest: Redundancy Does Not Imply Fault Tolerance:Analysis of Distributed Storage Reactions toSingle Errors and Corruptions. The authors look at single file system fault impact on Redis, ZooKeeper, Cassandra, Kafka, RethinkDB, MongoDB, LogCabin, and CockroachDB. Turns out most systems are not able to handle these type of faults very well. It seems that a single node persistency layer error can have an adversarial ripple effect as distributed system seems to have put way to much trust in the reliability of this layer. Sadly they lack tools for recovering from errors or corruption emerging from file systems.
• Stacker : remind 101 tools for creating and updating AWS formation stacks. Looks like an interesting alternative to terraform. 

Links of the day 03/11/2015 : Automation memory lapse danger, disaggregating disk and complex data-sets visualisation

• Automation memory lapse danger : forgetting how to do things after automating everything is one of the major risk. Relearning , making the same mistake again when you need to upgrade / change or worse fix the system after a major breakdown becomes a major challenge.
• Dis-aggregating disk : well its all about trade off but slowly the classic storage paradigm is moving away from the current models. However it will take some times until all the different pieces fall into place enabling seamless dis-aggregation. In the mean time the idea is slowly percolating throughout the community.
• Mirador : tools for visual exploration of complex datasets

Links of the day 18 - 11 - 2014

Today's links 18/11/2014: #microservices, #cloud , oracle CPU, automated deployment, #RDMA, Intel Omni-path

• Colossus : a lightweight framework for building high-performance applications in Scala that require non-blocking network I/O. In particular Colossus is focused on low-latency stateless microservices by tumlbr
• M7: Next Generation Oracle Processor [ slides ] 
• Appolo : shared internal deployment service at Amazon and its public pendant AWS code deploy 
• RDMA in the cloud : Vmware demonstrate RDMA for virtual machine in cloud env but as usual with vmware presentation the graph scale are poorly chosen... 
• Omnipath : Intel is gunning at infiniband with its new 100 Gbps fabric.

Links of the day 06 - 10 - 2014

Today's links 06/10/2014: storage and docker, automation , and human interaction.

• Storage Scalability in Docker : a look into how choice of storage affects scalability, container start up time, stability, supportability 
• Boxen : automate deployment of development environment ( for Mac OSx mainly) - By the github crowed.
• Talking to Human :  ( you can download the book for free here)

Amdahl's law and automation

The theory (and a little bit of practice)

Automation in datacenter and now utility computing is heavily used to drive down TCO cost. However it is rather hard to find out what to automate in order to get the maximum benefit out of it. Some automation that seems obvious often has a low return on investment.

Hopefully, we can use ( or abuse) amdahl's law here. It is used to find the maximum expected improvement to an overall system when only part of the system is improved. Interpreted simply, Amdahl's Law says focus on improving the things that make the biggest difference overall.







Pic: Amdahl's law


If we adapt this law for TCO reduction 1 is the cost of running the system (datacenter, service etc.. ) for a discreet amount of time. The new cost is will be the length of cost the unimproved fraction takes, (which is 1 - P), plus the cost the improved fraction takes. The cost for the improved part of the system is the cost of the automated part's former cost divided by the automation cost factor, making cost of the improved part P/S. The final cost is computed by dividing the old running time by the new running time, which is what the above formula does.

Applied to datcenters, cloud or IT operation, this logic suggests that organizations should start with automation that makes the biggest impact, particularly IT staff productivity.

The reality

The reality


However, if we look at the reality the OPEX cost for a server and a datecenter is represent a very small part of the overall cost. According to google paper it varies between 7% and 9 % of the overall cost. Which means that if we still follow Amdahl law automation can provide only a very limited impact on the overall cost while maximising server utilisation guaranty a better return on investment ( not to mention being smart with hardware acquisition).



But there is no small economy.