There is an interesting trend within the storage ecosystem. We are witnessing a polarization of the offer. On one side, we are seeing the rise of high performance rack scale solution(DSSD, NVMe over fabric solution, etc..) . And on the other side we have the object storage solution which are more datacenter scale. While both leverage heavily non volatile memory they play a different different role within the ecosystem.
The rack scale storage target “very” high performance solution, delivering very low latency high bandwidth access time. Often in the 100 of usec or less. However often these solution come at a higher financial cost due to more expensive hardware (custom NVM), network fabric (IB+NVMe, PCIe + NVMe, Omnipath +NVMe, Pure PCIe, etc..) and significant power consumption (
>2000W/5U for DSSD). Finally, these offer access via a specialized API that needs to be either accessed natively or adapted to other more standard one.
On the other side we have the object storage solution. Users access object storage through applications that will typically use a REST API. This makes object storage ideal for all online, Cloud environments. Moreover they tend to be a lot more cost efficient especially with the rise of
Ethernet connected drives (up to 50% less TCO).
Stuck in the middle is the classic Filer / POSIX compliant solution that seems to slowly dwindle away. To a certain extent the rack scale solution should have a bright future in the niche (but still significant) market for enterprise that still consider that their application requires custom for what they think is a custom problem. On the other side the object storage is gaining momentum by riding the unstoppable cloud tide.
While, both technology can and should co-evolve, they both suffer from
software limitation and
to a certain extend hardware one as bandwidth and latency get dangerously close to what cpu are capable to handle. This requires a drastic shift on how applications are developed if user want to actually get any benefit from these solution. However, few company are willing to risk specializing their code using an API that can become obsolete when the next generation of storage solution pops up.
Storage startup/company out there start to discover that API is playing a significant role success of their product and performance while still important will lose of its importance. It will either make rewriting applications to access your storage infinitely easier task or transform it into a painful experience by forcing to go through hoops and/or adaptation layers with the performance cost associated.
The fight for the next generation storage API only started. There is and will be more push toward standardization which will be fueled by the customer tiredness with every revolving siloed point solution. People who use object storage one want it to behave more like POSIX storage, but they also want to keep the storage costs at an object level and improve the performance. On the other hand people using rack scale storage want to retain the performance but increase its simplicity and also want the price to come down. It is going to be extremely hard to deliver both but hopefully we might finally see a rationalization of the storage market as having an object storage system that allows byte range access is very appealing.
