I Finally got the green light from SAP to open source the project I started a little bit more than a year ago. What is Hecatonchire about ?
The cloud computing paradigm is fundamentally changing the expectations for how computing, storage, and networking resources get consumed. As these resources become fungible rather than self-managed, end users and service developers expect resources to be available on demand in response to their real time requirements. However, existing technologies to facilitate cloud infrastructures are inadequate, expensive and does not scale. Over time, server-centric infrastructure management systems have evolved to become a powerful but complex mesh of layered system functionalities that are both knowledge and labour intensive to setup, maintain, and most importantly, adapt to a constantly changing set of customer requirements.
As a result, renting a ﬁxed combination of cloud resources does not and cannot fulﬁl the demands of cloud users for multiple reasons. One one hand, as the number of cores and amount of memory on servers continues to increase (over the next few years, we are expect to see servers with hundreds of cores and hundreds of gigabytes of memory per server commonly used), leasing an entire server equivalent may be too large for many customer’s needs with resources wasted. On the other hand, with the emergence of a broad class of high-end applications for analytics, data-mining, etc., the available amounts of memory and compute power on a single server may be insufﬁcient. Moreover, leasing cloud infrastructure resources in a ﬁxed combination of CPU, memory, etc. is only efﬁcient when the customer load requirements are both known in advance and remains constant over time. As neither of these conditions are met for a majority of customers, the ability to dynamically mix-and-match different amounts of compute, memory, and I/O ressources is highly valuable.
Hecatonchire aim to address these gaps by developing key enabling technologies that allow virtualization to go beyond the boundaries of the underlying hardware, breaking the barrier of using a single physical server for a virtual machine. With Hecatonchire, compute, memory, and I/O resources will be provisioned across multiple hosts and can also be consumed in dynamically changing quantities instead of in ﬁxed bundles. This will effectively enable a ﬂuid transformation of cloud infrastructures from commodity sized physical nodes to very large virtual machines to meet the rapidly growing demand for cloud-based services.
What can you do with Hecatonchire so far:
- Application Memory extension : With Hecatonchire you can extend the reach of an applications to remote memory. effectively delivering a scale-out solutions for the applications by leveraging LibHeca APIs.
- Virtual Machine Memory pooling : we have a modified KVM version that supply reliable memory scale-out solutions that do not require changes in existing applications. Vm can simply tap into remote memory and also tier it ( to remote ram , remote compressed ram or flash, we also support memory deduplication)
- Flash Live migration : Consistent and high performance post copy live migration regardless of the workload and memory footprint ( we tested with TB+ while running in memory db under load)
- Flash Cloning : instantaneously clones a VM into multiple replicas running on different hosts.
- Applications and Virtual Machine Shared Memory : Posix shared memory but across multiple node
- Memory Cloud : Complete integration of all the feature within one cloud platform (openstack integration). We aim eliminates memory segmentation by creating a pool of shared memory for the entire cluster. Hecatonchire will combines memory from dedicated appliances with memory from existing servers to create a large memory cache which can be transparently shared among any or all servers or data centers in a cluster. It provides a shared network memory resource large enough to accommodate huge data sets, signiﬁcantly reducing storage access and improving processing performance by orders of magnitude. Basically you will be able to manipulate memory the same way you manipulate storage (I.e. : Tiering RAM ->Remote Ram-> Compressed Ram->Flash , Deduplication , etc..)
- Compute Cloud: enable the aggregation of CPU resources across multiple COTS nodes providing VMs with a large number of vCPUs
- I/O Cloud: break the boundaries of the hypervisor, off-loading the I/O virtualization logic and aggregating all the I/O resources distributed across multiple servers to a dedicated server.
I will discuss in subsequent posts the different features , performance and roadmap of Hecatonchire. Stay Tuned !