In the past decade, with the increasing adoption of Internet as theprimary means of electronic interaction and communication, web-based
datacenters have become a central requirement for providing online
services. Today, several applications and services have been deployed
in such datacenters in a variety of environments including e-commerce,
medical informatics, genomics, etc. Most of these applications and
services share significant state information that are critical for the
efficient functioning of the datacenter. However, existing mechanisms
for sharing the state information are both inefficient in terms of
performance and scalability, and non-resilient to loaded conditions in
the datacenter. In addition, existing mechanisms do not take complete
advantage of the features of emerging technologies which are gaining
momentum in current datacenters.
This dissertation presents an efficient soft state sharing substrate
that leverages the features of emerging technologies such as
high-speed networks, Intel's I/OAT and multicore architectures to
address the limitations mentioned above. Specifically, the
dissertation targets three important aspects: (i) designing efficient
state sharing components using the features of emerging technologies,
(ii) understanding the interactions between the proposed components
and (iii) analyzing the impact of the proposed components and their
interactions with datacenter applications and services in terms of
performance, scalability and resiliency.
Our evaluations with the soft state sharing substrate not only show an
order of magnitude performance improvement over traditional
implementations but also demonstrate the resiliency to loaded
conditions in the datacenter. Evaluations with several datacenter
applications also suggest that the substrate is scalable and has a
low-overhead. The proposed substrate is portable across multiple
modern interconnects such as InfiniBand, iWARP-capable networks like
10-Gigabit Ethernet both in LAN and WAN environments. In addition, our
designs provide advanced capabilities such as one-sided
communication, asynchronous memory copy operations, etc., even on
systems without high-speed networks and I/OAT. Thus, our proposed
designs, optimizations and evaluations demonstrate that the substrate
is quite promising in tackling the state sharing issues with current
and next-generation datacenters.