Skip to main content
ARES

Project Coordinator (EU) :

Algolysis Ltd

Country of the EU Coordinator :

Cyprus

Organisation Type :

SME

Project participants :

Algolysis LTD

Dr. Nicolas Nicolaou is the co-founder and a senior scientist and algorithms engineer at Algolysis. Dr. Nicolaou will participate in the project utilizing his long expertise on the formalization and analysis of Distributed Algorithms for Atomic R/W Storage Systems.

Dr. Efstathios Stavrakis [Algolysis LTD] is a co-founder and a senior scientist and algorithms engineer at Algolysis Ltd and has been conducting research in academia and the industry for 15 years. His research is in the areas of computer graphics, animation, virtual reality and games.

Dr. Theophanis Hadjistasi [Algolysis LTD] is conducting research in Fault-tolerant Parallel and Distributed Computing, with emphasis on Distributed Atomic Storage Implementations. His research interests span “Theory” and “Practice” with a focus on Algorithms and Complexity.

Ms Andria Trigeorgi [Algolysis LTD], PhD candidate in Computer Science at the University of Cyprus and researcher at Algolysis since the beginning of the project.

Penn State University

Dr. Viveck Cadambe [PSU], is an expert on distributed algorithms and erasure coding, and has previously participated in development of linearizable shared memory emulation algorithms via erasure coding, as well as provably consistent non-blocking reconfigurable algorithms.


Dr. Bhuvan Urgaonkar [PSU], brings expertise in systems software, distributed computing, and performance modeling. He has made contributions to resource management in clouds, power/cost management of DCs and storage systems.

 

 

Starting date :

ARES: A Next-Generation, Erasure Coded, Shared Distributed Storage System Experiment


Experiment description

Distributed Storage Systems (DSS) store large amounts of data in an affordable manner. Cloud vendors deploy hundreds to thousands of commodity machines, networked together to act as a single giant storage system. Yet, component failures, and network delays are the norm, thus ensuring consistent data-access and availability at the same time is a challenging task. Vendors often solve availability by replicating data across multiple servers. However, keeping these copies consistent, especially when they can be accessed concurrently by different operations, is very difficult and costly. The problem of keeping copies consistent becomes even more challenging when failed servers need to be replaced or new servers are added, without interrupting the service. Any type of service interruption in a heavily used DSS usually translates to immense revenue loss.

Commercial DSS avoid providing strong consistency guarantees as they are considered costly and difficult to implement in an asynchronous, fail prone, message passing environment. Indeed, initial implementations of Atomic DSS had high demands in communication, storage, and sometimes computation. Recent works however, investing in algorithms that may reduce the overheads on the aforementioned parameters. In particular, there exist algorithms that reduce the communication cost by trading (cheaper) computation resources, and other algorithms that use erasure coded techniques to reduce the communication overhead as well as the storage demands in the replica hosts, trading however the number of faults tolerated.

ARES goes one step further and attempts to harvest the efficiency of the proposed algorithms by incorporating an adaptive approach that allows algorithm switching based on the application needs. ARES offers strong consistency guarantees (atomicity), providing the illusion that data are accessed sequentially when in reality multiple processes may read and write the same data object concurrently. In addition, ARES provides the capability of dynamically changing the membership of the replica host, enabling the service to stay alive even in the presence of server failures or server replacement. Atomicity is the most intuitive semantic, which if implemented correctly and efficiently, may relief developers from a major headache of writing complex code or communication protocols and will provide a transparent shared memory service alleviating low level synchronization tasks for distributed applications. By utilizing communication lightweight algorithms via its adaptive nature, together with erasure coding techniques, ARES promises a communication and storage efficient service, which may attract the attention of potential stakeholders.

Successful completion of this project will provide clear indications for the possibility of deploying a global-scale shared memory (storage) space, introducing the new concept of Memory-as-a-Service (MaaS). This will open new horizons to the development of distributed, scalable, and high-performance applications, not just over dedicated hardware but also over commodity devices. It will also put the participating organizations, as well as Europe and the USA at the front of a new and innovative technology, open for further exploitation and commercialization. Our experiments may also expose weaknesses of the methodologies, leading the researchers to the drawing board for the identification of bottlenecks and the design of necessary optimizations.

Implementation plan :

N/A

Impacts :

Impact 1: Enhanced EU – US cooperation in Next Generation Internet, including policy cooperation.
The main impact of this collaboration is to facilitate the use of various networking testbeds between the EU and the US. This project will enable researchers at Penn State University (USA), to collaborate closely with the Algolysis LTD (Cyprus, EU) team, and develop solutions that will be tested in a cross-
Atlantic setups. This teamwork may lay the foundation of next generation international network protocols and further strengthen the relationships of EU and US partners towards future collaboration.
Impact 2: Reinforced collaboration and increased synergies between the Next Generation Internet and the Tomorrow's Internet programmes.
Dissemination of the results of this project will help to generate new interest in distributed shared memory storage services, with strong potential of creating new collaborations and thus, devising new market products that will serve as MaaS for cloud applications. Through the diverse connections of the partners, this project will reach a very wide network of established professionals spanning over the EU and the North America, and would reinforce the chances of collaboration and result in increased synergies between the Next-Generation Internet and Tomorrow's Internet programmes.

Impact 3: Developing interoperable solutions and joint demonstrators, contributions to standards.
The collaboration between the EU and US partners will yield an ADSS that will operate across borders, demonstrating a new protocol for robust, and consistent data sharing. At its heart, ADSS algorithms (like ARES) promote interoperability, as they expose a simple application interface making them attractive for a large set of Cloud Applications and emerging technologies (i.e., IoT, VR, AR etc.). The implementation and deployment of ARES will be one of the few ADSS implementations that exist at the current stage. Exploitation of the implementation as a minimum viable product will consist of the first practical implementation of a strongly consistent shared storage service to date. Therefore, promising outcomes from our experiments may trigger the composition of new data sharing standards, across countries, applications, and digital assets.

Impact 4: An EU - US ecosystem of top researchers, hi-tech start-ups / SMEs and Internet-related communities collaborating on the evolution of the Internet
Successful completion of these experiments will help the participating organizations to pioneer towards the creation of robust shared distributed storage systems that will power the next generation of cloud applications. The collaboration with PSU will be integral for Algolysis (an SME in the South-eastern Med area of the EU), allowing the company’s researchers to interact, collaborate, and exchange knowledge with top researchers and engineers in the field. This will give a competitive advantage to the company which will establish strong collaborative bonds with researchers from one of the leading academic institutions in the world, and will gain access to state-of-the-art lab equipment to supplement the experiments of the project. On the other hand, this collaboration will enable PSU to get access to the industrial ecosystem in Europe, and benefit from potential future exploitation of the developed technology. Moreover, the researchers will benefit from the generation of scientific manuscripts, which will bring the experimental outcomes to the attention of the scientific community.

 

Results :

Upon successful experimental evaluation, we expect the analytical results of ARES to match its theoretical findings, thus ARES being efficient and practically adoptable. In particular, we expect to meet the following Key Performance Indicators (KPI):

 

KPI# Measure Target
KPI-1 Scalability 1: number of processes to access a single shared object concurrently

 
 Allow more than 250 (read/write) concurrent processes in the service
KPI-2 Scalability 2: examine performance of read/write operations when modifying the # of replica hosts
 
Measure performance in seconds and expect linear increment as the # of replica hosts grows.
KPI-3 Stress-Test 1: number of read operations completed in a second (throughput) for objects under 1MB and under different concurrency and congestion loads
 
Strict concurrency to affect negatively the performance of the algorithm while more  stochastic (more realistic) loads will per form better.
KPI-4 Stress Test 2: time to take a read/write to complete under different object sizes
 
Keep operations latency close to the expected delay of transferring an object of a predefined size over a dedicated connection bandwidth.
KPI-5 Fault Tolerance 1: time that takes for the service to reconfigure on replica failure or removal Measured in seconds and targeted to be linear with respect to the number of concurrent reconfigurations. Service to experience no interruptions.
KPI-6 Performance Comparison 1: compare the algorithm performance with other ADSS We expect that ARES may have additional  performance overhead over simpler solutions that do not support dynamicity or use centralized control.

 

Future Plan :

Not available yet

Expected TRL at experiment completition :

4

NGI related Topic :

Open Internet Architecture and Renovation

Call Reference :

4

The 30-months project NGIatlantic.eu will push the Next Generation Internet a step further by providing cascade funding to EU-based researchers and innovators in carrying out Next Generation Internet related experiments in collaboration with US research teams.




contact action add button