DISTRIBUTED COMPUTING SEMINAR


(TELE-SEMINAR)

Erasure-Codes for Optimal Storage Cost in Atomic Memory Emulation

Kishori Mohan Konwar
MIT

11:00 am
September 15, 2017 (Friday)
239 CSL


Abstract: Distributed Storage Systems (DSS)  that store massive data  
sets across several hundreds of servers are increasingly being used  
for both industrial and scientific applications, ranging from  
sequencing genomic data to those used for e-commerce. Many  
applications demand concurrent and consistent access to the stored  
data by multiple writers and readers. The most desirable form of  
consistency is atomicity, which in simple terms, gives the users of  
the data service the impression that the various concurrent read and  
write operations take place sequentially. Implementations of atomicity  
on an asynchronous system under message passing framework, in the  
presence of failures, is often challenging. Traditional  
implementations use replication of data as the mechanism of  
fault-tolerance; however, they suffer from the problem of having high  
storage cost, and communication costs for read and write operations.  
For example, to store a file of size 1 TB across a 100 server system,  
the ABD algorithm replicates the value in all the 100 servers, which  
blows up the worst-case storage cost to 100 TB. Additionally, every  
write or read operation has a worst-case communication cost of 100 TB.  
  Erasure codes provide an efficient way to decrease storage and  
communication cost in atomicity implementations. Erasure codes are  
increasingly being studied in the context of implementing atomic  
memory objects in large-scale asynchronous distributed storage  
systems. When compared with the traditional replication based schemes,  
erasure codes have the potential of significantly lowering storage and  
communication costs while simultaneously guaranteeing the desired  
resiliency levels. In this talk, I will begin with a short overview of  
the existing algorithms and systems, followed by some recently  
developed algorithms.