TY - GEN
T1 - Coupling prefix caching and collective downloads for remote dataset access
AU - Ma, Xiaosong
AU - Freeh, Vincent W.
AU - Yang, Tao
AU - Vazhkudai, Sudharshan S.
AU - Simon, Tyler A.
AU - Scott, Stephen L.
PY - 2006
Y1 - 2006
N2 - Scientific datasets are typically archived at mass storage systems or data centers close to supercomputers/instruments. End-users of these datasets, however, usually perform parts of their workflows at their local computers. In such cases, client-side caching can offer significant gains by reducing the cost of wide-area data movement.Scientific data caches, however, traditionally cache entire data-sets, which may not be necessary. In this paper, we propose a novel combination of prefix caching and collective download. Prefix caching allows the bootstrapping of dataset downloads by caching only a prefix of the dataset, while collective download facilitates efficient parallel patching of the missing suffix from an external data source. To estimate the optimal prefix size, we further present an analytical model that considers both the initial download over-head and the downloading speed. We implemented our proposed approach in the FreeLoader distributed cache prototype. Experimental results (using multiple scientific data repositories and data transfer tools, as well as a real-world scientific dataset access trace) demonstrate that prefix caching and collective download can be implemented efficiently, our model can select an appropriate prefix size, and the cache hit rate can be improved significantly without hurting the local access rate of cached datasets.
AB - Scientific datasets are typically archived at mass storage systems or data centers close to supercomputers/instruments. End-users of these datasets, however, usually perform parts of their workflows at their local computers. In such cases, client-side caching can offer significant gains by reducing the cost of wide-area data movement.Scientific data caches, however, traditionally cache entire data-sets, which may not be necessary. In this paper, we propose a novel combination of prefix caching and collective download. Prefix caching allows the bootstrapping of dataset downloads by caching only a prefix of the dataset, while collective download facilitates efficient parallel patching of the missing suffix from an external data source. To estimate the optimal prefix size, we further present an analytical model that considers both the initial download over-head and the downloading speed. We implemented our proposed approach in the FreeLoader distributed cache prototype. Experimental results (using multiple scientific data repositories and data transfer tools, as well as a real-world scientific dataset access trace) demonstrate that prefix caching and collective download can be implemented efficiently, our model can select an appropriate prefix size, and the cache hit rate can be improved significantly without hurting the local access rate of cached datasets.
UR - http://www.scopus.com/inward/record.url?scp=34547489494&partnerID=8YFLogxK
U2 - 10.1145/1183401.1183435
DO - 10.1145/1183401.1183435
M3 - Conference contribution
AN - SCOPUS:34547489494
SN - 1595932828
SN - 9781595932822
T3 - Proceedings of the International Conference on Supercomputing
SP - 229
EP - 238
BT - Proceedings of the 20th Annual International Conference on Supercomputing, ICS 2006
T2 - 20th Annual International Conference on Supercomputing, ICS 2006
Y2 - 28 June 2006 through 1 July 2006
ER -