<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>Hello,</div>
<div> I just came across the StarCluster project and was wondering how popular the HDF5 file format is with StarCluster users. One challenge with using HDF5 in a cluster is that HDF5 over NFS is not supported. Currently parallel usage of HDF5 now is restricted
to MPI-I/O with parallel filesystems such as Lustre or GPFS. </div>
<div> At the HDFGroup we’ve been proto-typing a REST-based service implementation of HDF5 – see <a href="http://www.hdfgroup.org/pubs/papers/RESTful_HDF5.pdf">http://www.hdfgroup.org/pubs/papers/RESTful_HDF5.pdf</a> for a description of the API. I can imagine
this service being a good solution for cluster based data I/o. In this scenario each node in the cluster would read/write data via http requests (rather than using NFS). Performance is one aspect that would be critical in a cluster environment. It’s unknown
at this point how much overhead the http request/response processing would add. </div>
<div> I touched on this idea in my presentation at this summer’s HDF Workshop. You can see the slides here: <a href="http://www.slideshare.net/HDFEOS/jxr?ref=http://hdfeos.org/workshops/ws17/slideshare.php">http://www.slideshare.net/HDFEOS/jxr?ref=http://hdfeos.org/workshops/ws17/slideshare.php</a>.
From the architecture slide you can see that clients can access the REST API directly or use the HDF5 library (for C/Fortran applications). The library backend will convert the calls to http requests. </div>
<div> If anyone is interested in this approach it would be great to discuss. Please reply to this thread or contact me directly.</div>
<div><br>
</div>
<div>John Readey</div>
<div>HDF Group </div>
<div><a href="http://www.hdfgroup.org/">http://www.hdfgroup.org/</a></div>
<div>Jreadey AT hdfgroup.org</div>
<div><br>
</div>
<div> </div>
</body>
</html>