While focused on the task of generating data for astrophysics Reliability Assurance of Big Data in the Cloud is a worthy read when focused around designing cloud service contacts.
The work of authors Yang, Li and Yuan surround capturing big data reliability, and measuring disk storage solutions including from noted cloud vendors.
Their work at Centre for Astrophysics and Supercomputing at Swinburne University of Technology focused on reducing cloud-based storage cost and energy consumption methods.
They also share the impact of multiple replication-based data storage approaches based upon Proactive Replica Checking for Reliability (PRCR). That was very interesting in their research data gathering.
I found Reliability Assurance of Big Data in the Cloud also supports moving data into the cloud across advanced research networks including Internet2.
Processing raw data inside the data center impacts network models (based upon available bandwidth) in their work. Their research gathers and stores 8 minute segments of telescope data that generates 236GB of raw data. By no means in the petabyte stage (yet) but it still sets a solid understanding of contractual demands on big data cloud storage.
My interest peaked around impacts developing knowledgeable contracts for cloud services. Their background regarding data gathering and processing should influence procurement contract language. This is even more applicable when applied to petabyte data sets and the SLAs regarding data reliability requirements. Never leave money on the table when scaling to the petabyte range. Must read for purchasing agents and corporate (and university) CPSMs.
Internet2 has partnered to transfer real time data between China and New South Wales. Crossing the globe merging multiple countries and processing huge amounts of data. Very cool to see advanced research and education networks grabbing 512 megs per second (per telescope — seven total) and streaming it live to Shanghai. More info