Massive Data Processing for Agricultural Simulations Using Cloud Computing.

Climate data have been used in agricultural simulation studies including estimation of production potential and strategic management decisions. For geospatial simulations, e.g., soil erosion prediction using climate data, considerable computing resources would be required as the spatial resolution of the simulation increases. Furthermore, a number of ensemble members could be used to minimize uncertainties in the simulation, which would demand substantially increased computing resources. Cloud computing can provide massive computing resources on demand without building or maintaining physical ones. In our pilot study, Amazon Web Service – Elastic Compute Cloud (EC2) was used to process daily sets of climate projection data, which were about 70 gigabytes in total, using virtual machines with a customized database transaction application. The application was used to retrieve daily precipitation data from an internet database, and calculate and store monthly rainfall frequency into a local database in a cloud computing system. Using one server and 10 clients in the cloud computing system, it took about 32 hours to process 17 billion rows of daily precipitation data on a global scale over the 21^st century. Our study showed that cloud computing would provide the high level of performance for agricultural simulations that requires massive amount of climate data.