I. Dutra, D. Page, V. Santos Costa, J. Shavlik & M. Waddell (2003).
Toward Automatic Management of Embarrassingly Parallel Applications. Proceedings of International Conference on Parallel and Distributed Computing (Euro-Par), Klagenfurt, Austria.

This publication is available in PDF and available in postscript.

Abstract:

Large-scale applications that require executing very large numbers of tasks are only feasible through parallelism. In this work we present a system that automatically handles large numbers of experiments and data in the context of machine learning. Our system controls all experiments, including re-submission of failed jobs and relies on available resource managers to spawn jobs through pools of machines. Our results show that we can manage a very large number of experiments, using a reasonable amount of idle CPU cycles, with very little user intervention.

Return to the publications of the Univ. of Wisconsin Machine Learning Research Group.