Grid Benchmarking Research Group. For information on the Global Grid Forum, click here

Chairs:

• Rob F. Van der Wijngaart
  Computer Sciences Corporation
  Mail stop T27A-1
  NASA Ames Research Center
  Moffett Field, CA 94035
  Phone: (650) 604-4397
  E-mail: wijngaar@nas.nasa.gov

• Allan Snavely
  San Diego Supercomputer Center
  Department of Computer Science and Engineering
  University of California, San Diego
  9500 Gilman Drive,
  La Jolla, CA 92093
  Phone: (858) 534-5158
  E-mail: allans@sdsc.edu

Secretary:

• Shyamal Mitra
  Office: CMS 1.143C
  Texas Advanced Computing Center, R8700
  Pickle Research Campus
  University of Texas
  Austin, TX 78758
  Phone: (512) 471-8194
  E-mail: mitra@tacc.utexas.edu

Mailing List: gridmark@gridforum.org

Description and Objectives

• Focus/purpose: We will advance efficient usage of grids by defining metrics to measure performance of grid applications and architectures and rate functionality and efficiency of grid architectures. These metrics should facilitate good engineering practices by allowing alternative implementations to be compared quantitatively. Also, they should provide grid users with information about systems capabilities so that they can develop and tune their applications towards informed objectives. Therefore, we propose a set of tasks that assess grid performance at the level of user applications. The tasks will be specified as paper-and-pencil benchmarks that can be implemented, in principle, using any of the existing and future grid environments. We will also provide some reference implementations of the tasks that can be used by grid users and developers as starting points for high-quality implementations. The tasks will be representative of the kinds of computations that are naturally suited for grid environments, including data-intensive applications. Therefore, we will characterize existing and emerging grid applications to understand and capture their computation needs and data usage patterns.
• Goals: The Grid Benchmarking Research Group specifically sets out to provide the following:
  1. A small set (five or six) of families of grid benchmarks (GB) suitable for capturing the capabilities of grids for performing distributed computations and for accessing data that may reside anywhere across a hierarchy of storage devices, ranging from local disk to remote archival storage. Each family will be defined by a parameterized data flow graph, specifying a number of distinct computational tasks and the communications between them. The parameterization is used to define a collection of different problem sizes within each family. As problem sizes increase, the amount of work per task in the data flow graph will increase, as will the number of tasks and the amount of data communicated between them. GB will be specified unambiguously as paper-and-pencil benchmarks. The specification will be released as an official GGF document.
  2. At least one reference implementation of the whole set of GB, to be published on the GGF web site.
  3. A framework that specifies the requirements for constructing new data flow graphs to be used as grid benchmarks. This will enable users to augment GB with their own applications. Obviously, GB themselves need to satisfy the framework. The framework will be published as part of the GB specification document.
• Milestones:
  1. GGF5: Identification of all synthetic distributed applications to be used to contruct data flow graphs for GB. Four candidate applications, based on the NAS Grid Benchmarks, have been proposed so far.
  2. GGF6: Reference implementations of all GB for at least one problem size each.
  3. GGF7: Reference implementations of all GB for all currently envisioned problem sizes.
  4. GGF8: Document containing paper-and-pencil specifications and framework requirements; GB research group disbands, unless other needs for expansion of GB will be identified.