This talk describes MPI State Profiling as being prototyped at Sun in the ClusterTools and Sun Studio Performance Tools groups. MPI State Profiling consists of having the MPI library keep track of whether each thread is executing outside the MPI library, whether it is working inside the MPI Library, or whether it is waiting inside the MPI Library. Working is set when the library is actually processing calls, messages, or moving data. Waiting is set whenever the library is waiting for some external event, whether the wait is a busy-wait, or the thread gives up the CPU.

The library also supports an API to read the state. The API call is thread- and async-signal-safe, and is called from the clock-profiling signal handler in the Performance Tools. The MPI State information, like OpenMP state information, is associated with the profile packets, and use to compute MPI Work and MPI Wait metrics of performance, at the level of functions, callers and callees, source-lines or instructions. Since the data is associated with statistical profiling, it potentially offers better scalability than any tracing methodology.