Wasatch

Wasatch

Summary

Wasatch is a low-Mach, variable density, reacting-flow simulation component for the Uintah computational
framework. It aims at addressing algorithm complexity and scalability by using novel software technologies
developed at the University of Utah (see: SpatialOps, ExprLib). These technologies help address complexity
and scalability by exposing new levels of parallelism (domain decomposition, algorithm decomposition,
stencil operations), facilitating the use hybrid architectures (MPI/Threads/GPU), and providing a high-level
language interface for developers.

Wasatch may be used as a standalone component as well as a tool for other Uintah components to make use of
the technologies provided by SpatialOps and ExprLib. At the time of writing, Wasatch has the following
capabilities:

  1. Full scalar transport
  2. Constant density momentum transport
  3. Turbulence: Large Eddy Simulation – constant Smagorinsky, WALE, and Vreman
  4. Population balance equation: single internal coordinate using the Quadrature Method of Moments (QMOM)
  5. Complex geometry

Wasatch is also integrated into both ARCHES and MPMARCHES to provide a convergent development path. This
effort made the implementation of our precipitation model straightforward and resulted in production
simulations for the carbon sequestration group at ICSE.

Detailed Description

Wasatch is a low-Mach, variable density, reacting-flow simulation component for the Uintah computational
framework. It aims at addressing software complexity and scalability by using novel software technologies
developed at the University of Utah. These technologies take form in two standalone libraries (SpatialOps and ExprLib) that explore the data dependencies in a given simulation. Both software libraries help address
complexity and scalability by exposing new levels of parallelism (domain decomposition, algorithm
decomposition, stencil operations), facilitating the use hybrid architectures (MPI/Threads/GPU), and
providing a high-level language interface for developers.

The idea behind Wasatch is simple. For a given set of mathematical models, the governing equations are
dissected into elemental expressions (e.g. convection, diffusion, particle growth rate, etc…). Each of
these expressions computes one or more quantities and requires other quantities (i.e. expressions). These are
then written as reusable C++ code using high-level syntax provided by SpatialOps (divergence, gradient,
etc…). At the outset, one can represent a complicated set of equations as a directed acyclic graph that
represents the data computed and required by the elemental expressions of the governing equations. In summary,
expressions encapsulate physical phenomena into reusable code using a high-level C++ syntax that is akin to
well known formats such as MatLab.

sample navier stokes graph

Once a graph is generated (at runtime) one is capable of making important decisions about the simulation.
For example, certain nodes on the graph (expressions) may be evaluated concurrently using threads while other
nodes may be passed on to GPUs. Furthermore, a graph may be decomposed into subgraphs that can be executed in
parallel (algorithm decomposition).

Wasatch can be thought of as a (successful) case study that illustrates the use of SpatialOps and ExprLib as well as providing evidence for alleviating complexity and scalability. It can be used a standalone
component as well as a tool for other Uintah components to make use of the technologies provided by SpatialOps and ExprLib.

At the time of writing, Wasatch has the following capabilities:

  1. Scalar transport: Wasatch is capable of solving generic, coupled scalar transport equations of any
    complexity.
  2. Momentum transport: Wasatch is capable of solving the constant density momentum transport equations
    using pressure projection via the pressure Poisson equation.

  3. Turbulence: Wasatch addresses turbulence by using Large Eddy Simulation (LES). It currently supports
    the constant Smagorinsky model, the WALE model, and the Vreman model. The dynamic Smagorinsky model
    is currently under development.

    graph

  4. Complex boundary conditions: Wasatch makes full use of the graph approach not only on domain interiors, but also at domain boundaries. In this respect, graphs of random complexity and dependencies may be executed on domain boundaries.

  5. Population balances: Wasatch can solve the population balance equation (PBE) with a single internal
    coordinate using the method of moments. Moment closure is achieved via the Quadrature Method of
    Moments (QMOM). Population balances with multiple internal coordinates require more complex methods
    such as Direct QMOM (DQMOM) whose development is currently under way.
  6. Complex geometry: Wasatch can handle geometries of arbitrary complexity via a stair-stepping
    representation. It can also handle certain cases of moving geometry. Current development in this
    area is focused on the addition of complex boundary conditions to embedded geometries.

Wasatch is also integrated into both ARCHES and MPMARCHES to provide a convergent development path. This
effort made the implementation of our precipitation model straightforward and resulted in production
simulations for the carbon sequestration group at ICSE.

Contact

Address   155 South 1452 East
Room 350
Salt Lake City, UT 84112
Phone (801) 585-1233
Email icsewebmaster@utah.edu

Mission

The mission of ICSE is education through interdisciplinary research on high-temperature fuel utilization processes for energy generation, and associated environmental, health, policy, and performance issues.