# Wasatch

### Summary

Wasatch is a low-Mach, variable density, reacting-flow simulation component for the Uintah computational

framework. It aims at addressing algorithm complexity and scalability by using novel software technologies

developed at the University of Utah (see: SpatialOps, ExprLib). These technologies help address complexity

and scalability by exposing new levels of parallelism (domain decomposition, algorithm decomposition,

stencil operations), facilitating the use hybrid architectures (MPI/Threads/GPU), and providing a high-level

language interface for developers.

Wasatch may be used as a standalone component as well as a tool for other Uintah components to make use of

the technologies provided by SpatialOps and ExprLib. At the time of writing, Wasatch has the following

capabilities:

- Full scalar transport
- Constant density momentum transport
- Turbulence: Large Eddy Simulation – constant Smagorinsky, WALE, and Vreman
- Population balance equation: single internal coordinate using the Quadrature Method of Moments (QMOM)
- Complex geometry

Wasatch is also integrated into both ARCHES and MPMARCHES to provide a convergent development path. This

effort made the implementation of our precipitation model straightforward and resulted in production

simulations for the carbon sequestration group at ICSE.

### Detailed Description

Wasatch is a low-Mach, variable density, reacting-flow simulation component for the Uintah computational

framework. It aims at addressing software complexity and scalability by using novel software technologies

developed at the University of Utah. These technologies take form in two standalone libraries (SpatialOps and ExprLib) that explore the data dependencies in a given simulation. Both software libraries help address

complexity and scalability by exposing new levels of parallelism (domain decomposition, algorithm

decomposition, stencil operations), facilitating the use hybrid architectures (MPI/Threads/GPU), and

providing a high-level language interface for developers.

The idea behind Wasatch is simple. For a given set of mathematical models, the governing equations are

dissected into elemental expressions (e.g. convection, diffusion, particle growth rate, etc…). Each of

these expressions computes one or more quantities and requires other quantities (i.e. expressions). These are

then written as reusable C++ code using high-level syntax provided by SpatialOps (divergence, gradient,

etc…). At the outset, one can represent a complicated set of equations as a directed acyclic graph that

represents the data computed and required by the elemental expressions of the governing equations. In summary,

expressions encapsulate physical phenomena into reusable code using a high-level C++ syntax that is akin to

well known formats such as MatLab.

Once a graph is generated (at runtime) one is capable of making important decisions about the simulation.

For example, certain nodes on the graph (expressions) may be evaluated concurrently using threads while other

nodes may be passed on to GPUs. Furthermore, a graph may be decomposed into subgraphs that can be executed in

parallel (algorithm decomposition).

Wasatch can be thought of as a (successful) case study that illustrates the use of SpatialOps and ExprLib as well as providing evidence for alleviating complexity and scalability. It can be used a standalone

component as well as a tool for other Uintah components to make use of the technologies provided by SpatialOps and ExprLib.

At the time of writing, Wasatch has the following capabilities:

- Scalar transport: Wasatch is capable of solving generic, coupled scalar transport equations of any

complexity. - Momentum transport: Wasatch is capable of solving the constant density momentum transport equations

using pressure projection via the pressure Poisson equation. - Turbulence: Wasatch addresses turbulence by using Large Eddy Simulation (LES). It currently supports

the constant Smagorinsky model, the WALE model, and the Vreman model. The dynamic Smagorinsky model

is currently under development. - Complex boundary conditions: Wasatch makes full use of the graph approach not only on domain interiors, but also at domain boundaries. In this respect, graphs of random complexity and dependencies may be executed on domain boundaries.
- Population balances: Wasatch can solve the population balance equation (PBE) with a single internal

coordinate using the method of moments. Moment closure is achieved via the Quadrature Method of

Moments (QMOM). Population balances with multiple internal coordinates require more complex methods

such as Direct QMOM (DQMOM) whose development is currently under way. - Complex geometry: Wasatch can handle geometries of arbitrary complexity via a stair-stepping

representation. It can also handle certain cases of moving geometry. Current development in this

area is focused on the addition of complex boundary conditions to embedded geometries.

Wasatch is also integrated into both ARCHES and MPMARCHES to provide a convergent development path. This

effort made the implementation of our precipitation model straightforward and resulted in production

simulations for the carbon sequestration group at ICSE.