Direct Simulation Monte Carlo (DSMC) Development
 
 

Motivation

Direct simulation Monte Carlo (DSMC) is a particle method that simulates the Boltzmann equation. The Boltzmann equation is the governing equation for dilute gases ranging from continuum to free-molecular conditions. The DSMC method is therefore able to accurately model non-equilibrium (low-density, rarefied) flows and is also accurate for continuum flows where it reproduces results from continuum computational fluid dynamics (CFD). The DSMC method tracks a large number of simulated gas molecules through a computational domain allowing for collisions between gas molecules and also collisions with surfaces. The main assumption inherent in the DSMC method is that molecular movement can be decoupled from molecular collisions (an accurate assumption for dilute gases). Simulated molecules are moved in a straight line for a timestep close to the local mean-collision-time. Afterwards, nearby particles (located within the same computational cell) are collided in a statistical manner. As long as the cell size is less than the mean-free-path (the average distance travelled between molecular collisions), and the collision rate/probability is chosen correctly, the DSMC method is able to accurately simulate real, complex gas flows.


Per cell, the DSMC method is only slightly more expensive than CFD, however, the restrictions of very small cells and small timesteps results in DSMC being quite expensive for near-continuum 3D flows. However, as parallel computing resources continue to rapidly grow, larger and more complex DSMC simulations become possible each year. Typical applications include computing the flow over low-orbit satellites, hypersonic planetary entry simulations, high-altitude high-speed vehicles as well as simulating rocket plumes, DSMC is also used in the manufacture process and function of micro-electromechanical systems (MEMS) as a result of gas flow at extremely small length scales where the molecular nature of the gas must be accounted for.



Results and Publications

 

A 3-Level Cartesian Grid Framework with Adaptive Mesh Refinement:

    Gao, D., Zhang, C., and Schwartzentruber, T.E., “Particle Simulations of Planetary Probe Flows    

    Employing Automated Mesh Refinement”, Journal of Spacecraft and Rockets, Vol. 48, No. 3 (2011)

    397-405.

For both accuracy and efficiency, an ideal DSMC simulation would have cells that are precisely adapted to the local mean-free-path and would move all particles according to their local mean-collision-time. This results in an accurate solution using the least number of computational cells and particles. In order to achieve this, our group has implemented a 3-Level Cartesian Grid DSMC code enabling precise and efficient adaptive mesh refinement (AMR) during the simulation with little user input. The 3 level framework is able to achieve smooth (non-binary) variations in cell size in the presence of strong density gradients. Our implementation is called the Molecular Gas Dynamic Simulator (MGDS) code.

Robust and Efficient “Cut-Cell” Algorithms for Complex Geometry:

    Zhang, C. and Schwartzentruber, T.E., “Robust Cut-cell Algorithms for DSMC Implementations

    Employing Multi-level Cartesian Grids”, Computers & Fluids, Vol. 69 (2012), pp. 122-135.

We have developed robust, efficient software to ‘cut’ arbitrary triangulated surface geometries out of multi-level Cartesian grids. The cut-cell algorithms are called after each AMR procedure to “re-cut” the geometry from the new flow field mesh during a DSMC simulation. The figure shows an example test case for a geometry resembling the MIR space station. The software was developed under a NASA grant in collaboration with researchers from NASA Johnson Space Center. The algorithms have been tested on a wide range of NASA vehicle geometries (including Shuttle and the International Space Station), and the algorithms are included in the latest release version of NASA’s DSMC code, DAC (as of 2013).

Particle Selection Procedures for DSMC:

    Zhang, C. and Schwartzentruber, T.E., “Inelastic Collision Selection Procedures for direct

    simulation Monte Carlo calculations of gas mixtures”, Physics of Fluids, 25, 106105 (2013).

When colliding particles within each DSMC collision cell, the manner in which particles are organized into pairs and selected for collisions is a subtle, but very important, aspect of any DSMC implementation. This is referred to as a particle “selection procedure” and is particularly important when simulating gas mixtures with internal energy (rotation and vibration) relaxation. Specifically, given a set of collision numbers (Zrot,i-j and Zvib,i-j, where i-j represents collisions between species i and j), if two different DSMC codes use different selection procedures, the simulations results will not agree. This subtlety is thus important for the transferability of collision models (i.e. Zrot, Zvib, etc.) between DSMC codes and is also important for consistency with continuum models in the limit of near-equilibrium.

Satisfying Detailed Balance for Collision-Quantity-Based Collision Models:

    Zhang, C., Valentini, P., and Schwartzentruber, T.E., “Nonequilibrium-Direction-Dependent

    Rotational Energy Model for use in Continuum and Stochastic Molecular Simulation”, AIAA

    Journal, Vol. 52, No. 3 (2014), pp. 604-617.

The standard method for redistributing internal energy after a reactive and/or inelastic collision in DSMC is the method of Borgnakke and Larsen (BL). Essentially, post collision energies are sampled from equilibrium distributions. However, the key question is: precisely what distribution should be sampled? Roughly speaking, one should not sample the energy distribution for the gas at equilibrium, rather one should sample the energy distribution found in selected collisions in the gas at equilibrium. Indeed, particles selected for specific collision events (translational and/or rotational and/or vibrational energy exchange) can be biased in a number of ways depending on the collision model and the selection procedure discussed in the previous paragraph! If this bias is not properly accounted for, the DSMC simulation will not satisfy detailed balance and equipartition of energy.

Comparisons of Existing Rotation-Vibration-Dissociation DSMC Models:

    Zhang, C. and Schwartzentruber, T.E., “Numerical Assessment of Vibration and Dissociation

    Models in DSMC for Hypersonic Stagnation Line Flows”, AIAA Paper 2012-2992, June 2012,

    presented at the 43rd AIAA Thermophysics Conference, New Orleans, LA.

Using the consistent DSMC procedures discussed above, we have implemented a number of existing DSMC collision models in the MGDS code and compared their predictions for hypersonic stagnation-line simulations. These models are phenomenological, and when parameterized to the same experimental data set, they should produce similar solutions. In general, we find the agreement to be very good with some interesting subtle differences.

Parallel Implementation Strategies for DSMC:

    Nompelis, I. and Schwartzentruber, T.E., “Strategies for Parallelization of the DSMC Method”,

    AIAA Paper 2013-1204, Jan. 2013, presented at the 51st AIAA Aerospace Sciences Meeting,

    Grapevine, TX.

    Gao, D. and Schwartzentruber, T.E., “Optimizations and OpenMP implementation for the direct

    simulation Monte Carlo method”, Computers & Fluids, 42 (2011) 73-81.

We have recently re-developed the MGDS code, by writing a new parallel MPI code from the ground-up. The new MGDS code utilizes the “hdf5” file system for parallel input/output, as well as a number of novel data structures that enable parallel simulations on large high-performance computing clusters.

We also performed a study regarding parallel implementation of DSMC using combined shared/distributed memory systems (MPI across nodes and OpenMP within a node). Although good scaling on small systems was obtained, the scaling was not consistent between different shared/distributed memory architectures. While such parallelization schemes for multi-core nodes is most certainly going to be the future trend, it is our opinion that either the consistency of hardware architectures and/or restrictions inherent in existing software (such as OpenMP) need to be addressed.