Vectorization and Parallelization of the Adaptive Mesh Refinement N-body Code

Abstract

In this paper, we describe our vectorized and parallelized adaptive mesh refinement (AMR) N-body code with shared time steps, and report its performance on a Fujitsu VPP5000 vector-parallel supercomputer. Our AMR N-body code puts hierarchical meshes recursively where higher resolution is required and the time step of all particles are the same. The parts which are the most difficult to vectorize are loops that access the mesh data and particle data. We vectorized such parts by changing the loop structure, so that the innermost loop steps through the cells instead of the particles in each cell, in other words, by changing the loop order from the depth-first order to the breadth-first order. Mass assignment is also vectorizable using this loop order exchange and splitting the loop into $2^{N_{dim}}$ loops, if the cloud-in-cell scheme is adopted. Here, $N_{dim}$ is the number of dimension. These vectorization schemes which eliminate the unvectorized loops are applicable to parallelization of loops for shared-memory multiprocessors. We also parallelized our code for distributed memory machines. The important part of parallelization is data decomposition. We sorted the hierarchical mesh data by the Morton order, or the recursive N-shaped order, level by level and split and allocated the mesh data to the processors. Particles are allocated to the processor to which the finest refined cells including the particles are also assigned. Our timing analysis using the $\Lambda$-dominated cold dark matter simulations shows that our parallel code speeds up almost ideally up to 32 processors, the largest number of processors in our test.

21pages, 16 figures, to be published in PASJ (Vol. 57, No. 5, Oct. 2005)

Cited by in corpus (12)

• The Sizes of Early-type Galaxies
• The origin of globular cluster systems from cosmological simulations
• On the Color Magnitude Relation of Early-type Galaxies
• Numerical Galaxy Catalog -I. A Semi-analytic Model of Galaxy Formation with N-body simulations
• On spatial distributions of old globular clusters in clusters of galaxies
• Formation of intracluster globular clusters
• The radial alignment of dark matter subhalos: from simulations to observations
• Gravitational Microlensing: A parallel, large-data implementation
• On the Origin of Mass--Metallicity Relations, Blue Tilts, and Scaling Relations for Metal-poor Globular Cluster Systems
• The U-shaped distribution of globular cluster specific frequencies in a biased globular cluster formation scenario
• Formation of the Galactic globular clusters with He-rich stars in low-mass halos virialized at high redshift
• Dark energy constraints and correlations with systematics from CFHTLS weak lensing, SNLS supernovae Ia and WMAP5