Scaling tests

Sources

The HPC Wiki provides a good source for the theory and how to perform scaling tests.

Scaling Tests for Belt

To perform scaling tests for belt, you can use the -maxsteps n starting option, which tells belt to only run for n steps and then print out the walltime and time step information. One can then proceed by taking the average time/step of the different runs and compute the speedup and efficiency for the strong and weak scaling tests.

Note: There is a bug in the current belt version, which leads to wrong average time/step when restarting a simulation. When restarting, belt measures the new walltime from the restart of the simulation and then divides by the number of time steps since the start of the simulation. To work around this bug, one could change this line in the main.c:

if (g_stepNumber > 0) print ("> Average time/step       %10.2e  (sec)  \n", 
                              difftime(tend,tbeg)/(double)g_stepNumber);

and swap g_stepNumber with the actual amount of time steps that you want to do. This is not a nice fix and is luckily also not necessary:

If you want to do a scaling test it is recommended to start with a simulation that has already evolved for some time (ideally it is in an equilibrium state). To not always start a new simulation and let it run to this point for every resolution that you want to test, you can use the Pluto interpolation function to start a new simulation with any snapshot as initial condition. This feature is well documented in the Pluto user guide and requires us to add the following lines of code to void InitDomain (Data *d, Grid *grid) in the init.c:

int i, j, k, id, size[3];
  long int offset, offset1;
  double *x1 = grid->x[IDIR], *x1r = grid->xr[IDIR];
  double *x2 = grid->x[JDIR], *x2r = grid->xr[JDIR];
  double *x3 = grid->x[KDIR], *x3r = grid->xr[KDIR];
  char grid_file[32];
  char data_input_file[32];
  sprintf (grid_file,"data/grid_input.out");
  sprintf (data_input_file,"data/data.0050.dbl");

  // Interpolate density (1st record in the file) 
  id = InputDataOpen(data_input_file,grid_file," ", 0, CENTER);
  InputDataGridSize (id, size); // Get grid data size 
  offset = (long)size[0]*(long)size[1]*(long)size[2]; // Offset of a single data block
  TOT_LOOP(k,j,i) d->Vc[RHO][k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
  InputDataClose(id);

  // Interpolate x-velocity (2nd record in the file) 
  id = InputDataOpen(data_input_file,grid_file," ",1*offset, CENTER);
  TOT_LOOP(k,j,i) d->Vc[VX1][k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
  InputDataClose(id);

  // Interpolate y-velocity (3rd record in the file) 
  id = InputDataOpen(data_input_file,grid_file," ",2*offset, CENTER);
  TOT_LOOP(k,j,i) d->Vc[VX2][k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
  InputDataClose(id);

  // Interpolate z-velocity (4th record in the file)
  id = InputDataOpen(data_input_file,grid_file," ",3*offset, CENTER);
  TOT_LOOP(k,j,i) d->Vc[VX3][k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
  InputDataClose(id);


  // Interpolate pressure (5th record in the file) 
  id = InputDataOpen(data_input_file,grid_file," ",4*offset, CENTER);
  TOT_LOOP(k,j,i) d->Vc[PRS][k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
  InputDataClose(id);

  // Interpolate erad (9th record in the file) 
  id = InputDataOpen(data_input_file,grid_file," ",8*offset, CENTER);
  TOT_LOOP(k,j,i) d->RadiationEnergyDensity[k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
  InputDataClose(id);

  // Interpolate Tdust (7th record in the file) 
  id = InputDataOpen(data_input_file,grid_file," ",6*offset, CENTER);
  TOT_LOOP(k,j,i) d->DustTemperature[k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
  InputDataClose(id);

  // Interpolate Tgas (8th record in the file) 
  id = InputDataOpen(data_input_file,grid_file," ",7*offset, CENTER);
  TOT_LOOP(k,j,i) d->GasTemperature[k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
  InputDataClose(id);

This code expects a "grid_input.out" for the grid that we interpolate from and "data.0050.dbl" for the data. This code was written for the single_file output option, for multiple files the code has to be changed accordingly.