Arbeitsgruppe Kuiper - User contributions [en]

Scaling tests

2024-04-22T08:57:24Z

Dennis: added information for scaling tests

== Sources ==
The [https://hpc-wiki.info/hpc/Scaling HPC Wiki] provides a good source for the theory and how to perform scaling tests.

== Scaling Tests for Belt ==
To perform scaling tests for belt, you can use the '''-maxsteps n''' starting option, which tells belt to only run for '''n''' steps and then print out the walltime and time step information. One can then proceed by taking the '''average time/step''' of the different runs
and compute the speedup and efficiency for the strong and weak scaling tests.

Note: There is a bug in the current belt version, which leads to wrong '''average time/step''' when restarting a simulation. When restarting, belt measures the new walltime from the restart of the simulation and then divides by the number of time steps since the start of the simulation.
To work around this bug, one could change this line in the main.c:

<syntaxhighlight lang="c">
if (g_stepNumber > 0) print ("> Average time/step %10.2e (sec) \n",
difftime(tend,tbeg)/(double)g_stepNumber);
</syntaxhighlight>

and swap '''g_stepNumber''' with the actual amount of time steps that you want to do. This is not a nice fix and is luckily also not necessary:

If you want to do a scaling test it is recommended to start with a simulation that has already evolved for some time (ideally it is in an equilibrium state). To not always start a new simulation and let it run to this point for every resolution that you want to test, you can use the Pluto interpolation function to start a new simulation with any snapshot as initial condition. This feature is well documented in the [https://plutocode.ph.unito.it/userguide.pdf Pluto user guide] and requires us to add the following lines of code to '''void InitDomain (Data *d, Grid *grid)''' in the '''init.c''':
<syntaxhighlight lang="c">
int i, j, k, id, size[3];
long int offset, offset1;
double *x1 = grid->x[IDIR], *x1r = grid->xr[IDIR];
double *x2 = grid->x[JDIR], *x2r = grid->xr[JDIR];
double *x3 = grid->x[KDIR], *x3r = grid->xr[KDIR];
char grid_file[32];
char data_input_file[32];
sprintf (grid_file,"data/grid_input.out");
sprintf (data_input_file,"data/data.0050.dbl");

// Interpolate density (1st record in the file)
id = InputDataOpen(data_input_file,grid_file," ", 0, CENTER);
InputDataGridSize (id, size); // Get grid data size
offset = (long)size[0]*(long)size[1]*(long)size[2]; // Offset of a single data block
TOT_LOOP(k,j,i) d->Vc[RHO][k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
InputDataClose(id);

// Interpolate x-velocity (2nd record in the file)
id = InputDataOpen(data_input_file,grid_file," ",1*offset, CENTER);
TOT_LOOP(k,j,i) d->Vc[VX1][k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
InputDataClose(id);

// Interpolate y-velocity (3rd record in the file)
id = InputDataOpen(data_input_file,grid_file," ",2*offset, CENTER);
TOT_LOOP(k,j,i) d->Vc[VX2][k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
InputDataClose(id);

// Interpolate z-velocity (4th record in the file)
id = InputDataOpen(data_input_file,grid_file," ",3*offset, CENTER);
TOT_LOOP(k,j,i) d->Vc[VX3][k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
InputDataClose(id);

// Interpolate pressure (5th record in the file)
id = InputDataOpen(data_input_file,grid_file," ",4*offset, CENTER);
TOT_LOOP(k,j,i) d->Vc[PRS][k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
InputDataClose(id);

// Interpolate erad (9th record in the file)
id = InputDataOpen(data_input_file,grid_file," ",8*offset, CENTER);
TOT_LOOP(k,j,i) d->RadiationEnergyDensity[k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
InputDataClose(id);

// Interpolate Tdust (7th record in the file)
id = InputDataOpen(data_input_file,grid_file," ",6*offset, CENTER);
TOT_LOOP(k,j,i) d->DustTemperature[k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
InputDataClose(id);

// Interpolate Tgas (8th record in the file)
id = InputDataOpen(data_input_file,grid_file," ",7*offset, CENTER);
TOT_LOOP(k,j,i) d->GasTemperature[k][j][i] = InputDataInterpolate (id,x1[i],x2[j],x3[k]);
InputDataClose(id);
</syntaxhighlight>

This code expects a '''"grid_input.out"''' for the grid that we interpolate from and '''"data.0050.dbl"''' for the data. This code was written for the '''single_file''' output option, for multiple files the code has to be changed accordingly.

Scaling tests

2024-04-22T08:25:39Z

Dennis: Created page with "== Sources == The [https://hpc-wiki.info/hpc/Scaling HPC Wiki] provides a good source for the theory and how to perform scaling tests. == Scaling Tests for Belt == To perform scaling tests for belt, you can use the '''-maxsteps n''' starting option, which tells belt to only run for '''n''' steps and then print out the walltime and time step information. One can then proceed by taking the '''average time/step''' of the different runs and compute the speedup and efficienc..."

Software

2024-04-22T07:59:23Z

Dennis: +Scaling tests

== Overview ==
* Astro stuff:
** [https://git.uni-due.de/agkuiper/belt Belt] (which needs [[PETSc]]): Rolf's PLUTO version ([[compile-belt|manual]] for compiling)
** [[Visit]] Tips and tricks
** [[pyPLUTO]]: a Python module to read and analyse PLUTO output
** [[Scaling tests]] (for belt)
* [[Software#Two-Factor Authentification|Two-Factor Authentification (2FA)]]
* [[VPN]]
* [[SSH and SCP]]
* [https://git.uni-due.de/ University Gitlab]
* [https://uni-due.zoom.us/profile University Zoom]
* [https://nxcl.e-services.physik.uni-due.de University Nextcloud]: This is where we have the group calendar as well!
* [https://www.uni-due.de/zim/services/software/office-ma.php MS Office]. Note that [https://www.libreoffice.org/ LibreOffice] (Windows, Linux; use package manager) is an excellent and free alternative but ''caveat emptor'' about some formatting compatibilities.
* [[Zotero]]
* Making bibliographies compact (e.g. for proposals): https://github.com/lmytime/tex_compact_bib

== Two-Factor Authentification ==
The two-factor authentification (2FA) is needed for most university services as VPN or Nextcloud.

After activating your user-ID you need to activate it in the selfcare portal (https://selfcare.uni-due.de/). The website will walk you through the installation. You'll need to have e.g. an app for the generation of the second factor. I recommend using the app OpenOTP. Please make sure that you scan the shown QR-codes, they are very hard to regain.

However, it is also possible to do it without a phone or application. For this, you only need to install forcepoint and oathtool. Then, you call:
~$ sudo forcepoint-client vpn.uni-due.de
Enter your normal/central university username and password. Then, if warnings come, say yes, and it will ask you for the second factor. Copy the timestamp for the line <code>*** Please enter reply for challenge ***</code> and run—in the same terminal after having paused forcepoint (Ctrl+Z, then "bg", then <Enter>) or in a separate terminal—the following command:
~$ oathtool --totp -b <YOUR "SECRET"> --now="<TIMESTAMP GIVEN BY FORCEPOINT>"
It will return a six-digit number that you then give to forcepoint (<code>fg</code> to bring it back to the foreground, or switching to the tab). You have all the time in the world because you are giving the timestamp explicitly :). You get the "secret" from the selfcare portal when you generate a token (the QR code is just an URL and there is a "…?secret=…&" part to it; the string between "=" and "&" is the "secret").

In general, to parse a QR code: you can use <code>zbarimg</code>, e.g. from a quick screenshot:
~$ zbarimg "/home/georgeslemaistre/Bilder/Bildschirmfoto von 2023-10-05 15-00-00.png"
otpauth://totp/TOTP1938HK33?secret=33IHS343AMDGPH123456CO777JPIIJMJV&period=30&digits=6&issuer=uni-duisburg-essen&image=https%3A//selfcare.uni-due.de/_Resources/Static/Packages/UDE.Portal/images/logo/logo--otp.png
(No worries, this is a fake "secret" :).)

'''Important''': make sure to execute the oathtool command prefixed with a space on the command line so that it not get saved to your command history file because your "secret" is in plain text there. You can call the command once without writing the "secret" (which of course will not work) just to have the command in the history (searching in the history on the command line: <code>Ctrl</code>+<code>R</code>). Typically, you need to close the terminal (exit) for the history actually to be written to the history file (~/.bash_history if you are using bash.) Saving the secret itself in plain text is not the best idea, obviously, so you need to find a solution for that part…

If you forgot to use a space at the beginning of the line and the terminal is still open, go up in command history with the up arrow until you get to that command, delete the command on the line and '''do not press enter''' but first go back "down to the present" with the down arrow and '''then''' press enter. You can check that it has worked by going back up (with the up arrow) and you will see a blank line where the command was. Then you can exit. If you have already closed the terminal, open the history file in an editor and delete the line but make sure that no back-up copy of the history file (containing the command...) is made, or delete it manually afterwards.

Visit

2024-04-09T12:03:26Z

Dennis: /* How To ... */

Some frequent operations, even though surely covered somewhere in the [https://visit-sphinx-github-user-manual.readthedocs.io/en/develop/using_visit/index.html manual] or in the [https://visit-sphinx-github-user-manual.readthedocs.io/en/develop/tutorials/index.html tutorials]:

= How To ... =

# Q: ... specify the axis range in numbers? A: Ctrl+V opens the ''View'' window, in the Tab ''2D view'' the input field ''Window'' contains the space-separated ''xmin xmax ymin ymax''.
# Q: ... combine (add, subtract, ...) fields? A: Ctrl+Shift+E opens the ''Expressions'' Window. You can create a new expression and select the type on the right hand side (e.g. scalar or vector) and insert functions or variables from your current database to apply mathematical operations on them and combine them. After saving the expression by clicking ''apply'', you can add them like any other data field.
# Q: ... combine fields from different time snapshots or databases? A: Open the ''Data Level Comparison Wizard'' with Ctrl+Shift+D. It will guide you through the process of creating a new expression with the field that you want to compare with.

Visit

2024-04-09T12:02:26Z

Dennis:

Some frequent operations, even though surely covered somewhere in the [https://visit-sphinx-github-user-manual.readthedocs.io/en/develop/using_visit/index.html manual] or in the [https://visit-sphinx-github-user-manual.readthedocs.io/en/develop/tutorials/index.html tutorials]:

= How To ... =

# Q: ... specify the axis range in numbers? A: Ctrl+V opens the ''View'' window, in the Tab ''2D view'' the input field ''Window'' contains the space-separated ''xmin xmax ymin ymax''.
# Q: ... combine (add, subtract, ...) fields? A: Ctrl+Shift+E opens the ''Expressions'' Window. You can create a new expression and select the type on the right hand side (e.g. scalar or vector) and insert functions or variables from your current database to apply mathematical operations on them and combine them. After saving the expression by clicking ''apply'', you can add them like any other data field.
# Q: ... combine fields from different time snapshots or databases? A: Open the ''Data Level Comparison Wizard'' with Ctrl+Shift+D. It will guide you through the process of creating a new expression with the field that you want to compare with.

Visit

2024-04-09T12:01:12Z

Dennis:

Some frequent operations, even though surely covered somewhere in the [https://visit-sphinx-github-user-manual.readthedocs.io/en/develop/using_visit/index.html manual] or in the [https://visit-sphinx-github-user-manual.readthedocs.io/en/develop/tutorials/index.html tutorials]:

= How To ... =

# Q: ... specify the axis range in numbers? A: Ctrl+V opens the ''View'' window, in the Tab ''2D view'' the input field ''Window'' contains the space-separated ''xmin xmax ymin ymax''.
# Q: ... combine (add, subtract, ...) fields? A: Ctrl+Shift+E opens the ''Expressions'' Window. You can create a new expression and select the type on the right hand side (e.g. scalar or vector) and insert functions or variables from your current database to apply mathematical operations on them and combine them. After saving the expression by clicking ''apply'', you can add them like any other data field.
# Q: ... combine fields from different time snapshots or databases? A: Open the ''Data Level Comparison Wizard'' with Ctrl+Shift+D. It will guide you through the process of creating a new expression with the field that you want to compare with.

Visit

2024-04-09T12:00:41Z

Dennis: Added Visit instructions for creating expressions and combine different fields.

Some frequent operations, even though surely covered somewhere in the [https://visit-sphinx-github-user-manual.readthedocs.io/en/develop/using_visit/index.html manual] or in the [https://visit-sphinx-github-user-manual.readthedocs.io/en/develop/tutorials/index.html tutorials]:

= How To ... =

# Q: ... specify the axis range in numbers? A: Ctrl+V opens the ''View'' window, in the Tab ''2D view'' the input field ''Window'' contains the space-separated ''xmin xmax ymin ymax''.
# Q: ... combine (add, subtract, ...) fields? A: Ctrl+Shift+E opens the ''Expressions'' Window. You can create a new expression and select the type on the right hand side (e.g. scalar or vector) and insert functions or variables from your current database to apply mathematical operations on them and combine them. After saving the expression by clicking ''apply'', you can add them like any other data field.
# Q: ... combine fields from different time snapshots or databases? A: Open the ''Data Level Comparison Wizard'' with Ctrl+Shift+D. It will guide you through the process of creating a new expression with the field that you want to compare with.