SPINS User Guide: Difference between revisions

From Fluids Wiki
Jump to navigation Jump to search
m (Fix typo in git remote set-url command)
(Remove outdated information)
Line 54: Line 54:
  git remote set-url origin https://git.uwaterloo.ca/SPINS/SPINS_main.git
  git remote set-url origin https://git.uwaterloo.ca/SPINS/SPINS_main.git
before running <code>git pull</code>.
before running <code>git pull</code>.
== Examples of common operations ==
You can find examples of how to do various operations by digging through the case files. Some of the common operations are reproduced here.
=== Using wave_reader.cpp ===
The case file wave_reader.cpp is a special case used for initialising the model with specified velocity and density fields. A configuration file, spins.conf, is used to hand parameters to SPINS. An example spins.conf is provided below (# are comments) for a periodic box initialised from 3D files:
<syntaxhighlight lang="text" enclose="div">
  Nx = 256
  Ny = 64
  Nz = 128
  type_x = FOURIER
  type_y = FOURIER
  type_z = FREE_SLIP
  Lx = 20
  Ly = 5
  Lz = 1
  min_x = 0
  min_y = -1
  min_z = 0
  mapped_grid = false
  # xgrid = xgrid.bin
  # zgrid = zgrid.bin
  file_type = FULL
  u_file = input_u
  v_file = input_v
  w_file = input_w
  rho_file = input_rho
  enable_tracer = false
  # tracer_file = tracer.bin
  # tracer_kappa = kappa
  g = 9.81
  rot_f = 0.0
  visc = 0.0
  kappa = 0.0
  perturbation = 0.0
  init_time = 0.0
  final_time = 100
  plot_interval = 1
  restart = false
  # restart_time = 0
  # restart_sequence = 0
</syntaxhighlight>
For a short explanation of each option, the wave_reader help is reproduced here:
<syntaxhighlight lang="text">
  mpirun -np 1 ./wave_reader_x --help
  SPINS: baseline options:
    --config arg (=spins.conf) Configuration file
    --help                    Print this set of options and exit
  Grid Options:
    --Nx arg                Number of points in X
    --Ny arg (=1)          Number of points in Y
    --Nz arg                Number of points in Z
    --type_x arg            Grid type in X.  Valid values are:
                              FOURIER: Periodic
                              FREE_SLIP: Cosine expansion
                              NO_SLIP: Chebyhsev expansion
    --type_y arg (=FOURIER) Grid type in Y
    --type_z arg            Grid type in Z
    --Lx arg                X-length
    --Ly arg (=1)          Y-length
    --Lz arg                Z-length
    --min_x arg (=0)        Unmapped grids: Minimum X-value
    --min_y arg (=0)        Minimum Y-value
    --min_z arg (=0)        Minimum Z-value
  Grid mapping options:
    --mapped_grid arg (=0) Use a mapped (2D) grid -  Note: this must be a 2D grid even if you are restarting a 3D simulation.
    --xgrid arg            x-grid filename
    --zgrid arg            z-grid filename
  Input data:
    --file_type arg      Format of input data files, including that for the
                          mapped grid.Valid options are:
                            MATLAB: Row-major 2D arrays of size Nx x Nz
                            CTYPE:  Column-major 2D arrays (including that
                                    output by 2D SPINS runs)
                            FULL:  Column-major 3D arrays; implies CTYPE for
                                    grid mapping if enabled
    --u_file arg          U-velocity filename
    --v_file arg          V-velocity filename
    --w_file arg          W-velocity filename
    --rho_file arg        Rho (density) filename
  Passive tracer:
    --enable_tracer      Enable evolution of a passive tracer
    --tracer_file arg    Tracer filename
    --tracer_kappa arg    Diffusivity of tracer
  Physical parameters:
    --g arg (=9.8100000000000005) Gravitational acceleration
    --rot_f arg (=0)              Coriolis force term
    --visc arg (=0)              Kinematic viscosity
    --kappa arg (=0)              Thermal diffusivity
    --perturbation arg (=0)      Velocity perturbation (multiplicative white
                                      noise) applied to read-in data.
  Running options:
    --init_time arg (=0)  Initial time
    --final_time arg      Final time
    --plot_interval arg  Interval between output times
  Restart options:
    --restart              Restart from prior output time.  OVERRIDES many other
                            values.  NOT TO BE USED TO EXTEND FROM 2D DATA. For extending set restart to false
    --restart_time arg (=0) Time to restart from
    --restart_sequence arg  Sequence number to restart from (if plot_interval has
                            changed)
</syntaxhighlight>
=== Generating the grid files: regular grid ===
For an unmapped grid, include the following call in your case file's constructor to generate the grid files and grid file readers:
<syntaxhighlight lang="cpp">
  automatic_grid(MinX,MinY,MinZ);
</syntaxhighlight>
where MinX, MinY and MinZ are the coordinates of the starting corner of your grid.
=== Using a mapped grid ===
<span style="color:#FF0000"> Can someone write this? </span>
=== Analytic initialisation ===
<span style="color:#FF0000"> Can someone write this? </span>
=== Boundary conditions ===
<span style="color:#FF0000"> Can someone write this? </span>
=== Forcing / sponge regions ===
<span style="color:#FF0000"> Can someone write this? </span>
=== Online Analysis ===
Some analysis can be done online, some examples are shown below.
==== Energy Diagnostics ====
If you're using a periodic grid, use this for computing kinetic energy diagnostic
<syntaxhighlight lang="cpp">
double dV = (Lx/Nx)*(Ly/Ny)*(Lz/Nz);
double ke = 0.5*rho_0*pssum(sum( u*u+v*v+w*w ))*dV; // KE
</syntaxhighlight>
If you're on a Chebyshev grid, you can use this for the KE computation
<syntaxhighlight lang="cpp">
double ke = pssum(sum((*get_quad_x())(ii)*(*get_quad_y())(jj)*(*get_quad_z())(kk)*
                  (pow(u(ii,jj,kk),2)+pow(v(ii,jj,kk),2)+pow(w(ii,jj,kk),2))));
</syntaxhighlight>
and you will need to compute the quadrature weights in the constructor
<syntaxhighlight lang="cpp">
// Compute the quadrature weights
compute_quadweights(size_x(),size_y(),size_z(),
                    length_x(),length_y(),length_z(),
                    type_x(),type_y(),type_z());
</syntaxhighlight>
(Please note: cleaning up the quadrature code is somehwere buried on the to-do list, so at some point this rather long system should become nicer.)


== Sending SPINS Output to a Log-File ==
== Sending SPINS Output to a Log-File ==
Line 228: Line 62:
</syntaxhighlight>
</syntaxhighlight>
The '''> logfile.log 2>&1''' part routes both stdout ('''1''') and stderr ('''2''') to '''logfile.log'''. The '''< /dev/null''' part tells mpirun to accept 'null' as input instead of the command-line (this allows you to close the terminal while the job is running). Finally, the '''&''' at the end makes the whole process run in the background and returns you to the shell prompt.
The '''> logfile.log 2>&1''' part routes both stdout ('''1''') and stderr ('''2''') to '''logfile.log'''. The '''< /dev/null''' part tells mpirun to accept 'null' as input instead of the command-line (this allows you to close the terminal while the job is running). Finally, the '''&''' at the end makes the whole process run in the background and returns you to the shell prompt.
== SPINS problem resolution ==
SPINS problems and their solutions are listed here.
'''Restarting 3D simulations with mapped grids (wave_reader.x)'''
Problem:  I've had some problems restarting 3D simulations with mapped grids using wave_reader.x. It seems that wave_reader.x requires 2D (x,z) grid files when restarting a simulation with mapped grids, even if the original simulation was 3D.
Resolution:  I suggest building a 2D grid and referring to this grid in the spins.conf file.  I'm not sure if the same issue occurs with non-mapped grids since I haven't been working with non-mapped grids.
'''Initializing 3D simulations with 2D data (wave_reader.x)'''
Make sure that restart=false in spins.conf.  The restart flag represents a true restart simulation and cannot be used to extend a 2D simulation to 3D.
Additionally, ensure that the 2D intialization files have a different name than the 3D output files.

Revision as of 16:16, 5 March 2019

Welcome to the SPINS user guide.


The basics

The SPINS model is a Navier-Stokes solver that gets parameters and initial/boundary conditions from calls to user-provided routines. The user-provided routines are encapsulated in class derived from BaseCase (see BaseCase.hpp).

Creating your own custom configuration involves supplying the user-provided routines in a derived class based on BaseCase. The case file cases/doc_minimal.cpp shows the structure of a case file. It usually makes sense to start with a similar case file and customise it.

SPINS components

SPINS consists of a bunch of C++ source files and a bunch of case files, and it requires four libraries. UMFPack, AMD and Blitz++ are supplied with SPINS, and it uses the system-installed FFTW.

Directory structure:

  • spins/src - SPINS source files
  • spins/src/cases - A few dozen example case files
  • spins/matlab - Some helper functions for MATLAB analysis

How to get SPINS running

SPINS is hosted in a git repository on the UW git server. A guide to using git can be found here: http://git-scm.com/book. If your system does not have a working copy of git that can access http-based repositories (winisk and kazan are currently notable examples), you may get a full copy of the current repository from Christopher Subich (this replaces step #1 below).

You will need to get the code, build the dependencies, build the model and then run it.

Extracting the code from the git repository

Building SPINS

One-time setup

  • Go to the systems directory.
  • Type ./makemake.sh [system]. This will construct the system-specific settings for the makefile.
    • This script reads and processes an appropriate script from the systems/ subdirectory; these files contain variable definitions for compiler names, include/library options, and other attributes that are necessary at build time.
    • There are several scripts in the systems/ subdirectory, and in general one needs to be written for each unique system based on its idiosyncrasies.
    • makemake.sh takes the system name as an optional argument. If not specified, it will try to guess the appropriate host file based on the current hostname. E.g., building on winisk will try to read systems/winisk.sh. If this is inappropriate (such as on clusters, where login nodes are numbered), then the system name can be specified at the command-line, e.g. ./makemake.sh graham to read systems/graham.sh.
  • Execute ./make_deps.sh in the main repository by typing ./make_deps.sh [system] -j. This file may need to be edited to select which libraries aren't present in the current build environment; when in doubt build everything. FFTW and Boost are the libraries most likely to be already installed system-wide.

To build your case-file

  • Enter the src directory.
  • Type make cases/case_directory/your_case.x
  • This requires a file called your_case.cpp in the cases/case_directory directory. There are several cases included with the code so you may want to start with one of those.
  • After successful compilation, an executable called your_case.x is created.

Running SPINS

  • Please be careful not to run in your home directory on machines like boogaloo or graham. Typical cases output a great deal of data, and can quickly fill up your /home quota (or the full partition, on non-quota systems).
  • The code can be executed using mpirun e.g "mpirun -np 4 ./your_case.x".
  • Most cases require a configure file called spins.conf. Options included in the command-line or spins.conf are configured in the main() function of the respective case.

Updating SPINS

Development of spins is on-going and it is advisable to have the most up-to-date version.

To update spins, make sure the SPINS directory is first clean in the git sense (no uncommitted and changed files). Check this by running git status. Once the directory is clean, run git pull.

SPINS was recently (Spring 2018) moved from Belize (which has been decommissioned) to the UW git server. If you installed SPINS prior to this transfer, a git pull will not work as the remote branch is still looking to Belize. Update the remote with 

git remote set-url origin https://git.uwaterloo.ca/SPINS/SPINS_main.git

before running git pull.

Sending SPINS Output to a Log-File

For single-processor jobs, it is quite simple to route standard output (stdout) from the terminal to a log-file using myrun_x > logfile.log. With mpirun, this won't do the trick and the process is a bit more convoluted. The command would be: 

mpirun -np <numProcs> myrun_x > logfile.log 2>&1 < /dev/null &

The > logfile.log 2>&1 part routes both stdout (1) and stderr (2) to logfile.log. The < /dev/null part tells mpirun to accept 'null' as input instead of the command-line (this allows you to close the terminal while the job is running). Finally, the & at the end makes the whole process run in the background and returns you to the shell prompt.

Retrieved from "http://wiki.math.uwaterloo.ca/fluidswiki/index.php?title=SPINS_User_Guide&oldid=1550"