spins2netcdf: Difference between revisions
mNo edit summary |
|||
| (8 intermediate revisions by 3 users not shown) | |||
| Line 2: | Line 2: | ||
spins2netcdf is readily available on Boogaloo. See [[#Installing| Installing]] for where to find spins2netcdf on Hood/Bow/Minnewanka/Waterton. Installation is necessary for use on Graham or Orca. | spins2netcdf is readily available on Boogaloo. See [[#Installing| Installing]] for where to find spins2netcdf on Hood/Bow/Minnewanka/Waterton. Installation is necessary for use on Graham or Orca. | ||
If the data file is large (~>10G) then the visualization might take a while even with many processors. If you know that the location of interest is only a subset of your domain then you can use ncks to crop the domain. See [[NetCDF#ncks|ncks]] on how to do that. | |||
== Requirements and Usage == | == Requirements and Usage == | ||
| Line 24: | Line 26: | ||
== Options == | == Options == | ||
A variety of | A variety of optional arguments are available which give further control over the use of spins2netcdf. If none of the following are used, then all fields for all indices will be converted serially. The most important are: | ||
;-f [ --input-fields ] | ;-f [ --input-fields ] | ||
: Sets the fields to be included in the netcdf file | : Sets the fields to be included in the netcdf file | ||
: Useful for only selecting a few fields from many | : Useful for only selecting a few fields from many | ||
: Usage: -frho -fu etc. (place field name immediately after -f) | : Usage: -frho -fu etc. (place field name immediately after -f) | ||
: If this option is not given, then all fields will be converted | |||
;-i [ --input-indices ] | ;-i [ --input-indices ] | ||
: Sets the indices to be converted | : Sets the indices to be converted | ||
: Useful for only converting those outputs that are needed | : Useful for only converting those outputs that are needed | ||
: Usage: -i0 -i5 etc. (place index immediately after -i) | : Usage: -i0 -i5 etc. (place index immediately after -i) | ||
: If this option is not given, then all indices will be converted | |||
;-n [ --num-threads ] | ;-n [ --num-threads ] | ||
: Sets the number of threads to use in conversion | : Sets the number of threads to use in conversion | ||
| Line 38: | Line 42: | ||
: usage: -n 4 | : usage: -n 4 | ||
: spins2netcdf is not efficiently parallelized, so 4 processors is a suggested value | : spins2netcdf is not efficiently parallelized, so 4 processors is a suggested value | ||
: If this option is not given, then a single processor is used | |||
Example: | Example: | ||
| Line 43: | Line 48: | ||
will convert the rho and u fields at the 10th output into the netcdf file output_10.nc. | will convert the rho and u fields at the 10th output into the netcdf file output_10.nc. | ||
Other options can be seen by running <code>spins2netcdf -h </code>. You can choose the netcdf file name by using the -o option. | Other options can be seen by running <code>spins2netcdf -h </code> or reading the man page with <code>man spins2netcdf</code>. You can choose the netcdf file name by using the -o option. | ||
Do not use the mem-map option! It is approximately 5 times slower than the default. The fastest conversion is accomplished with the -k command which does not swap axes into the preferred orientation (z,y,x), but leaves them in the spins format (y,z,x). This results in plots being rotated by 90 degrees in VisIt and ParaView with incorrectly labeled axes. Only use the -k option if you need the additional speedup in conversion and are ok with the extra work of rotating plots in your visualization software. | Do not use the mem-map option! It is approximately 5 times slower than the default. The fastest conversion is accomplished with the -k command which does not swap axes into the preferred orientation (z,y,x), but leaves them in the spins format (y,z,x). This results in plots being rotated by 90 degrees in VisIt and ParaView with incorrectly labeled axes. Only use the -k option if you need the additional speedup in conversion and are ok with the extra work of rotating plots in your visualization software. | ||
== Errors == | |||
If you get the following message: | |||
Writing output_0.nc | |||
- rho.0 ... Killed | |||
then you do not have sufficient memory to convert the data. On graham you can ask for more when you log into a compute node with salloc, on a local machine you will need to check if other processes are taking up memory and what is available. | |||
== Installing == | == Installing == | ||
| Line 56: | Line 68: | ||
=== Graham / Cedar === | === Graham / Cedar === | ||
There are two ways to install spins2netcdf on Compute Canada systems. They are using nix, or creating your own local copy. | |||
==== Nix ==== | ==== Nix ==== | ||
'''[April 19 2023: Nix temporarily unavailable on Graham]''' | |||
Installation on national platforms (Graham/Cedar) using nix: | Installation on national platforms (Graham/Cedar) using nix: | ||
| Line 70: | Line 85: | ||
==== Create local copy ==== | ==== Create local copy ==== | ||
Installation on national platforms (Graham/Cedar) by creating a local copy: | Installation on national platforms (Graham/Cedar) by creating a local copy: | ||
$ module load boost/1. | $ module load boost/1.80.0 netcdf netcdf-c++4-mpi | ||
$ git clone https://git.sharcnet.ca/DP/spins-tools.git | $ git clone https://git.sharcnet.ca/DP/spins-tools.git | ||
$ cd spins-tools | $ cd spins-tools | ||
$ mkdir build | $ mkdir build | ||
$ cd build | $ cd build | ||
$ cmake -DCMAKE_INSTALL_PREFIX=~ \ | $ cmake \ | ||
-DNETCDF_INCLUDE_DIR= | -DCMAKE_INSTALL_PREFIX=~ \ | ||
-DNETCDF_INCLUDE_DIR=$EBROOTNETCDF/include \ | |||
-DNETCDF_CXX_INCLUDE_DIR= | -DNETCDF_LIBRARY=$EBROOTNETCDF/lib/libnetcdf.so \ | ||
-DNETCDF_CXX_LIBRARY= | -DNETCDF_CXX_INCLUDE_DIR=$EBROOTNETCDFMINCPLUSPLUS4/include \ | ||
-DNETCDF_CXX_LIBRARY=$EBROOTNETCDFMINCPLUSPLUS4/lib/libnetcdf_c++4.so \ | |||
-DBoost_NO_BOOST_CMAKE=1 \ | |||
.. | |||
$ make install | $ make install | ||
=== Orca === | === Orca === | ||
Latest revision as of 13:56, 29 May 2023
A tool for converting SPINS binary outputs into NetCDF files is available from spins2netcdf. The netCDF files are then easily read and visualized within ParaView and VisIt.
spins2netcdf is readily available on Boogaloo. See Installing for where to find spins2netcdf on Hood/Bow/Minnewanka/Waterton. Installation is necessary for use on Graham or Orca.
If the data file is large (~>10G) then the visualization might take a while even with many processors. If you know that the location of interest is only a subset of your domain then you can use ncks to crop the domain. See ncks on how to do that.
Requirements and Usage
spins2netcdf requires a spins.conf in the directory storing the data files to be converted. The current version of SPINS has case files which already uses a spins.conf which satisfies this requirement. Explicitly, the spins.conf requires the following information:
- Nx, Ny, Nz
- Lx, Ly, Lz
- type_x, type_y, type_z
- plot_interval
- mapped_grid (default is false)
On Graham and Orca the module load command presented above in the installation must always be run before running spins2netcdf. To remember this (or rather have it automatically done for you) place the module load command in your .bashrc file. This way it is always run when you log in.
If running on Graham, the login nodes do not have sufficient memory for most jobs so you must log into a compute node with the salloc command. It is suggested to request 4 tasks (cpus) to use make use of the weak parallelization of spins2netcdf.
spins2netcdf collects files of a common suffix (eg. *.0) into a single netcdf file (output_0.nc). To do this for all files and all output indices use,
$ spins2netcdfIf the grid is mapped, then an additional field (zc) is saved as the correct vertical coordinates. When the netcdf file is initially loaded by Paraview or Visit, it will be plotted on a rectilinear grid and will require adjustment onto this new field. See Visualization for details.
Options
A variety of optional arguments are available which give further control over the use of spins2netcdf. If none of the following are used, then all fields for all indices will be converted serially. The most important are:
- -f [ --input-fields ]
- Sets the fields to be included in the netcdf file
- Useful for only selecting a few fields from many
- Usage: -frho -fu etc. (place field name immediately after -f)
- If this option is not given, then all fields will be converted
- -i [ --input-indices ]
- Sets the indices to be converted
- Useful for only converting those outputs that are needed
- Usage: -i0 -i5 etc. (place index immediately after -i)
- If this option is not given, then all indices will be converted
- -n [ --num-threads ]
- Sets the number of threads to use in conversion
- Large files will take a very long time to be read and converted, multiple threads helps to parallelize this
- usage: -n 4
- spins2netcdf is not efficiently parallelized, so 4 processors is a suggested value
- If this option is not given, then a single processor is used
Example:
spins2netcdf -n 4 -i10 -frho -fu
will convert the rho and u fields at the 10th output into the netcdf file output_10.nc.
Other options can be seen by running spins2netcdf -h or reading the man page with man spins2netcdf. You can choose the netcdf file name by using the -o option.
Do not use the mem-map option! It is approximately 5 times slower than the default. The fastest conversion is accomplished with the -k command which does not swap axes into the preferred orientation (z,y,x), but leaves them in the spins format (y,z,x). This results in plots being rotated by 90 degrees in VisIt and ParaView with incorrectly labeled axes. Only use the -k option if you need the additional speedup in conversion and are ok with the extra work of rotating plots in your visualization software.
Errors
If you get the following message:
Writing output_0.nc
- rho.0 ... Killed
then you do not have sufficient memory to convert the data. On graham you can ask for more when you log into a compute node with salloc, on a local machine you will need to check if other processes are taking up memory and what is available.
Installing
Installation on local MFCF-administered machines (hood, bow, minnewanka, waterton):
- it's already installed for you
- hood: standard search rules will already have it (/software/spinstools/bin/spins2netcdf)
- bow, minnewanka, waterton: /usr/local/spinstools/bin/spins2netcdf
Installation on a local machine can be done by following the steps in INSTALL.txt.
Graham / Cedar
There are two ways to install spins2netcdf on Compute Canada systems. They are using nix, or creating your own local copy.
Nix
[April 19 2023: Nix temporarily unavailable on Graham]
Installation on national platforms (Graham/Cedar) using nix:
Note: If nix is used to install spins2netcdf, the nix module must be loaded to enable the use of that package. If nix is not loaded, then spins2netcdf will not be available since it is wrapped up with nix.
$ module load boost/1.60.0 netcdf/4.4.1.1 netcdf-c++4-mpi $ git clone https://git.sharcnet.ca/DP/spins-tools.git $ cd spins-tools $ git pull $ module load nix $ nix-env -if spins-tools.nix
Create local copy
Installation on national platforms (Graham/Cedar) by creating a local copy:
$ module load boost/1.80.0 netcdf netcdf-c++4-mpi $ git clone https://git.sharcnet.ca/DP/spins-tools.git $ cd spins-tools $ mkdir build $ cd build $ cmake \ -DCMAKE_INSTALL_PREFIX=~ \ -DNETCDF_INCLUDE_DIR=$EBROOTNETCDF/include \ -DNETCDF_LIBRARY=$EBROOTNETCDF/lib/libnetcdf.so \ -DNETCDF_CXX_INCLUDE_DIR=$EBROOTNETCDFMINCPLUSPLUS4/include \ -DNETCDF_CXX_LIBRARY=$EBROOTNETCDFMINCPLUSPLUS4/lib/libnetcdf_c++4.so \ -DBoost_NO_BOOST_CMAKE=1 \ .. $ make install
Orca
Installation on SHARCNET systems (Orca):
$ module load cmake/3.7.2 netcdf/4.3.2 boost/intel12-openmpi162/1.55.0 $ git clone https://git.sharcnet.ca/DP/spins-tools.git $ cd spins-tools $ mkdir build $ cd build $ cmake -DCMAKE_INSTALL_PREFIX=~ \ -DNETCDF_INCLUDE_DIR=/opt/sharcnet/netcdf/4.3.2/include \ -DNETCDF_LIBRARY=/opt/sharcnet/netcdf/4.3.2/lib/libnetcdf.so .. $ make install