Visualization

Visualization on Onyx

See Lab visualization tutorial for a basic tutorial on running ParaView on Onyx.

Accessing Networked Drives

As of April 2018, only the NAS is accessible from Onyx. It is generally available as the Z drive, but may need to be mounted the first time. There is a sticky-note on the computer giving instructions on how to do this. This will only need to be done once.

Paraview

To start a parallel session of ParaView, open Command Prompt and run the command:

mpiexec -np N "C:\Program Files\ParaView 5.5.0-RC3-Qt5-MPI-Windows-64bit\bin\pvserver"

where N is the number of cores to use. Onyx has 6 cores (12 with hyper-threading) available to be run, so don't ask for more than 12. Remember that the resources are shared so check how many processors are available before running.

VisIt

To run a parallel session of VisIt, find VisIt in the Start Menu (for non-Windows people, the thing in the bottom left corner) and find VisIt 2.13.1 and choose VisIt Parallel.

Visualization on Graham

All of what you need to know about running Paraview or VisIt on Graham is presented in Graham Visualization.

The general idea is that you run / interact with VisIt/Paraview on your local machine, while have the computing done on Graham. This provides the advantages of:

• not needing to transfer many and/or large data files to your local machine
• running VisIt/ParaView on many processors with a large amount of available memory.

The link above provides a fairly thorough guide to getting up-and-running, just remember to replace cedar with graham where appropriate.

Keep in mind that the requested processors go through a slurm allocation (salloc), and so the more processors that you request, the longer that you will have to wait before receiving the allocation.

VisIt

Note: as of July 2018:

• you must use VisIt 2.13
• it is unclear how to take advantage of GPU accelerations
• parallel VisIt is now available (unofficially)

More information can be gathered at VisIt.

Setting up for Remote VisIt

1. Download the host profile from here.
2. Put host_file.xml in ~/.visit/hosts on your local machine
   • Modify line 42 as appropriate to your user profile

Starting a Remote VisIt Session

1. Run VisIt 2.13 on your local machine
2. Choose Open and under Host choose Graham
   • You will need to wait a minute or so while it connects
3. The Directories and Files panels should now show your home directory on Graham
   • Navigate through to locate the files you wish to visualize
4. After selecting the files and choosing OK, you will be prompted to request processors
   • This will submit a salloc request to get processors for your VisIt session
   • Choose the slurm profile option (should be default) and request the number of processors / time that is appropriate for your purposes
   • You'll need to wait again until the salloc request is allocated.
     • As with any salloc request, the more you request, the longer you'll have to wait
5. Proceed to use VisIt as usual.
   • Saving movies or frames will output the files on your local machine
   • Be aware that the session will die without warning when you reach the time limit for your session, so save often!

Specifying Start Times

If you want/need to request a large number of processors for your VisIt session, you may find that you need to wait a while. This can be troublesome since you won't really know when the session will start (although you can check the status on the job on Graham).

You can, however, specify a requested start time for a session. For example, if you want to run a large visualization job, you could submit the request on Tuesday to have the job start Wednesday morning. That way when you get in on Wednesday you don't have to wait for the connection, since your job will have be gaining priority overnight.

To specify a start time, you can use the --begin=YYYY-MM-DDTHH:MM:SS flag for salloc (as in Graham Tips#Interactive Jobs).

Use the following steps to set this flag in VisIt.

1. go to Options -> Host profiles
2. choose Graham from the list of hosts
3. Launch Profiles -> Parallel -> Advanced
4. Add the appropriate begin flag to Launcher Arguments
5. Select Apply

Increasing available memory

The host profile provided defaults to 2G per processor. If you require more memory, you can

1. go to Options -> Host profiles
2. choose Graham from the list of hosts
3. Launch Profiles -> Parallel -> Advanced
4. Set the desired memory per processor via the mem-per-cpu flag (if unset, defaults to 256M)
5. Select Apply
6. (Optional) to set the new value as the default, use Options -> Save Settings

Paraview

Note: as of Jan 2018:

• you must use ParaView 5.3.0 when using CPU-based computation on Graham
• you must use ParaView 5.4.0 when using GPU-based computation on Graham

See the Compute Canada Visualization page for further updates and details.

More information can be gathered at ParaView.

To submit a paraview job to the slurm scheduler use the following submission script. This will run the python script script.py which must contain all relevant lines of code (read file, create contours/volume plot, adjust color scheme, save screenshot, etc.). No interaction with this job is possible.

#!/bin/bash
# bash script for submitting a Paraview job to the sharcnet Graham queue

#SBATCH --ntasks=4              # number of MPI processes
#SBATCH --mem-per-cpu=32G       # memory per processor (default in Mb)
#SBATCH --time=00-04:00:00      # time (DD-HH:MM:SS)
#SBATCH --job-name="paraview"   # job name
##SBATCH —dependency=afterok:<jobid>  # Wait for job to complete

#SBATCH --output=para-%j.log                # log file
#SBATCH --error=para-%j.err                 # error file
#SBATCH --mail-user=ddeepwel@uwaterloo.ca   # who to email
#SBATCH --mail-type=FAIL                    # when to email
#SBATCH --account=ctb-mmstastn              # UW Fluids designated resource allocation

module load paraview-offscreen
mpirun pvbatch script.py

Mapped netcdf SPINS file

Netcdf files need their data be represented on a rectilinear grid. Mapped (curvilinear) coordinates are accomplished by writing additional fields to which a visualization software can later designate as the correct grid. In the case of mapped SPINS files converted into a netcdf file (through spins2netcdf) this additional field is called zc (for the z-coordinate or curvilinear z). Below are the steps required to plot a field with the correct mapping.

ParaView

• Read in netcdf file (choose "NetCDF files generic and CF conventions")
• Unselect the checkbox for Spherical Coordinates (this seems to only sometimes be necessary)
• Select the Output Type: Structured
• Click Apply
• In the Pipeline, select the file just opened and apply the calculator filter to it (either select the calculator icon, or under Tools->Common in menu bar)
• In the Calculator Properties page:
  • Select Coordinate Result checkbox
  • Change Result Array Name to something useful (such as "grid". This is unnecessary, but helpful if multiple operations are done)
  • In the text box directly below this, input either
    • for 2D: iHat*coordsX+jHat*zc
    • for 3D: iHat*coordsX+jHat*coordsY+kHat*zc
  • Click Apply

VisIt

• Read in netcdf file and add a plot
• Add the displace operator to the plot (Operators->Transforms->Displace)
• Click the arrow to the left of the plot in the Plot List to show the subitems
• Double click the Displace subitem
  • Select Create new expression in the Displacement variable drop down menu
    • Give expression the name "grid" and Type "Vector Mesh Variable"
    • Input the definition as
      • for 2D: "{0*zc,zc - coord(meshN1xN2)[1]}" where meshN1xN2 must be replace with the mesh variable found under Insert Variable->Meshes
      • for 3D: "{0*zc,0*zc,zc - coord(meshN1xN2xN3)[2]}" where meshN1xN2xN3 must be replace with the mesh variable found under Insert Variable->Meshes
    • Click Apply in the Expressions window
  • Select grid in the Displacement variable drop down menu (under Vectors)
  • Click Apply in the Displace operator attributes window
• Click Draw in the main menu

3D Printing

3D print of lobe-cleft instability forming in a gravity current.

Thanks to the marvels of technology, it's not only possibly for you to 3D-print iso-surfaces from your simulations, it's actually fairly straight-forward!

Create .stl for the Iso-surface

For VisIt

• File -> Export Databases
• Choose STL under Export to
• Under Add variable choose the appropriate variable to export
• Choose Export and select Binary in the pop-up window.

For ParaView

• File -> Save Data -> Choose STL from the filetype
• Choose Binary instead of Ascii

Scaling and Thickening

Iso-surfaces produced with the above method will have zero thickness, which causes problems for printing. You will also likely need / want to scale the dimensions of your stl image to fit the dimensions of the 3D printer. The following python script uses Blender to scale and thicken the image.

You will need to have Blender installed on your machine for this to work. To run the script, you can call Blender from the command line via: /Applications/Blender/blender.app/Contents/MacOS/blender --background --python scale_and_thicken.py , where scale_and_thicken.py is the python script included below. Depending on where you installed it, you may need to change the path to the executable.

• You will also need to modify lines 3 and 4 to specify your filenames.
• Depending on the nature of the shape, it may be easiest to print it on its side. To rotate the object, uncomment line 34 and specify the appropriate 'up' axis.
• You can change the thickness by modifying line 31

Note that the produced file has dimensions in mm.

scale_and_thicken.py

import bpy

in_path  = "/Users/bastorer/Desktop/test.stl"
out_path = "/Users/bastorer/Desktop/out.stl"

# Delete all objects
bpy.ops.object.delete()

# Import object
bpy.ops.import_mesh.stl(filepath = in_path)

obj = bpy.context.visible_objects[0].name

print("Modifying <" + obj + "> object")

# Rescale to fit in box
#  with mm dimensions
#  406.4 x 355.6 x 406.4
dims = bpy.data.objects[obj].dimensions
scale_x = 400. / dims[0]
scale_y = 350. / dims[1]
scale_z = 400. / dims[2]
scale = min(scale_x, scale_y, scale_z)

bpy.data.objects[obj].scale[0] = scale
bpy.data.objects[obj].scale[1] = scale
bpy.data.objects[obj].scale[2] = scale

# Add thickness to the object
bpy.ops.object.modifier_add(type = 'SOLIDIFY')
bpy.context.object.modifiers["Solidify"].thickness = 2./scale

# Decimate to reduce file size
bpy.ops.object.modifier_add(type = 'DECIMATE')
bpy.context.object.modifiers["Decimate"].ratio = 0.1

# Rotate the object (if desired)
#bpy.data.objects[obj].up_axis = 'Y'

# Export mesh
bpy.ops.export_mesh.stl(filepath = out_path)