Skip to content

WW3 in UFS

Jump to bottom
Jessica Meixner edited this page May 12, 2021 · 9 revisions

WW3 is being used in multiple UFS models including the ufs-weather-model, coastal and HAFS. The following information is intended to help developers and users of UFS.

(This is a work in progress, always feel free to submit issues)

Frequently Asked Questions

Which WW3 switch is used?

model/esmf/switch

Which program(s) is the NUOPC cap?

model/ftn/wmesmfmd.ftn

How to port WW3 to a new machine to use in a UFS model?

Here are two example commits of porting WW3 to new platforms:

Note that you will need to add the new machine in the following WW3 subroutines:

  • model/esmf/Makefile
  • model/bin/w3_setup
  • model/bin/cmplr.env

In cmplr.env make sure the options for comp_seq and comp_mpi are appropriate for your machine. Likely you can just follow a template of another machine that is similar to the one you are porting to. If you also add your 'optc' flags, make sure to include '-assume byterecl'.

How to make new mod_def files for regression tests?

Use the routine in ${INPUTDATA_ROOT_WW3}/createmoddefs/creategridfiles.sh where INPUTDATA_ROOT_WW3 is defined by ufs-weather-model/tests/rt.sh

The creategridfiles.sh needs to be copied to a location you can edit. Follow the instructions at the top of the script replacing the needed variables. Run this script, create your new mod_def files and then update your INPUTDATA_ROOT_WW3 to get new mod_def files.

How to turn on debug flags?

Currently, WW3 debug compiler flags cannot be triggered by top level build information, so if you wish to compile WW3 with debug flags, you will need to manually uncomment the "debug" options in ufs-weather-model/WW3/model/bin/cmplr.env for your particular machine/target/compiler setup. For example, search for "$cmplr" == "orion.intel" within cmplr.env and then scroll down to the debug option: if [ ! -z "$(echo $cmplr | grep debug)" ] ; then and uncomment the "if" parts so that you will get the compiler options:

optc="$optc -O0 -debug all -warn all -check all -check noarg_temp_created -fp-stack-check -heap-arrays -traceback -fpe0"
optl="$optl -O0 -traceback"`

Or modify them as your needs arise.

Eventually, this will be streamlined so that a top-level debug option can be passed down from ufs-weather-model, however this is currently not possible.

How to turn on field dumps?

Activate the appropriate cpp flag in wmesmfmd.ftn, which are all by default disabled:

#define TEST_WMESMFMD___disabled
#define TEST_WMESMFMD_GETIMPORT___disabled
#define TEST_WMESMFMD_SETEXPORT___disabled
#define TEST_WMESMFMD_CREATEIMPGRID___disabled
#define TEST_WMESMFMD_CREATEEXPGRID___disabled
#define TEST_WMESMFMD_SETUPIMPBMSK___disabled
#define TEST_WMESMFMD_CHARNK___disabled
#define TEST_WMESMFMD_ROUGHL___disabled
#define TEST_WMESMFMD_BOTCUR___disabled
#define TEST_WMESMFMD_RADSTR2D___disabled
#define TEST_WMESMFMD_STOKES3D___disabled
#define TEST_WMESMFMD_PSTOKES___disabled

For example, to get field dumps for all export fields set:

#define TEST_WMESMFMD_SETEXPORT

Where to get additional WW3 specific timing information from WW3 cap?

If print_esmf is set to true, for the wave PETS, at the end of the log files PET???.ESMF_LogFile, you will get the following timing information:

20210323 165120.988 INFO             PET519 WAV: wtime:                timer     count          time
20210323 165120.988 INFO             PET519 WAV: wtime:         InitializeP0         1  0.137806E-03
20210323 165120.988 INFO             PET519 WAV: wtime:         InitializeP1         1  0.118056E+01
20210323 165120.988 INFO             PET519 WAV: wtime:         InitializeP3         1  0.187102E+00
20210323 165120.988 INFO             PET519 WAV: wtime:       DataInitialize         2  0.713091E-01
20210323 165120.988 INFO             PET519 WAV: wtime:         ModelAdvance        96  0.700550E+03
20210323 165120.988 INFO             PET519 WAV: wtime:             Finalize         1  0.577927E-03
20210323 165120.988 INFO             PET519 WAV: wtime:            GetImport        97  0.506196E+01
20210323 165120.988 INFO             PET519 WAV: wtime:            SetExport        97  0.174132E+02
20210323 165120.988 INFO             PET519 WAV: wtime:          FieldGather       485  0.484256E+01
20210323 165120.988 INFO             PET519 WAV: wtime:            FieldFill       691  0.661621E-01

(For information about using ESMF profiling tools in ufs-weather-model please see: https://github.com/ufs-community/ufs-weather-model/wiki/Advanced-Topics-for-developers#profiling-timing-across-components)

Where to find information about cap in manual?

The manual for the last public release can be found here: https://github.com/NOAA-EMC/WW3/wiki/Tech-Docs or you can build the latest version of the manual from the manual directory. See the section in the appendix titled "Coupling with NUOPC".

Quick Links

Clone this wiki locally