Skip to content

Commit

Permalink
- Document new 3rd party lib build environment variable ESMF_NVML.
Browse files Browse the repository at this point in the history
- Document Darwin's dynamic preload environment variable DYLD_INSERT_LIBRARIES.
- Add URL to ESMX page and add documentation for `devList` ESMX component option.
  • Loading branch information
theurich committed Oct 25, 2023
1 parent 33c76f0 commit 12119ea
Show file tree
Hide file tree
Showing 3 changed files with 45 additions and 5 deletions.
12 changes: 7 additions & 5 deletions src/Infrastructure/Trace/doc/Trace_usage.tex
Original file line number Diff line number Diff line change
Expand Up @@ -248,11 +248,13 @@ \subsubsection{Include MPI Communication in the Profile}
Note that for either option, ESMF must be built with {\tt ESMF\_TRACE\_BUILD\_LIB=ON},
which is the default.

In {\em dynamically linked applications}, the {\tt LD\_PRELOAD} environment variable
must be used when executing the MPI application. This instructs the dynamic
linker to interpose certain MPI symbols so they can be captured by the ESMF
profiler. To simplify this process, a script is provided at
{\tt \$(ESMF\_INSTALL\_LIBDIR)/preload.sh} that sets the {\tt LD\_PRELOAD} variable.
In {\em dynamically linked applications}, the {\tt LD\_PRELOAD} (Linux) or
{\tt DYLD\_INSERT\_LIBRARIES} (Darwin) environment variable must be used when
executing the MPI application. This instructs the dynamic loader to interpose
certain MPI symbols so they can be captured by the ESMF profiler. To simplify
this process, a script is provided at {\tt \$(ESMF\_INSTALL\_LIBDIR)/preload.sh}
that sets the appropriate variable.

For example, if you typically execute your application as as follows:

\begin{verbatim}
Expand Down
3 changes: 3 additions & 0 deletions src/addon/ESMX/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

ESMX is the **E**arth **S**ystem **M**odel e**X**ecutable layer.

http://earthsystemmodeling.org/esmx/

The ESMX layer is built on top of the ESMF and NUOPC APIs.

The idea behind ESMX is to make it as simple as possible for a user to build, run, and test NUOPC based systems. The approach implemented is the same whether applied to a single component, or a fully coupled system of NUOPC-compliant components. ESMX user interfaces are implemented through [YAML](https://yaml.org/) based configuration files.
Expand Down Expand Up @@ -289,6 +291,7 @@ This section affects the specific component instance.
| --------------------- | --------------------------------------------------------------------- | --------------- |
| `model` | string associating the instance with a *component-name* defined in `esmxBuild.yaml` | *non-optional* |
| `petList` | list of PETs on which the component executes | *None* |
| `devList` | list of DEVs (accelerator devices) to be associated with the component| *None* |
| `ompNumThreads` | setting of /NUOPC/Hint/PePerPet/MaxCount (see NUOPC ref doc) | *None* |
| `attributes` | map of key value pairs, each defining a component attribute | *None* |
| *model specific yaml* | each model can define its own YAML section, e.g. with key value pairs, etc. | *None* |
Expand Down
35 changes: 35 additions & 0 deletions src/doc/ESMF_install.tex
Original file line number Diff line number Diff line change
Expand Up @@ -549,6 +549,41 @@ \subsubsection{MOAB}

\end{description}

\subsubsection{NVML}
\label{sec:NVML}

The NVIDIA Management Library
(\htmladdnormallink{NVML}{https://docs.nvidia.com/deploy/nvml-api/index.html})
can be used to discover NVIDIA GPUs that are accessible at run-time.

\begin{description}

\item[ESMF\_NVML] Possible values: {\tt "ON"}, {\tt "standard"}, {\tt "OFF"} (default).

\begin{description}
\item[{\tt "ON"}/{\tt "standard"}] The NVML dependent code inside of ESMF will
be enabled.
The {\tt ESMF\_NVML\_LIBS} environment variable will be set to
{\tt "-lnvidia-ml"}.

\item[{\tt "OFF"} (default)] Disables NVML dependent code.
\end{description}

\item[ESMF\_NVML\_INCLUDE] Typically not needed. (no default).

Specifies the path where the NVML header files are located.

\item[ESMF\_NVML\_LIBPATH] Typically not needed. (no default).

Specifies the path where the NVML library file is located.

\item[ESMF\_NVML\_LIBS] Typical value: {\tt "-lnvidia-ml"} (no default).

Specifies the linker directives needed to link the NVML library to
the application.

\end{description}

\subsection{ESMF Environment Variables}
\label{EnvironmentVariables}

Expand Down

0 comments on commit 12119ea

Please sign in to comment.