Jeffrey Novotny
0689797736
* Add Sphinx and Read the Docs configs * Add documentation workflow configurations * Changed macros verbprintf and verbprintf_bare so they write to stdout… (#346) Flush stdout when listing keys + bump verbose level for GPU count * Removing static version asserts. (#347) It is causing failures on our internal builds Signed-off-by: David Galiffi <David.Galiffi@amd.com> * Check for an empty vector before popping (#350) Protect from possible seg. fault Signed-off-by: David Galiffi <David.Galiffi@amd.com> * Add release links to installation.md (#351) * Initial infrastructure rework for Omnitrace refactoring and a rewrite of the What is file * Add files in conceptual section, along with images and infrastructure changes. * Formatting and style fixes for files in conceptual directory * Add quick start install guide and fix spelling errors in other files * Add install document and fix code tags. Infrastructure changes * Add two how-to guides along with infra changes and spelling fixes * Add two new how to files and fix errors in the last commit * Fix spelling mistakes * Add new how to file on causal profiling and infra changes. * Add how to file on interpreting Omnitrace output, fixes, and images * Add remaining how-to guides and reference materials along with fixes and infrastructure * Add YouTube file and fix spelling and formatting * Fix a few loose ends and add link to license page * Add Sphinx and Doxygen infrastructure and some additional corrections * Update rocm-docs-core * Fix Doxyfile * Fix path to API header files * Run doxysphinx in conf.py * Add back custom css for doxygen * Remove doxygenlayout * Add api to toc * Update Doxyfile Generate from source .in * Proofreading edits and other changes * Add .gitignore for Doxygen and remove deprecated words and typos * Fix one additional typo * Turn off dot * Update doxyfile strip from path * Workflow, submodules, and thread info Updates (#352) * Update CI workflows - use node20 workflow packages * Update tests/source/CMakeLists.txt - Use OMNITRACE_TRACE and OMNTRACE_PROFILE instead of perfetto/timemory * Update timemory submodule - argparse: requires -> required - parse callbacks * Update thread_info.cpp - fix causal::delay::get_local usage * Update timemory submodule * Update kokkos submodule - release 3.7.02 * Revert opensuse.yml and ubuntu-bionic.yml to use node16 workflows * Update docs.yml * ROCm 6.1 Installers (#349) * Add ROCm 6.1 to packages * Bump version to 1.11.3 * Add 6.1 support to the docker build support. Simplified this by adding 6.* to case statements, now that repo links have been standardized. * Update timemory submodule (#354) - fix argparse::argument::required template deduction * Build omnitrace-rt library (#355) * Build omnitrace-rt library - Explicitly build dyninstAPI_RT as omnitrace-rt so that the SONAME in the ELF is omnitrace-rt instead of dyninstAPI_RT - Create symbolic link lib/omnitrace/libdyninstAPI_RT.so which points to lib/libomnitrace-rt.so - Simplify build tree location of libomnitrace-rt.so since it is ../lib from the bin directory even in the build tree - Update dyninst submodule with minor tweaks to dyninstAPI_RT/CMakeLists.txt * Update source/lib/omnitrace-rt/cmake/platform.cmake * Use ftpmirror.gnu.org instead of ftp.gnu.org - in timemory and dyninst submodules - minor .clang-tidy tweak * Executables append omnitrace library directory to LD_LIBRARY_PATH (#356) - omnitrace-run, omnitrace-sample, and omnitrace-causal now automatically append the LD_LIBRARY_PATH with the directory containing the omnitrace libraries - this helps ensure that binary rewritten exes can resolve omnitrace-rt library location * Fix a few typos and formatting issues * Additional fixes and minor formatting changes. * More fixes and minor formatting changes. * Complete second proofreading with fixes and minor formatting changes. * Make changes to table of contents and disable linting * Update links in the README doc to reflect the new structure. * Align intro on the Omnitrace index page with the first paragraph of the what-is page * Changes and edits based on review comments * Additional changes and edits based on external review * Additional updates and changes from the external review of Omnitrace * Additional changes based on the external review * New round of edits based on the external review * Additional edits based on the external review * Changes to address comments from the internal review * Correct to the RHEL SELinux note in the troubleshooting guide * One additional change to the development guide code example * Move troubleshooting to post-install of install.rst and other minor edits. * Remove troubleshooting page and modify new post-install troubleshooting section on install.rst * Refactor the how Omnitrace works page into seperate topics and redo infrastructure * API ToC changes * Additional API and ToC changes * Back out API and ToC changes and update requirements.txt * Additional API and ToC changes * Add commit for signing purposes * Add ElfUtils and BinUtils Download URL Overrides (#358) * Add CMake CACHE Variable ElfUtils_DOWNLOAD_URL Used to override the default URL to download ElfUtils from. Useful for internal builds Also, include a mirror to fallback to if the override URL fails. * Update timemory submodule Updating to include the BINUTIL_DOWNLOAD_URL override cmake variable. --------- Signed-off-by: David Galiffi <David.Galiffi@amd.com> * Remove Ubuntu 18.04 and SUSE 15.2 * Update checkout action to v4 * Add `docs/**` to `paths-ignore` Document location is being refactored. * Modified submodules dyninst and timemory. (#361) --------- Signed-off-by: David Galiffi <David.Galiffi@amd.com> Co-authored-by: Peter Jun Park <peter.park@amd.com> Co-authored-by: ajanicijamd <Aleksandar.Janicijevic@amd.com> Co-authored-by: David Galiffi <David.Galiffi@amd.com> Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com> Co-authored-by: Sam Wu <22262939+samjwu@users.noreply.github.com>
137 linhas
4.3 KiB
ReStructuredText
137 linhas
4.3 KiB
ReStructuredText
.. meta::
|
|
:description: Omnitrace documentation and reference
|
|
:keywords: Omnitrace, ROCm, profiler, tracking, visualization, tool, Instinct, accelerator, AMD
|
|
|
|
***************************************
|
|
The Omnitrace feature set and use cases
|
|
***************************************
|
|
|
|
`Omnitrace <https://github.com/ROCm/omnitrace>`_ is designed to be highly extensible.
|
|
Internally, it leverages the `Timemory performance analysis toolkit <https://github.com/NERSC/timemory>`_
|
|
to manage extensions, resources, data, and other items. It supports the following features,
|
|
modes, metrics, and APIs.
|
|
|
|
Data collection modes
|
|
========================================
|
|
|
|
* Dynamic instrumentation
|
|
|
|
* Runtime instrumentation: Instrument executables and shared libraries at runtime
|
|
* Binary rewriting: Generate a new executable and/or library with instrumentation built-in
|
|
|
|
* Statistical sampling: Periodic software interrupts per-thread
|
|
* Process-level sampling: A background thread records process-, system- and device-level metrics while the application runs
|
|
* Causal profiling: Quantifies the potential impact of optimizations in parallel code
|
|
|
|
.. note::
|
|
|
|
Critical trace support was removed in Omnitrace v1.11.0.
|
|
It was replaced by the causal profiling feature.
|
|
|
|
Data analysis
|
|
========================================
|
|
|
|
* High-level summary profiles with mean, min, max, and standard deviation statistics
|
|
|
|
* Low overhead and memory efficient
|
|
* Ideal for running at scale
|
|
|
|
* Comprehensive traces for every individual event and measurement
|
|
* Application speed-up predictions resulting from potential optimizations in functions and lines of code based on causal profiling
|
|
|
|
Parallelism API support
|
|
========================================
|
|
|
|
* HIP
|
|
* HSA
|
|
* Pthreads
|
|
* MPI
|
|
* Kokkos-Tools (KokkosP)
|
|
* OpenMP-Tools (OMPT)
|
|
|
|
GPU metrics
|
|
========================================
|
|
|
|
* GPU hardware counters
|
|
* HIP API tracing
|
|
* HIP kernel tracing
|
|
* HSA API tracing
|
|
* HSA operation tracing
|
|
* System-level sampling (via rocm-smi)
|
|
|
|
* Memory usage
|
|
* Power usage
|
|
* Temperature
|
|
* Utilization
|
|
|
|
CPU metrics
|
|
========================================
|
|
|
|
* CPU hardware counters sampling and profiles
|
|
* CPU frequency sampling
|
|
* Various timing metrics
|
|
|
|
* Wall time
|
|
* CPU time (process and thread)
|
|
* CPU utilization (process and thread)
|
|
* User CPU time
|
|
* Kernel CPU time
|
|
|
|
* Various memory metrics
|
|
|
|
* High-water mark (sampling and profiles)
|
|
* Memory page allocation
|
|
* Virtual memory usage
|
|
|
|
* Network statistics
|
|
* I/O metrics
|
|
* Many others
|
|
|
|
Third-party API support
|
|
========================================
|
|
|
|
* TAU
|
|
* LIKWID
|
|
* Caliper
|
|
* CrayPAT
|
|
* VTune
|
|
* NVTX
|
|
* ROCTX
|
|
|
|
Omnitrace use cases
|
|
========================================
|
|
|
|
When analyzing the performance of an application, do NOT
|
|
assume you know where the performance bottlenecks are
|
|
and why they are happening. Omnitrace is a tool for analyzing the entire
|
|
application and its performance. It is
|
|
ideal for characterizing where optimization would have the greatest impact
|
|
on an end-to-end run of the application and for
|
|
viewing what else is happening on the system during a performance bottleneck.
|
|
|
|
When GPUs are involved, there is a tendency to assume that
|
|
the quickest path to performance improvement is minimizing
|
|
the runtime of the GPU kernels. This is a highly flawed assumption.
|
|
If you optimize the runtime of a kernel from one millisecond
|
|
to 1 microsecond (1000x speed-up) but the original application never
|
|
spent time waiting for kernels to complete,
|
|
there would be no statistically significant reduction in the end-to-end
|
|
runtime of your application. In other words, it does not matter
|
|
how fast or slow the code on GPU is if the application has a
|
|
bottleneck on waiting on the GPU.
|
|
|
|
Use Omnitrace to obtain a high-level view of the entire application. Use it
|
|
to determine where the performance bottlenecks are and
|
|
obtain clues to why these bottlenecks are happening. Rather than worrying about kernel
|
|
performance, start your investigation with Omnitrace, which characterizes the
|
|
broad picture.
|
|
|
|
.. note::
|
|
|
|
For insight into the execution of individual kernels on the GPU,
|
|
use `Omniperf <https://github.com/rocm/omniperf>`_.
|
|
|
|
In terms of CPU analysis, Omnitrace does not target any specific vendor.
|
|
It works just as well on AMD and non-AMD CPUs.
|
|
With regard to the GPU, Omnitrace is currently restricted to HIP and HSA APIs
|
|
and kernels running on AMD GPUs. |