Commit Graph

8920 Commits

Author SHA1 Message Date
Elena Sakhnovitch 0f88f59ddd [rocm_smi.py] Hiding unnecessary N/A lines
Hiding not applicable/unsupported sensors under INFO

Signed-off-by: Elena Sakhnovitch
Change-Id: I89c80ca7c6365ef3a2dd751a575ddf90044c8a2e
2022-06-23 11:02:13 -04:00
Jonathan R. Madsen be7b03e76f Update dyninst submodule (#65)
* Update dyninst submodule

* Update dyninst submodule

- fixes to install interface libs of Boost, ElfUtils, and LibIberty
2022-06-22 18:07:57 -05:00
Graham Sider e17b159230 libhsakmt: Make queue memory allocation non-paged
Non-paged allocation for queue memory necessary for binding wptr to
GART. Required to support usermode queue oversubscription with MES on
GFX11.

Change ensures queue memory does not specify ATS.

Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I10b23b0205c90dad902c711a88cfb5e9b4979617
2022-06-22 18:37:26 -04:00
Jonathan R. Madsen e2ee542760 Update timemory submodule (#64)
* Update timemory submodule

* Update timemory submodule

- updates timemory's pybind11 submodule due to VS 2022 debug issue

* Drop a commit in timemory submodule
2022-06-22 15:52:13 -05:00
Ranjith Ramakrishnan 52bea549e3 : Use GNUInstallDirs
Use GNUInstallDirs variables in post install scripts

Change-Id: Id0e3e37d412a30521d9846082d025a9e19a43942
2022-06-22 16:28:06 -04:00
Jonathan R. Madsen 335f309b8f libomnitrace uses common headers (#62)
* libomnitrace uses common headers

- JOIN(...) uses ::omnitrace::common::join instead of ::tim::mpl::apply<>::join

* Update lib/common CMake
2022-06-22 15:19:34 -05:00
Jonathan R. Madsen 25e42be09f Update PTL submodule (#63) 2022-06-22 15:18:44 -05:00
Jonathan R. Madsen 354bbf2a32 Rename OMNITRACE_ROCM_SMI_DEVICES to OMNITRACE_SAMPLING_GPUS (#58)
- support ranges in OMNITRACE_SAMPLING_GPUS
2022-06-22 15:01:13 -05:00
Ammar ELWazir 6f97e15d55 Enabling ROCP_STATS_OPT
Enabling the new methodology of ROCP_STATS_OPT of getting HIP activities while the application is running

Change-Id: I19d09e2f2abe16e6528240bf55d6048645aaa09f
2022-06-21 19:30:38 -05:00
Ammar ELWazir 8c28237a2c Changing hcc to hip in hip activity file handle
Change-Id: If4207b72757775d31404768807fe1ad1204521f6
2022-06-21 19:03:33 -04:00
Ammar ELWazir 7cbee920b7 Enabling ROCP_STATS_OPT
Enabling the new methodology of ROCP_STATS_OPT of getting HIP activities while the application is running

Change-Id: I94b3311b0740db804643dba0e4f77c1f9de0319b
2022-06-21 14:58:28 -05:00
Jonathan R. Madsen fe3d15a6c9 Fix attaching to running process, i.e. omnitrace -p <PID> (#60)
* Fix attaching to a process

- e.g. omnitrace -p <PID>

* Update /proc/sys/kernel/yama/ptrace_scope in CI

* Query /proc/sys/kernel/yama/ptrace_scope

* Use AUTHOR_WARNING instead of WARNING for ptrace_scope
2022-06-21 00:28:20 -05:00
Jonathan R. Madsen dfda902092 Improved the determination of MPI rank (#61)
* Improved the determination of MPI rank

* C-style cast of MPI_Comm
2022-06-21 00:27:52 -05:00
Jonathan R. Madsen 5583168dbc Fix find_path in omnitrace-dl (#59)
- added some verbosity
2022-06-21 00:27:37 -05:00
Philip Yang 405fbd6f93 libhsakmt: add open SMI event handle
System Management Interface event is read from anonymous file handle,
this helper wrap the ioctl interface to get anonymous file handle for
GPU nodeid.

Define SMI event IDs, event triggers, copy the same value from
kfd_ioctl.h to avoid translation.

Change-Id: I5c8ba5301473bb3b80bb4e2aa33a9f675bedb001
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
2022-06-20 08:49:13 -04:00
Philip Yang 88778e13dc libhsakmt: hsaKmtGetNodeProperties add gpu_id
Add KFDGpuID to HsaNodeProperties to return gpu_id to upper layer,
gpu_id is hash ID generated by KFD to distinguish GPUs on the system.
ROCr and ROCProfiler will use gpu_id to analyze SMI event message.

Change-Id: I6eabe6849230e04120674f5bc55e6ea254a532d6
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
2022-06-20 08:48:44 -04:00
Jonathan R. Madsen 8eff363ed3 Remove reliance on MPI_Comm_rank (#56)
* Remove reliance on MPI_Comm_rank

- read /proc/<PID>/tasks/<PID>/children of parent process to deduce the rank
- Old format relied on user calling MPI_Comm_rank(MPI_COMM_WORLD, ...)
- if MPI_Comm_rank called with subcommunicators only, multiple ranks would write to same file

* Tweak mpi example
2022-06-20 00:50:49 -05:00
Jonathan R. Madsen f27f062e88 Fixes OMNITRACE_SUPPRESS_CONFIG handling (#53)
Fixes OMNITRACE_SUPPRESS_CONFIG

- now, if set to ON, the config file will not be read
2022-06-20 00:49:57 -05:00
Jonathan R. Madsen 0ef9a1531a Inclusive range for OMNITRACE_SAMPLING_CPUS (#54)
- previously, a value of 0-3 would be CPUs 0, 1, and 2
- now, a value of 0-3 would be CPUs 0, 1, 2, and 3
2022-06-20 00:46:55 -05:00
Jonathan R. Madsen 32a8d40d25 Define new function attributes (#55)
- decorates several functions with OMNITRACE_HOT
2022-06-20 00:44:31 -05:00
Jonathan R. Madsen cafed84d70 Fixes excluded-instr output, fini functions, tweaks MPI (#51)
- fixes population of excluded_module_functions
- omnitrace-compile-definitions have OMNITRACE_USE_MPI and OMNITRACE_USE_MPI_HEADERS
- Do not enable mpi support if no full or partial MPI support
- New option --all-functions
2022-06-19 23:49:47 -05:00
Laurent Morichetti fd1469a573 Enable tests validation in the CI environment
Change-Id: Ied3d203db35083f0045fae052d82ec5e1a867581
2022-06-18 13:50:35 -07:00
Laurent Morichetti 44dbb56fce Fix warnings due to deprecated include path
In file included from roctracer/src/roctracer/tracker.h:24,
                 from roctracer/src/roctracer/roctracer.cpp:44:
/opt/rocm/hsa/include/hsa/amd_hsa_signal.h:26:246: note: ‘#pragma message: amd_hsa_signal.h has moved to ...’
   26 | ssage("amd_hsa_signal.h has moved to ...")
      |                                          ^

Change-Id: I38d151d836688083a4fdb0e86a04fc40923a369f
2022-06-18 13:50:33 -07:00
Jonathan R. Madsen 8837b744ca Fixes excluded-instr output, fini functions, tweaks MPI (#50)
- fixes population of excluded_module_functions
- omnitrace-compile-definitions have OMNITRACE_USE_MPI and OMNITRACE_USE_MPI_HEADERS
- Do not enable mpi support if no full or partial MPI support
- New option --all-functions
2022-06-17 15:52:58 -05:00
Jonathan R. Madsen 20e7b573d4 CI for OpenSUSE (#12)
* CI for OpenSUSE

* OpenSUSE MPI --allow-run-as-root

- when testing in OpenSUSE docker container, tests using OpenMPI fail due to running as root
- This commit toggles OMNITRACE_CI_MPI_RUN_AS_ROOT to ON and tests whether --allow-run-as-root is supported

* Tweak to OMNITRACE_CI_MPI_RUN_AS_ROOT handling
2022-06-17 14:32:51 -05:00
David Salinas d8f236cd81 Handle case with no GPUs
Change-Id: I35b8d4d0c0b6e63d9851d56163686794c7bb8c1e
2022-06-17 14:40:44 -04:00
Ammar ELWazir 57fb3abb31 SWDEV-341811: backward compatibility fix for metrics.xml
Change-Id: Idd0535d399e211664a64390ebc97036495d02511
2022-06-17 13:29:09 -04:00
Hao Zhou 0635134df4 Merge amd-staging into amd-master 20220617
Signed-off-by: Hao Zhou <Hao.Zhou@amd.com>
Change-Id: I1bfa6a012b2bfb7f744e018f129539a495c2c5db
2022-06-17 11:08:54 +08:00
Kent Russell 6b6e840337 rocm_smi.py: Handle corrupted serial number
If the FRU has been corrupted, then the serial number will come in with
any manner of random bytes, which will cause decode() to fail
spectacularily. Check that the serial returned by the kernel is
alphanumeric, and print to the error log if not (then continue to the
next device).

Change-Id: If4f35b140b6089e02729b1490ed6b48d614a122a
2022-06-16 17:29:08 -04:00
Elena Sakhnovitch 4dd2398f3d [rocm_smi.py] error feedback improvement
Cleaning overally verbose error reporting system.

Signed-off-by: Elena Sakhnovitch
Signed-off-by: Sreekant Somasekharan
Change-Id: Icc96086810b8dcfc426848b8c349a2572026c3bd
2022-06-16 14:32:13 -04:00
Ranjith Ramakrishnan b72c464ac0 SWDEV-321112 - Use GNUInstallDirs
Use GNUInstallDirs variables to determine the location of LIBDIR, BINDIR, INCLUDEDIR, DOCDIR

Note that CMAKE_INSTALL_LIBDIR is overriden, since the default for RHEL
is lib64, but ROCm packaging wants it to be lib always. Distros or users
can easily override this.

Change-Id: I616152ccd2bc1f5a60bffa940312b38ca6e88c04
2022-06-16 13:22:49 -04:00
Ranjith Ramakrishnan 0fa2808779 SWDEV-295886 - Replaced hard coded path /opt/rocm with ROCM_PATH
HIP cmake module path set to actual cmake file location

Change-Id: I7a3e6eec2b51d0e550e68496eecf268a01a2d015
2022-06-16 00:11:35 -07:00
Kent Russell f7978b1ff6 run_kfdtest.sh: Add --exclude flag to exclude additional subtests
Currently, using --gtest_filter will override any exclusions from
kfdtest.exclude. To add an additional test (or set of tests) to the
exclusion list dynamically, the --exclude (-e) flag will allow the user
to pass in a string of GTest-style exclusions to add to the list
generated by kfdtest.exclude

e.g. run_kfdtest.sh -e "*KFDLocalMemoryTest.*:KFDEventTest.*"
will use kfdtest.exclude, but will also exclude all LocalMemory and
Event tests

Change-Id: Ic23ec271ba2cd2240d2e98558c0117ff2a064ed2
2022-06-15 16:08:38 -04:00
Ori Messinger 2b8d0ad70f ROCm SMI CLI: Fix setClockRange Error
This patch changes the error handling for setClockRange.

When a device does not support modifying a clock type (sclk/mclk),
an error message is printed through the python CLI.

Signed-off-by: Ori Messinger <Ori.Messinger@amd.com>
Change-Id: I37d9ea4189b1ca81e5deaab5efa6cfa4901b89b3
2022-06-15 15:47:51 -04:00
Jonathan R. Madsen 5bdfecb9dd Fixes the configuration file example (#45)
- example config file will fail due to PR #31
2022-06-14 12:44:41 -05:00
Jonathan R. Madsen a142b2029d Fix the main library stop routine for timemory (#39)
* Fix the main library stop routine for timemory

- the main pop_timemory function was popping too many calls
- this primarily affected recursive calls

* Lengthen the timeout for the Configure CMake step

* Fix python tests

- new validate-timemory-json.py script

* Documentation update

- Call-counts in timemory output examples in documentation were affected by the changes

* Fix the per-thread metrics during finalization

- pthread_create_mutex starts/stops the per-thread data
- removed unintentional continue statement

* Docs tweaks

* Fix lap counter on per-thread metrics
2022-06-13 15:57:44 -05:00
Alex Sierra 0e908f05bb libhsakmt: add env var to set max VA alignment order
This env variable sets the max VA alignment order size as
"PAGE_SIZE * 2^alignment order" during mapping. By default the order
size is set to 9(2MB).

Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I01ae4e0963f4d21c7c367464e60f865bc58d7fac
2022-06-13 15:46:54 -04:00
Ammar ELWazir 93a79eb99a Google-Chrome Tracer parser attaching flow events to the wrong time slice
This issue happens when we have slices that have overlapping
timestamp values (i.e. start_ts + dur = start_ts of next slice)
and the flow event references that shared timestamp value.
The google chrome parser sorts all the events
and just taking the most recent one.
Rocprof is outputting flow attachment points as start+dur;
the end ts of the slice that can overlap with the next slice
if no gap between them.

Solution proposal by Michael Steffen (Michael.Steffen@amd.com)

Change-Id: Ifcde142144033c9012b01a78c95f047384f972dd
2022-06-13 14:53:14 -04:00
Graham Sider 852d29bca2 kfdtest: Add temporary blacklist for GFX11
Add blacklist for tests on GFX11 that are under debug/not functional.

Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I32f375f29036a2fc1a54f2792b31ebc45f4d668b
2022-06-13 14:32:17 -04:00
Daniel Phillips 6da6058d4a libhsakmt: Add support for hsaKmtAvailableMemory
New API function to report available memory per GPU

Signed-off-by: Daniel Phillips <Daniel.Phillips@amd.com>
Change-Id: I63c1e4ca0020c657977ab3635947ab0ed0a81440
2022-06-13 09:37:32 -04:00
Jonathan R. Madsen ab395f86c4 Support omnitrace-dl as kokkos profile library (#37)
- add OnLoad and OnUnload to omnitrace-dl
- disable global fence for kokkos profiling tools
- tweak omnitrace_strip_target to use wildcards
- added dl-gen.py script for generating dlopen bindings
- added support for kokkosp_request_tool_settings
- added support for kokkosp_dual_view_sync
- added support for kokkosp_dual_view_modify
2022-06-10 18:54:22 -05:00
Jonathan R. Madsen b50a13c87e Update youtube links (#38)
- replaced videos with poor audio with videos which are better
2022-06-10 17:08:03 -05:00
Jonathan R. Madsen d4b8e25765 export libomnitrace-dl.so to OMP_TOOL_LIBRARIES (#27)
* export libomnitrace-dl.so to OMP_TOOL_LIBRARIES

* Disable ompt_context_t

- ompt.cpp enables/disables context
- Update timemory submodule :: handles disabled context
2022-06-10 13:16:11 -05:00
Jonathan R. Madsen 017e794d63 Implements --label option for python profiler (#34)
* Implements --label option for python profiler

- removes '-a', '-f', and '-l' options in favor of '--label args file line' for consistency with omnitrace exe

* cmake formatting

* Bump version to 1.1.0
2022-06-10 10:07:15 -05:00
Jonathan R. Madsen 90ab7a89fc Fix sampling counter time scales (#33)
* Fix sampling counter time scales

- All perfetto trace events have "begin_ns" and "end_ns" debug fields
- data for thread start and end timestamp in pthread_create_gotcha
- discard samples outside of thread start and end timestamps
- rename "CPU User CPU Time" perfetto counter to "CPU User Time"
- rename "CPU Kernel CPU Time" perfetto counter to "CPU Kernel Time"
- ensure CPU system samples in perfetto are set to zero at end
- backtrace uses comp::wall_clock record() for timestamps (consistency)
- "Peak Memory Usage [Thread X] (S)" renamed to "Thread Peak Memory Usage [X] (S)"
- "Context Switches [Thread X] (S)" renamed to "Thread Context Switches Usage [X] (S)"
- "Page Faults [Thread X] (S)" renamed to "Thread Page Faults Usage [X] (S)"
- "<PAPI_DESC> [Thread X] (S)" renamed to "Thread <PAPI_DESC> [X] (S)"
- samples

* Fix includes
2022-06-10 08:35:39 -05:00
Jonathan R. Madsen a640fbdb29 Fix loop-level instrumentation + more (#32)
- fix loop-level instrumentation
- support loop instrumentation w/o debug symbols via loop number
- improve module_function messages
- serialize num_basic_blocks
- serialize num_outer_loops
- serialize is_num_instructions_constrained
- serialize is_loop_num_instructions_constrained
- updated transpose example to use uniform_int_distribution
- added transpose loop test
- added fail regexes for tests which enable loop instrumentation
- use module->getFullName in get_loop_file_line_info
- use module->getFullName in get_func_file_line_info
- use module->getFullName in get_basic_block_file_line_info
2022-06-10 06:57:50 -05:00
Hao Zhou 90571416c1 Merge amd-staging into amd-master 20220610
Signed-off-by: Hao Zhou <Hao.Zhou@amd.com>
Change-Id: Iaefd0a9180925d91d2e3ec03be84e5b04cf262b6
2022-06-10 09:08:16 +08:00
Laurent Morichetti 836bab37d3 Remove EvtStats from the tracer_tool
The same information can be generated from the hcc_ops_trace.txt file,
so in a later commit, will add a stage to the tblextr.py script to
generate the .csv files when ROCP_STATS_OPT=1.

Change-Id: I3d1575e096bedf98c66068d9a4ca141421e5bb9d
2022-06-09 15:20:14 -07:00
Jonathan R. Madsen 1db3934f85 Rework sampling trace counter names + new trace counters (#30)
* Rework sampling trace counter names + new trace counters

- reformulate trace counter names for easier comparison (thread sampling)
- new process-level trace counters for context switches (thread sampling)
- new process-level trace counters for page faults (thread sampling)
- new process-level trace counters for CPU time (thread sampling)
- new thread-level trace counters for context switches (sampling)
- new thread-level trace counters for page faults (sampling)

* tweak header include in backtrace.cpp
2022-06-09 13:43:07 -05:00
Bill(Shuzhou) Liu 42f11bdd63 Remove python pyc file when uninstall rpm
Remove python pyc file when uninstall rpm.

Change-Id: I6520b51aac34060b5b90f94a016cec1827a4973f
2022-06-09 09:00:38 -04:00