This commit is for code cleanup and for optimizing kernel name search
in the API callback, making sure to get the kernel name accurately
for the hip functions that have any kernel names
Change-Id: Ie9ab917c895748bfb8eee9ddfcbcad81a0b9a9fa
* User regions in Python
* User-region testing + common submodule
- Updated examples/python/source.py to use user-regions
- Python-level user submodule
- Python-level common submodule
- clean-up of profiler python code
- extended source.py testing to include the user-regions
The debugger depends on the CWSR area being executable. Set the right
flag when registering SVM memory.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Tested-by: Laurent Morichetti <Laurent.Morichetti@amd.com>
Change-Id: I7441e214d1a4da8324d775e777976fabd1c81a6f
* Initial support for RCCL
* OMNITRACE_USE_RCCLP + sampling tweaks
- also OMNITRACE_SAMPLING_KEEP_INTERNAL option
- minor modifications to sampling to use keep internal option + discard funlockfile
* Update docker and workflows to download RCCL
* Update CPack DEB with rocprofiler dependency
* Rework rccl into library and library/components folder
- add tpls/rccl/rccl/rccl.h
* Fix timemory includes
* rcclp inline definitions when disabled
* Tweaks to ubuntu-focal-external-rocm
- disable ompt
- enable building testing
* Tweaks to ubuntu-focal-external-rocm
- ctest exclude
* Tweak ubuntu-focal.yml
- remove source /.../setup-env.sh, replace with $GITHUB_ENV
* Fix ubuntu-focal-rocm + OMPI + root
* Improved rocm-smi error handling
- Recover from rocm-smi errors
- Disabling rocm-smi after recovering from errors
- Werror in developer mode
- Remove State::DelayedInit
- Add State::Disabled
* formatting
* Fix merge of OMNITRACE_SAMPLING_KEEP_INTERNAL
* Update RCCL include directory
- based on ROCm version we need with <rccl/rccl.h> or <rccl.h>
* RCCL Testing
- updated tests to use configuration files
- many tests generate a configuration file
- tests how have GPU option
- enable ncclCommCount, disable ncclGetVersion
- add testing for RCCLP via rccl-tests
- working directory of tests is PROJECT_BINARY_DIR
- add nccl/rccl functions to get_whole_function_names
- some clang compiler fixes
* Handle RCCL include w/o HIP
* RCCL requires HIP
* Update OMNITRACE_SAMPLING_CPUS for testing
* Update tests/CMakeLists.txt
* Debug settings
* Install MPI even when USE_MPI=OFF
* exclude printf
* skip mpi tests w/o USE_MPI or USE_MPI_HEADERS
* update ubuntu rocm workflow
* Fix configure env step for ubuntu rocm
* exit gotcha + remove DelayedInit state + cleanup
- exit gotcha which wraps exit, quick_exit, abort
- minor refactor of mpi gotchas
- removed some redundant code in omnitrace_finalized_hidden
- exclude instrumenting functions starting with dlopen and dlsym
- exclude instrumenting exit, quick_exit, and abort functions
- update timemory submodule with support for new gotcha_invoker with (gotcha_data, <function pointer>, args...)
* Improved rocm_smi error handling
* Fix reliability when KOKKOS_PROFILE_LIBRARY is set in env
- in certain situations, an exe using kokkos may be instrumented
- this will cause libomnitrace to be dlopened via libomnitrace-dl
- if KOKKOS_PROFILE_LIBRARY is set to libomnitrace and not libomnitrace-dl, you will end up with different instances of libomnitrace trying to collect data
* Set OMNITRACE_MAX_THREADS=32 in CI
* Added new tests validating gotcha wrappers
* Update MPI example to use thread
* Tweaks to mpi-flat test and mpi_gotcha
- enabled MPI_Comm_size and MPI_Comm_rank in mpip so disabled them at runtime
- set test to collapse threads and processes
* Tweak to test and example
- mpi test sets GOTCHA_DEBUG=1 in env
- removed checking for MPI_{Comm_dup,Comm_group,Group_incl}
- tweaked tests so pthread_join is where it is expected
* v1.3.0
* ROCm 5.2 and extensions tweaks
* Container workflow + miscellaneous updates
* Misc fixes + timemory submodule update
- timemory submodule update for multiple definitions of variant_apply
* Increase timeouts
* Remove obsolete Julia docs and script
- support for rocprofiler makes rocprof merging obsolete
* Fix cpack testing and combine cpack workflows into single script
* Install components + omnitrace tpl exes
- Improved COMPONENT specification for installs
- Install PAPI executables with omnitrace- prefix and hyphens
- Install Perfetto executables with omnitrace- prefix and hyphens
* Update docs on perfetto and papi command-line tools
* remove ubuntu 22.04 from containers workflow
* remove containers workflow running on all pushes
* Fix CI_SCRIPT_ARGS
* Fix PAPI utils install
* Fixed traced test in workflow + removed return char
- validate-perfetto-proto.py had return character
* Fix test-docker-release.sh script to use correct container
* Release build bc RelWtihDebInfo using too much memory
The current state of hsa-rocr does
NOT requires thunk lib as its dependency.
Its unnecessary pulling thunk package while
installing rocr. This patch corrects
the same
Change-Id: Id98ede8b66ffd9aaf4a47da96ba2f981f4c3da73
Maintainer distribution list field had wrong information.
Adding the newly formed DL by the component team.
Change-Id: I61651e429375cdc512d0fe4b0768f917506b5392
KFDExceptionTest.SdmaQueueException allocates VRAM with host access. This
fails on small-BAR GPUs. This error was incorrectly ignored before
412b24137e ("kfdtest: Full TearDown and SetUp in child process").
The test doesn't really need host access to the memory. Therefore the fix
is to disable the HostAccess flag.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: Ifec279eeb6c1ecb1160db9b692e6dc8816d761a3
The CMA feature is deprecated and about to be removed from the DKMS
branch. It was never supported upstream. Leave dummy functions in
place for now.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: I9e51403d753cb91630553aff4f19e931af509740
The CMA feature is deprecated and about to be removed from the DKMS
branch. It was never supported upstream.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: I64b6213eb3adbdc550542e51181cd8ba6ca4cb45
This fixes the 'unknown' value being displayed
for Perf Level because of a missing mapping of
RSMI_DEV_PERF_LEVEL_DETERMINISM to its string
value.
Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: I479c2baea450f0ff61640ad81cbd4d08ad56ff8e
* tweak starting sampling on main thread
- rename omnitrace::common::utility to omnitrace::common::path
- tracing::thread_init_sampling does not start sampling on main thread
* Cancel sampling itimers + timemory submodule update
- updates timemory submodule with support for cancelling itimer and SIGRTMIN through SIGRTMAX
* fix omnitrace print-* with libraries
* timemory submodule update
* Update workflows to use ./bin/omnitrace instead of ./omnitrace
* cmake format
* update timemory submodule
- fix ODR violations in utility/procfs
* cmake updates
- uniform find_package for all ROCm-based libraries
* tweak transpose example
- throw exception instead of std::exit
* Inspect cmdv name before assuming not exe
- some ELF execs "think" they are libraries so only assume rewrite + simulate + all-functions if filename looks like library
- adds some test for --print-available -- <library>
* Fix _has_lib_prefix when command is < 3
* Updates and reverts to omnitrace exe
- update module_function operator< and operator==
- add function_signature operator<
- refactor module_function ctor
- revert some previous changes w.r.t. simulate and include_unninstr
* Fix source/bin/tests to use same output dir as tests
* cmake format
* Segfault mitigation + refactor + modify function iteration
- refactor module_function ctor to avoid segfaults
- string_t -> std::string
- replace std::string with std::string_view in some places
- get_name(module_t*)
- get_name(procedure_t*)
- disable using both app_modules and app_functions
- new option: --parse-all-modules to iterate over app_modules
- removed some unused code w.r.t. debug info
* Disable module_function address range for uninstrumentable functions
* Disable module_function address range for uninstrumentable functions
* Refactored getting file/line info and init/fini
- use dyninst insertInitCallback and insertFiniCallback if main not found
- fixed all issues with segmentation faults in --simulate --all-functions
* revert changes to Findrocprofiler.cmake
hsaKmtMapMemoryToGPU should not try to map VRAM on peer GPUs that don't
have an IO-Link to the memory. The new P2P mapping code in KFD will
fail otherwise.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: I6d59b55651b98756865a0f69eafef3e386372cf3
This allows init_process_apertures to use the whole consistent topoology
instead of taking its own partial snapshot.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: Ia13e7aa7fcd090ea8d6cacd4babb29a27c20207f
With the next patch, child processes need to fully reinitialize the
topology in order to recreate the process apertures. Just calling
hsaKmtOpenKFD is no longer sufficient. Tests based on
KFDMultiProcessTest already did this correctly (KFDHWSTest, eviction
tests). This patch fixes KFDExceptionTest and KFDIPCTest.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: Iaad24e88ddd29c1105bf791a77891cc55a6072ff
The purpose of this patch is to set RETCODE equal to 0 by default
unless an appropriate '--loglevel LEVEL' has been set.
To allow a non-zero RETCODE value, you must use any loglevel that
is not 'warning' or 'None' (default).
You can set the loglevel in the CLI with:
--loglevel <debug/info/warning/error/critical>
Signed-off-by: Ori Messinger <Ori.Messinger@amd.com>
Change-Id: I9484a750206a3f464c59952304e72c59c3d12465
* Initial support for GPU hardware counters
* Update find modules for roctracer and rocprofiler
- /opt/rocm/{rocprofiler,roctracer} path is deprecated so tweak search procedure
* Improve ConfigCPack for MPI
* Update rocprofiler
- rocm_metrics()
- minor cleanup
* Update rocm find modules
* declare rocm_metrics + call in omnitrace-avail
* relocate omnitrace-launch-compiler
* REALPATH and find_modules
* Examples cmake (may drop)
* omnitrace-avail
- hw_counter categories
- init rocm
* setenv updates for rocprofiler in library.cpp and dl.cpp
* get_rocm_events config
* gpu::hip_device_count()
* rocm_metrics returns hardware_counters::info
* - relocated library/components/roctracer_callbacks.* to library/roctracer.*
- relocated library/components/rocprofiler.* to library/rocprofiler.*
- cleaned up rocprofiler.hpp
- added perfetto output of rocprofiler
- added timemory output of rocprofiler
- renamed omni.roctracer thread to roctracer.hip
- added roctracer.hsa thread name
- updated timemory submodule to support std::variant
- updated timemory submodule to support = in config value
- updated timemory submodule to support standalone storage
- updated timemory submodule to support new hw counter apis
- updated timemory submodule to prevent label/description caching in data_tracker
* update omnitrace-avail info_type generation
* Update timemory submodule
* rocprofiler component
* cmake formatting
* omnitrace-avail handle no GPUs
- Add -c command-line option for --categories
- support verbosity
* hsa_rsrc_factory throws exceptions
- throw exceptions to avoid aborting on HSA_STATUS_ERROR_NOT_INITIALIZED when advantageous
- removed duplicate specialization of is_available for component::rocprofiler
* rocprofiler symbols for when disabled
* Fix warning in omnitrace-avail
- std::stringstream from initializer list would use explicit constructor
* Fix finalization after settings are deleted
* Reorganized rocprofiler source
* Updated formatting
* Miscellaneous tweaks
- added using statements from timemory
- tweaked the main and thread bundle names
- fixed timemory header includes
Making sure not to count duplicates for load_unload_reload_trace and
fixed the ignore-count option in check_trace.py.
Change-Id: I9e674aa624ec3b473bb7c6cc95260e240204627f
When separate debug info is requested, the test package
generation fails because /usr/bin/objcopy does not understand
the HSA code object format. We need a workaround to get
past this issue.
Change-Id: I9a307fcf532ce8219a9301850aae972303d19990
When ROCP_TRUNCATE_NAMES is not set, getenv returns NULL and std::atoi
crashes. Check that getenv returns a non-NULL string before calling
std::atoi.
Change-Id: Ie479a481f8d23f034b425d14e3cfefb3d62c84e8
Split the public and private HSA profiler/tracer interfaces. Only the
public interface should be exposed in include/roctracer.
Change-Id: I7e4424cd90023693350c31e6b02caca8c984ba84
Ammar ELWazir
Laurent Morichetti
Jonathan R. Madsen
Felix Kuehling
David Yat Sin
Ashutosh Mishra
Sean Keely
Hao Zhou
Saurabh Verma
Divya Shikre
Ori Messinger
Graham Sider