Transient exec usage is not required for GFX11 and will result in a NULL
return of s_sendmsg_rtn if directly returned to exec_lo.
Directly fetch and mask the doorbell ID to ttmp3 for GFX11 instead.
Change-Id: Ie17ed69d68d84ab18869b1c7871a0ed0482cd661
Update rocrtst packaging to add dependency on rocm-core so that rocrtst
gets uninstalled when rocm-core package is removed
Depends-On : I1e7ed52d7eed2c190d0b5651e7ded7192d7634b5
Change-Id: I7243dd29950b93a2665720a0062816c574f0f640
* Dyninst fix for BinaryEdit::getResolvedLibraryPath
- parsing of /sbin/ldconfig -p did not handle "Cache generated by: ..." line in Ubuntu 22.04
* Update ubuntu-focal-codecov workflow
- install texinfo
- set MPI_HEADERS_ALLOW_MPICH=ON
hipGraphs acts as a separate activity group as they have their own memory copies and their kernel dispatches and more activities, currently hip api that roctracer depend on doesn't have a way to represent this type, so we are temporarily suppressing the issue up till we have a complete support for the hipGraphs
Change-Id: I5d889be05c9414530672c781e5a712d572ea4104
* CDash name prefix {{ repo_owner }}-{{ ref_name }}
- remove /merge from CI name
* disable using BFD when sampling_include_inlines is OFF
- this consumes a lot of memory
* Improve finalization of rocprofiler
* update timemory submodule
- disable OMPT thread begin/end callbacks
- support hierarchies in signal handlers
- update operation::pop_node debugging
- settings_update_type + setting_supported_data_types
- fixed parsing args in timemory_init
* Improve timemory build time
* Remove kokkosp restrictions for perfetto
* omnitrace exe signal handler update
- configure signal handlers before main to allow libomnitrace to override
* Backtrace and timemory submodule updates
- Use unwind::cache w/o inline info
- update timemory submodule
- unwind::cache updates
- filepath updates
- fix termination_signal_message
- fix vsettings::report_change
* Update dyninst submodule
- updates BinaryEdit::getResolvedLibraryPath
* update timemory submodule
- update CpuArch support
* Cleanup configure warnings
* Update examples cmake and workflows
- (Mostly) eliminate configuration warnings
* omnitrace exe updates
- pass environ to BPatch::processCreate
- avoid trailing ":" in DYNINST_REWRITER_PATHS
* Update dyninst submodule
- Add flags to DyninstOptimization.cmake
- Remove strtok from BinaryEdit::getResolvedLibraryPath
* examples/mpi CMakeLists.txt update
- STATUS message about missing MPI during CI, otherwise AUTHOR_WARNING
* Dev build and linker flags
- use -gsplit-dwarf when OMNITRACE_BUILD_DEVELOPER is ON
- disable when OMNITRACE_BUILD_NUMBER > 1
- OMNITRACE_BUILD_LINKER option
- add -fuse-ld=${OMNITRACE_BUILD_LINKER}
- omnitrace_add_cache_option function
* Update workflows to set OMNITRACE_BUILD_NUMBER
* Fix generator expressions for -fuse-ld
* Suppress some configuration warnings during CI
- helps to keep track of real warnings when they arise
* Update timemory and dyninst submodules with CMP0135
* Add -V flag to run-ci script
- update timemory submodule
- support for load/save of ring_buffers
- new output keys, e.g. `%nid%`
- sampling allocator offloading data
- writing sampling data to temporary file
- new advanced config option `OMNITRACE_USE_TEMPORARY_FILES`
- new advanced config option `OMNITRACE_TMPDIR`
- SIGINT signal (i.e. `Ctrl+C`) triggers backtrace + finalization
- this behavior is common to other profilers
* update output.md docs
* Update omnitrace-avail output keys handling
* update writing metadata
* str format in perfetto_counter_track
* Fix fail regex for mpi-example
* config updates
- OMNITRACE_USE_TEMPORARY_FILES
- OMNITRACE_TMPDIR
- Enable finalization with SIGINT
- code supporting creation of temp files
* sampling offloading to temporary file
* Disable creation of empty temporary files when off
* Submitting jobs to cdash
* Fail on submit
* submit url env
* submit url env
* try passing submit url as arg
* fix submit url
* Updated default URL
* Add submissions for remaining ubuntu focal workflow jobs
* Replace g++ with gcc in dashboard build name
* Add --ctest-args to run-ci.sh
* Add cdash support for bionic, jammy, and opensuse workflows
* Decrease CTEST_CUSTOM_MAXIMUM_PASSED_TEST_OUTPUT_SIZE
* OMNITRACE_BUILD_CODECOV option
* Support code coverage in CDash script
* CI dyninst built with debug info
* Update ci-containers
- cron schedule moved 4 hours later to UTC+5
* Update implementation of config::configure_signal_handler
- using lambdas failed to compile with codecov flags
* Add codecov job to ubuntu focal workflow
* Fix support for --ctest-args in run-ci script
* Fix ubuntu workflows
* Fix quotation handling in run-ci script
* git safe directory for codecov
* New MPI examples
* Remove --stop-on-failure
* dynamic_library update
- find_library_path checks procfs maps
- invoke find_library_path with no additional args to resolve to mapped file
* RCCLP uses dynamic_library
* check if file exists for memory_map_files metadata
* Testing updates
- include new mpi examples in tests
- fix test labels
- test critical-trace exe
* Update MPI C examples tests (needed arg)
* Remove try/catch block from critical-trace
* Fix sampling max wait when shutting down
* Fix test env for critical-trace
* Fix settings for critical-trace
- disable time output: data is deterministic
- disable PID suffixes: not multiprocess
* Update critical-trace ctest
* Update critical-trace exe
- throw error if input cannot be opened
- throw error if input has no data
* Update lulesh example with more kokkos tools usage
* Fix tasking issue with critical_trace and roctracer
- were not setting pools to active
- also sync before critical_trace::get_entries
* Increase verbosity of critical-trace tests
* Update code coverage tests
- skip code coverage + preload
- code-coverage python example and test
* Remove duplication omnitrace.initialize function
* Skip python3.6 for ubuntu jammy
* Update MPI examples
- use MPI_Isend and MPI_Irecv
- explicitly use MPI_Bcast
* Update Formatting.cmake
- include C files in examples
* run-ci script does not check return of coverage
* mpi-allreduce link to libm
* Update ctest args in run-ci script
* Update dyninst submodule
- safety improvements in BinaryEdit::openResolvedLibraryName
* capture cmake error for ctest_coverage
Return value from ReadSysfsStr function that reads cu_occupancy file
was not handled correctly. Modified the script to handle any fail conditions.
Change-Id: I3c71e0f6f288f196ed1f833e8709255c2b6e78ee
merge_traces.sh was depending on tblextr.py, and the location of tblextr.py changed from bin to libexec/rocprofiler
Change-Id: I76f1a0f41ffcff26e6a1119dacfb8b2ebd56d17d
* Use execvpe instead of execve in omnitrace-sample
- previous implementation preferred exe in PATH over exe in PWD
- 'p' variants of exec duplicate the actions of the shell in searching for an executable file if the specified filename does not contain a slash character
* OpenMPI oversubscribe arg in testing
- Respect CMAKE_INSTALL_PREFIX and ignore RDC_CLIENT_INSTALL_PREFIX
- Move example and rdctst from rocm/bin to rocm/share/rdc
- Add README for examples
Signed-off-by: Galantsev, Dmitrii <dmitrii.galantsev@amd.com>
Change-Id: I0b1d996d206327fd1b51ac6e82d548829bdb1570
If hsa_amd_agents_allow_access is called for an imported IPC handle,
ignore the request as this pointer will already have these pointers
mapped to other GPUs during IPCAttach()
Change-Id: I4bf33ed57e93b5a3ead749d4f87ab6f2750bed58
- __libc_start_main in libomnitrace-dl wasn't wrapping bc of -fvisibility=hidden
- fix OMNITRACE_STRIP_TARGET
- omnitrace_reset_preload function in main library
- defer removing libomnitrace from LD_PRELOAD
Get more debug information about user pointers that were registered
through SVM API, and triggered by memory exception events.
A new kfdtest with this use case was also included inside
KFDExceptionTest.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I8e9df3c1c6c3f42d7b9235d12406d80d31746443
Register and map userptrs through Shared Virtual Memory(SVM) API at
the Kernel level when available. Using this approach, performance
will be improve as register/unregister memory will not trigger any
system call to KFD driver.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I20723cbeb340bf48b95e1115f0102c031397bc14
If a user queries the pointer info on an invalid pointer,
hsaKmtQueryPointerInfo will return error or unknown pointer. The other
fields in HsaPointerInfo are invalid, so we do not return them to the
user.
Also removing the assert and returning unknown pointer instead. As the
assert will not trigger in release builds.
hsaKmtQueryPointerInfo may also return unknown pointer for userptrs as
they are not always tracked by thunk. Adjusting code to still treat
these pointers as valid in this case.
Change-Id: Idf5cd8b61cd532d31b072f449839d223369bb138
:Since all public interface libraries are present in
same folder RUNPATH/RPATH is not required in the library itself.
Application shall provide the required RPATH/RUNPATH to load all
libraries.
Change-Id: I1d1ba920bf291eb89bd1f4c0fd0cfd80c7d739bd
- OMNITRACE_ROCTRACER_HSA_ACTIVITY is OFF by default
- OMNITRACE_ROCTRACER_HSA_API is OFF by default
- in real applications, this adds way too much tracing data to perfetto
* Documentation for omnitrace-sample
* Improve omnitrace-sample
- improve the printing of the env updates
- remove env settings when something is deactivated
- restore env settings when something is deactivated
When multiple ranks are used, each rank's first logical device always
has GPU ID 0, regardless of which physical device is selected with
CUDA_VISIBLE_DEVICES. Because of this, when merging trace files from
multiple ranks, GPU IDs from different processes may overlap.
The long term solution is to use the KFD's gpu_id which is stable
across APIs and processes. Unfortunately the gpu_id is not yet exposed
by the ROCr, so for now use the driver's node id.
Change-Id: I2f5af8d2a7e8a89efeb5e0a1b86bdfa547b25fc8
Fix the following error:
roctx.cpp:91:25: error: reinterpret_cast from 'const void *' to 'decltype(report_activity.load())' (aka 'int (*)(activity_domain_t, unsigned int, void *)') casts away qualifiers
report_activity.store(reinterpret_cast<decltype(report_activity.load())>(function),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
by replacing the 'const void *function' argument with the correct type.
Change-Id: I912239daf6f4a3f00fc753306b84833e5c75f74b
* Testing and CI support for Ubuntu 22.04
* Fixes for ROCm
- Jammy does not have ROCm installers
* Name, timeout, and python updates
- renamed ubuntu-jammy-external.yml to ubuntu-jammy.yml
- increased all 5 minute timeouts to 10 minutes
- include python 3.10 in testing
* Update dyninst to remove interposed definition of _r_debug
* Rebuild Dyninst + test install script
* Revert container change
* git safe directory
* pushd -> cd
* fix MPI include
* Fix testing step
* OMPI_ALLOW_RUN_AS_ROOT
* Test script changes
* Fix mismatched malloc / delete[]
* Jammy workflow tweaks
* CPack tweak for boost deb deps
* pthread_mutex_gotcha config returns when not enabled
* fix echoing config in CI
* USE_CLANG_OMP
- option to disable using LLVM OpenMP when building OpenMP test executables
- Jammy workflow sets USE_CLANG_OMP=OFF
* Dyninst submodule boost download
- updated containers workflow to include jammy
- updated workflow to use ci
* Updates to workflows + replace test-install.sh
- test-install.sh in this branch was replaced with one in main branch
* Expand jammy test-install.sh args
* Fix openmp-cg-sampling-duration test
* update timemory submodule
- use-after-free violation in popen::pclose
* revert some tweaks to sampling-duration test
* Fix env of test-install.sh
* cmake format
* jammy bash
* CPack install for jammy
* formatting workflow action version bump
* Update timemory submodule
- libunwind submodule via timemory sets SOVERSION to 99 to avoid ABI conflicts with v8
* Fix help menu for omnitrace-sample
* Support other boolean forms in test-install.sh
* Update docker files and build-docker.sh
- consolidated cases in build-docker.sh
- support rocm version of 0.0 (no rocm install)
- support rocm v5.3
- updated centos handling
* update opensuse actions/checkout version
* Tweaks to ubuntu-focal testing
- actions/checkout@v3
- use test-install script
* update cpack
- ubuntu 22.04
- rocm 5.3
- rename os matrix field to os-version
- remove CI_ROCM_VERSION (no longer necessary)
- remove default-rocm-version matrix field (no longer necessary)
- CentOS packaging
* fix argparsing and omnitrace-sample tests in install-tests.sh
* focal rocm test install workflow fix
* Fix omnitrace-sample build
* Dockerfile.centos + build-docker.sh updates
* Update actions/upload-artifact version
* Dockerfile.ubuntu: install rocm-device-libs
* Refactor cpack
* fix cpack if quotes
* Dockerfile.ubuntu rocm < 5 installs rocm-dev
* build-release.sh defaults to boost version 1.79.0
Strings ([const] char *, [const] char[]) passed as arguments to API
functions may not always contain printable characters. All string
arguments should be quoted and escaped in the trace logs.
Change-Id: Ie39058f2190048b1a0090df16d9ac6bc6507e28a
Amount of memory requested by user may be aligned-up internally to
the memory pool granularity. The extra padded memory should not be
considered when validating pointers from the user. Also return the
user requested size when user queries pointer information.
Change-Id: I28b25448ea03c836b44fafdb34b7330cf6887424
Main CMake improvements:
* Add rdctst with -DBUILD_TESTS=ON
* Set default ROCM_DIR to /opt/rocm/
* Split rdc_libs/CMakeLists.txt into subdirectories
* Package tests into rdc-tests.deb and .rpm
Misc improvements:
* Add .editorconfig to normalize code formatting
* Add .gitignore
* Expand RPATH for gRPC to reduce LD_LIBRARY_PATH usage
* Export compile_commands.json
* Show warning and do not install gRPC if GRPC_ROOT is left as default
* Move .in files into relevant subdirectories
* Move most variables into project CMakeLists.txt to avoid redefinitions
* Normalize CMakeLists.txt formatting (4 spaces indentation)
* Rename DIAGNOSTIC_LIB to RDC_ROCR_LIB
* Update gRPC version in README to 1.44.0
* Remove gtest source
* Pull gtest from github if not installed
Signed-off-by: Galantsev, Dmitrii <dmitrii.galantsev@amd.com>
Depends-On: I1039ef61247e3f0ff822925cc869fb0c2bf3af85
Change-Id: I879b21428e6642f19fda67092b365d8b78b7ba7b
- OMNITRACE_TRACE_THREAD_BARRIERS config option
- set to OFF to disable wrapping `pthread_barrier`
- OMNITRACE_TRACE_THREAD_JOIN config option
- set to OFF to disable wrapping `pthread_join`
- allow PAPI with perf_event_paranoid at level 2
- default to no PAPI events
- setenv LD_PRELOAD to not include libomnitrace after preload
- closes#175
- bump version to 1.7.1
For APU asics, the default configuration size of video memory is
relatively small, while the reserved region becomes larger in recent
generation asics, ratio of max alloc size to the pool size may below
the expected value, so adjust it.
Signed-off-by: Yifan Zhang <yifan1.zhang@amd.com>
Change-Id: I0e847c4c13e957cf6e811d3f379842619cf53370
What we want for libdrm-amdgpu is for it to be a recommended package.
Either libdrm or libdrm-amdgpu can be used, but we recommend the latter.
Using "SUGGESTS" does not seem like a strong enough requirement, but
CPACK does not support RPM recommends. Although, it does allow
customizing the RPM SPEC file template. By generating a template, which
is done by setting:
-DCPACK_RPM_GENERATE_USER_BINARY_SPECFILE_TEMPLATE=1
This template file can be trivially modified to allow adding a line to
implement CPACK_RPM_PACKAGE_RECOMMENDS.
Fixes
Signed-off-by: Jeremy Newton <Jeremy.Newton@amd.com>
Change-Id: I34467b1ba878827ced9b8db74977967815732552
- Raise min instructions default to 1024 instead of 64
- Default value of 64 has demonstrated tendency to slow down real-life
applications
- Improved the memory safety during `omnitrace_finalize()`
- new modifications guarantee that when `tim::manager::instance()` on
main thread is destroyed, omnitrace will finalize before
- Improved some warning w/ roctracer
- Improved the search for `ROCP_METRICS` and
`OMNITRACE_ROCPROFILER_LIBRARY`
- disable printing env by default
- Attempted to improve the sampling shutdown
KFDCWSRTest.BasicTest is parameterized to allow an easy method of
tweaking the number of work-items (and save/restores). The input/output
buffers were previously hardcoded to a single page, which would cause a
segmentation fault if the number of work-items specified is greater than
1024 for wave32.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: Ieefc819a5d81c77cee88081a287fd383e6378e74
Using a thread_local object is problematic as the thread local
destructors are called first before any global destructor, making
the object invalid while tearing down the process.
rocblas uses a global destructor to clean up the loaded HIP modules
and ends up calling hip_executable_destroy after the timestamp stack
is destructed. As a result the begin timestamp for that API function
is 0.
The solution is to store the phase_enter timestamp in the phase_data.
Change-Id: If143f4d123dfb111c72fb20365431d07e73fc570
Using rocprof with ROCP_MCOPY_DATA=1 while tracing HSA produces the
following error:
tblextr.py: Memcpy args "(0x7feb16a00000, 123handle=28593376125, 0x7feb12a00010, 123handle=27558560125, 4194304, 0, 0, 123handle=140661639440000125) = 1" cannot be identified
Profiling data corrupted: ' ./out/rpl_data_220930_143009_1826700/input_results_220930_143009/results.txt'
There are two issues:
1) The hsa_agent_t handle argument is misprinted: "123handle=...125"
Instead of printing '{' and '}', it prints '123' and '125'. The wrong
operator<<(unsigned char) is used and an integer value is printed
instead of a char.
Use std::operator<< instead of hsa_support::detail::operator<< to
print '{' and '}'
2) The result value is unitialized and in some cases printed as a
negative integer value. The leading '-' is not matched by the
mem_manager regular expresion for HSA api calls.
Correctly capture the HSA function's return value.
Change-Id: If13a1e62eeb4e598447c4b90d53d1b2e3b408696
- `omnitrace-sample` executable which executes sampling (no
instrumentation)
- fixes bug with OMPT ignoring value of `OMNITRACE_USE_OMPT`
- fixes some issues with sampling duration
- new `OMNITRACE_SAMPLING_INCLUDE_INLINES` configuration variable
- restricts process-sampling to 100 interrupts/sec when inheriting value
from `OMNITRACE_SAMPLING_FREQ`
- `OMNITRACE_PROCESS_SAMPLING_FREQ` still supports up to 1000
interrupts/sec
- fixes bug with colorized log not truly being disabled in all instances
- adds tests for `omnitrace-sample`
- adds tests for sampling duration
- settings ROCP_TOOL_LIB to libomnitrace-dl throws error
- rocprofiler does not configure correctly when this is done
- Quiet numa_gotcha warnings
- Fixed some shadowed variables
The timestamps coming from the HIP runtime for asynchronus memory
copies are corrupted (begin > end) because the HSA setting to record
timestamps is turned off by the tracer's HSA intercept.
The solution is to intercept hsa_amd_profiling_async_copy_enable and
remember the application/runtime's request so that it can be ORed with
IsEnabled(ACTIVITY_DOMAIN_HSA_OPS, HSA_OP_ID_COPY).
Change-Id: Ib687cbf36711563e86c2bb8bc934c7c51572bfde
Jonathan Kim
Nirmal Unnikrishnan
Jonathan R. Madsen
Ammar ELWazir
Ranjith Ramakrishnan
Sreekant Somasekharan
Galantsev, Dmitrii
David Yat Sin
Alex Sierra
Freddy Paul
Laurent Morichetti
Sriraksha Nagaraj
David Belanger
Yifan Zhang
Sean Keely
Jeremy Newton
Graham Sider
Hao Zhou