- Current Perfetto source repo is from android.googlesource
which is censored in China and causes an infinite
hang when building Omnitrace
- This commit changes the Perfetto source repo to Github which acts
as a read-only mirror to the googlesource perfetto
repo.
Co-authored-by: David Galiffi <David.Galiffi@amd.com>
[ROCm/rocprofiler-systems commit: 6f22b2d2e5]
Fix for "SWDEV-479652" - Perfetto-based tests are failing.
Updated version of perfetto submodule to v46.0.
Modified Omnitrace code that uses Perfetto, so it can compile.
Modified the testing code, so it can run the version of trace_processor_shell provided (v46.0).
---------
Signed-off-by: Aleksandar Janicijevic <Aleksandar.Janicijevic@amd.com>
Co-authored-by: David Galiffi <David.Galiffi@amd.com>
[ROCm/rocprofiler-systems commit: 96d7b8f0ab]
* Add ROCm 6.2 to CPACK workflow.
* Update APT-Get packages
- hipcc is no longer included in hip-runtime-amd meta packages. This change standardizes the installs with the other OSes.
* Bump version to 1.11.4
---------
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
[ROCm/rocprofiler-systems commit: 6b0627f5b7]
* Add texinfo to Ubuntu and RHEL docker files
* Add the `bison` package to Ubuntu dockerfile
* Update the CI docker files too.
---------
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
[ROCm/rocprofiler-systems commit: d0d97490b4]
* Modified submodules dyninst and timemory.
* Modified PAPI submodule to fix GCC 13 build for Ubuntu 24.04.
* Updated PAPI submodule URL.
* Fixed papi submodule URL.
* Using latest tag (papi-7-1-0-t) for papi submodule.
* Update submodule dyninst to new version
* Added a mirror to elfutils
[ROCm/rocprofiler-systems commit: c11abc4ea1]
* Add Sphinx and Read the Docs configs
* Add documentation workflow configurations
* Changed macros verbprintf and verbprintf_bare so they write to stdout… (#346)
Flush stdout when listing keys + bump verbose level for GPU count
* Removing static version asserts. (#347)
It is causing failures on our internal builds
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
* Check for an empty vector before popping (#350)
Protect from possible seg. fault
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
* Add release links to installation.md (#351)
* Initial infrastructure rework for Omnitrace refactoring and a rewrite of the What is file
* Add files in conceptual section, along with images and infrastructure changes.
* Formatting and style fixes for files in conceptual directory
* Add quick start install guide and fix spelling errors in other files
* Add install document and fix code tags. Infrastructure changes
* Add two how-to guides along with infra changes and spelling fixes
* Add two new how to files and fix errors in the last commit
* Fix spelling mistakes
* Add new how to file on causal profiling and infra changes.
* Add how to file on interpreting Omnitrace output, fixes, and images
* Add remaining how-to guides and reference materials along with fixes and infrastructure
* Add YouTube file and fix spelling and formatting
* Fix a few loose ends and add link to license page
* Add Sphinx and Doxygen infrastructure and some additional corrections
* Update rocm-docs-core
* Fix Doxyfile
* Fix path to API header files
* Run doxysphinx in conf.py
* Add back custom css for doxygen
* Remove doxygenlayout
* Add api to toc
* Update Doxyfile
Generate from source .in
* Proofreading edits and other changes
* Add .gitignore for Doxygen and remove deprecated words and typos
* Fix one additional typo
* Turn off dot
* Update doxyfile strip from path
* Workflow, submodules, and thread info Updates (#352)
* Update CI workflows
- use node20 workflow packages
* Update tests/source/CMakeLists.txt
- Use OMNITRACE_TRACE and OMNTRACE_PROFILE instead of perfetto/timemory
* Update timemory submodule
- argparse: requires -> required
- parse callbacks
* Update thread_info.cpp
- fix causal::delay::get_local usage
* Update timemory submodule
* Update kokkos submodule
- release 3.7.02
* Revert opensuse.yml and ubuntu-bionic.yml to use node16 workflows
* Update docs.yml
* ROCm 6.1 Installers (#349)
* Add ROCm 6.1 to packages
* Bump version to 1.11.3
* Add 6.1 support to the docker build support.
Simplified this by adding 6.* to case statements, now that repo links have been standardized.
* Update timemory submodule (#354)
- fix argparse::argument::required template deduction
* Build omnitrace-rt library (#355)
* Build omnitrace-rt library
- Explicitly build dyninstAPI_RT as omnitrace-rt so that the SONAME in the ELF is omnitrace-rt instead of dyninstAPI_RT
- Create symbolic link lib/omnitrace/libdyninstAPI_RT.so which points to lib/libomnitrace-rt.so
- Simplify build tree location of libomnitrace-rt.so since it is ../lib from the bin directory even in the build tree
- Update dyninst submodule with minor tweaks to dyninstAPI_RT/CMakeLists.txt
* Update source/lib/omnitrace-rt/cmake/platform.cmake
* Use ftpmirror.gnu.org instead of ftp.gnu.org
- in timemory and dyninst submodules
- minor .clang-tidy tweak
* Executables append omnitrace library directory to LD_LIBRARY_PATH (#356)
- omnitrace-run, omnitrace-sample, and omnitrace-causal now automatically append the LD_LIBRARY_PATH with the directory containing the omnitrace libraries
- this helps ensure that binary rewritten exes can resolve omnitrace-rt library location
* Fix a few typos and formatting issues
* Additional fixes and minor formatting changes.
* More fixes and minor formatting changes.
* Complete second proofreading with fixes and minor formatting changes.
* Make changes to table of contents and disable linting
* Update links in the README doc to reflect the new structure.
* Align intro on the Omnitrace index page with the first paragraph of the what-is page
* Changes and edits based on review comments
* Additional changes and edits based on external review
* Additional updates and changes from the external review of Omnitrace
* Additional changes based on the external review
* New round of edits based on the external review
* Additional edits based on the external review
* Changes to address comments from the internal review
* Correct to the RHEL SELinux note in the troubleshooting guide
* One additional change to the development guide code example
* Move troubleshooting to post-install of install.rst and other minor edits.
* Remove troubleshooting page and modify new post-install troubleshooting section on install.rst
* Refactor the how Omnitrace works page into seperate topics and redo infrastructure
* API ToC changes
* Additional API and ToC changes
* Back out API and ToC changes and update requirements.txt
* Additional API and ToC changes
* Add commit for signing purposes
* Add ElfUtils and BinUtils Download URL Overrides (#358)
* Add CMake CACHE Variable ElfUtils_DOWNLOAD_URL
Used to override the default URL to download ElfUtils from.
Useful for internal builds
Also, include a mirror to fallback to if the override URL fails.
* Update timemory submodule
Updating to include the BINUTIL_DOWNLOAD_URL override cmake
variable.
---------
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
* Remove Ubuntu 18.04 and SUSE 15.2
* Update checkout action to v4
* Add `docs/**` to `paths-ignore`
Document location is being refactored.
* Modified submodules dyninst and timemory. (#361)
---------
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
Co-authored-by: Peter Jun Park <peter.park@amd.com>
Co-authored-by: ajanicijamd <Aleksandar.Janicijevic@amd.com>
Co-authored-by: David Galiffi <David.Galiffi@amd.com>
Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
Co-authored-by: Sam Wu <22262939+samjwu@users.noreply.github.com>
[ROCm/rocprofiler-systems commit: 0689797736]
* Add CMake CACHE Variable ElfUtils_DOWNLOAD_URL
Used to override the default URL to download ElfUtils from.
Useful for internal builds
Also, include a mirror to fallback to if the override URL fails.
* Update timemory submodule
Updating to include the BINUTIL_DOWNLOAD_URL override cmake
variable.
---------
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
[ROCm/rocprofiler-systems commit: b0b1dca21e]
- omnitrace-run, omnitrace-sample, and omnitrace-causal now automatically append the LD_LIBRARY_PATH with the directory containing the omnitrace libraries
- this helps ensure that binary rewritten exes can resolve omnitrace-rt library location
[ROCm/rocprofiler-systems commit: 18833a0a5e]
* Build omnitrace-rt library
- Explicitly build dyninstAPI_RT as omnitrace-rt so that the SONAME in the ELF is omnitrace-rt instead of dyninstAPI_RT
- Create symbolic link lib/omnitrace/libdyninstAPI_RT.so which points to lib/libomnitrace-rt.so
- Simplify build tree location of libomnitrace-rt.so since it is ../lib from the bin directory even in the build tree
- Update dyninst submodule with minor tweaks to dyninstAPI_RT/CMakeLists.txt
* Update source/lib/omnitrace-rt/cmake/platform.cmake
* Use ftpmirror.gnu.org instead of ftp.gnu.org
- in timemory and dyninst submodules
- minor .clang-tidy tweak
[ROCm/rocprofiler-systems commit: 0cf017251e]
* Add ROCm 6.1 to packages
* Bump version to 1.11.3
* Add 6.1 support to the docker build support.
Simplified this by adding 6.* to case statements, now that repo links have been standardized.
[ROCm/rocprofiler-systems commit: 99c9618c39]
Fix HIP-API CTest failure
Check if stack is empty before popping data off of it.
Signed-off-by: Aleksandar Janicijevic <Aleksandar.Janicijevic@amd.com>
[ROCm/rocprofiler-systems commit: f0bd9126a5]
* CPACK: Enable features to generate dependency lists
For DEBIAN, enable CPACK_DEBIAN_PACKAGE_SHLIBDEPS to generate a package
dependency list.
For RPM, enable CPACK_RPM_PACKAGE_AUTOREQPROV to automatically generate
lists of shared libraries that this package requires and provides.
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
* Manually derive RPM provides list for libunwind.
[Why]
`dnf install` fails dependency resolution
[How]
Auto-generation did not add it to the "provides" list, despite it being
included in the package when OMNITRACE_BUILD_LIBUNWIND is enabled. So,
manually include these in the "CPACK_RPM_PACKAGE_PROVIDES" parameter.
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
* Fix cmake-format linting
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
---------
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
[ROCm/rocprofiler-systems commit: 129580b416]
* Update CPack packaging variables.
Look for ROCM_LIBPATH_VERSION environment variable to patch the
CPACK_PACKAGE_VERSION.
Add some status logs to output packaging information.
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
* Add CPack variables to "omnitrace_add_feature".
---------
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
[ROCm/rocprofiler-systems commit: 6bcd1d36cc]
* Fix omnitrace-avail component list
- remove omnitrace components from `omnitrace-avail -C` since these are no-ops in OMNITRACE_TIMEMORY_COMPONENTS
* Fix omnitrace-avail-filter-wall-clock-available test
[ROCm/rocprofiler-systems commit: 77d52814e9]
Update roctracer.cpp
- fix call to hip_exec_activity_callbacks when more runtime threads than compile time max
[ROCm/rocprofiler-systems commit: edd6f57cf3]
* Update build-docker.sh
- support rocm 6.0
* Update cpack workflow
- support rocm 6.0
* Update CI testing workflow paths-ignore
- changes to {docs,cpack,containers,formatting}.yml and docker do not require testing
* Update docker for OpenSUSE
- always use --non-interactive with zypper
- tweak to PERL_REPO when OS version >= 15.4
[ROCm/rocprofiler-systems commit: cfaace38a8]
Update lib/core/gpu.cpp
- use const std::array instead of constexpr std::array due to internal compiler errors on systems with older GCC compilers
[ROCm/rocprofiler-systems commit: adefde707c]
* Update lib/core/config.cpp
- Add OMNITRACE_ROCTRACER_HIP_API_BACKTRACE option
* Update lib/omnitrace/library/roctracer.cpp
- support perfetto debug annotation of backtrace in HIP API call
* Fix backtrace resolution and ordering in UI (#1)
* Fix backtrace resolution for non-omnitrace libraries
* Nicer Perfetto UI on long backtraces
* Make Perfetto annotation consistent
* clang-format
---------
Co-authored-by: Tal Ben-Nun <tbennun@users.noreply.github.com>
[ROCm/rocprofiler-systems commit: 608287ddad]
* Fix roctracer_flush_activity
- invoke roctracer_flush_activity() before disabling domains
* create comp::roctracer::flush()
- real issue was the global state when roctracer_flush_activity() was called
* formatting
* Update lib/omnitrace/library/components/roctracer.hpp
- provide definition of comp::roctracer::flush when OMNITRACE_USE_ROCTRACER is not defined
* omnitrace.cfg -> perfetto.cfg
- rename provided perfetto config file (omnitrace.cfg) to perfetto.cfg to avoid confusion
* Update lib/core
- gpu.hpp: defines for OMNITRACE_USE_{HIP,ROCTRACER,ROCPROFILER,ROCM_SMI}
- gpu.cpp
- include core/hip_runtime.hpp
- fix serialization of hipDeviceProp_t
- add hip_runtime.hpp
- ensure proper inclusion of hip_runtime.h
- add rccl.hpp
- ensure proper inclusion of rccl.h
* Update lib/omnitrace/library
- rcclp.cpp
- update includes for rccl
- roctracer.hpp
- update includes for hip_runtime
- components/comm_data.hpp
- update includes for rccl
- components/rcclp.hpp
- update includes for rccl
* Update bin/omnitrace-avail/avail.cpp
- update includes for hip_runtime
* Update examples/rccl/CMakeLists.txt
- fix find_package for rccl when CI enabled
* Update CMakeLists.txt
- set cmake policy CMP0135 to NEW for cmake >= 3.24
- Enable DOWNLOAD_EXTRACT_TIMESTAMP with ExternalProject_Add + URL download method
* Update timemory submodule
* Update pybind11 submodule
* Update pybind11 submodule
* Update lib/core/rccl.hpp
- include rccl.h only if OMNITRACE_USE_RCCL > 0
* Update lib/core/{gpu,hip_runtime}.hpp
* Update lib/core/gpu.cpp
- reintroduce some ppdefs
* Update lib/core/gpu.cpp
- fix ifdef on OMNITRACE_HIP_VERSION
* Update lib/core/gpu.cpp
- fix static assert for OMNITRACE_HIP_VERSION_MINOR when HIP version 4.x or older (unreliable minor versions)
* Update lib/core/gpu.cpp
- fix ifdef on OMNITRACE_HIP_VERSION
* Update lib/core/config.cpp
- disable OMNITRACE_PERFETTO_COMBINE_TRACES by default
* Update lib/core/perfetto.cpp
- if unable to open perfetto temp file, return the ReadTraceBlocking()
* Update lib/core/config.*
- flush tmpfile before closing
[ROCm/rocprofiler-systems commit: 7bc50f5a0a]
The CI test machines only have 2 MPI slots. MPI tests were failing
when requesting 4 CPUs. Update these tests to request 2 CPUs.
[ROCm/rocprofiler-systems commit: aeb346b6d6]
* Tests for exceeding OMNITRACE_MAX_THREADS
- tests which exceeds OMNITRACE_MAX_THREADS value for thread creation
* CMake Formatting.cmake update
- include source files in /tests/source directory
* Add unknown-hash= to OMNITRACE_ABORT_FAIL_REGEX
- fail if a timemory hash is not resolved to a name
* Tests for exceeding OMNITRACE_MAX_THREADS
- update
* omnitrace-sample update
- remove env disabling of critical-trace and process-sampling
* core library update
- make_unique in concepts.hpp
- add OMNITRACE_USE_ROCM_SMI to "process_sampling" category
- remove forced disabling of critical-trace in sampling mode
- parentheses for OMNITRACE_PREFER
- use tim::get_hash_id instead of tim::get_combined_hash_id
* core library update (containers)
- added aligned_static_vector.hpp
- similar to static_vector.hpp but attempts to align to cache line size
- alignment template parameter for stable_vector
- added missing aliases in static_vector
- consistent with aligned_static_vector aliases
* thread_info update
- track the peak number of threads created
- thread_info::get_peak_num_threads() returns the peak number of threads
* thread_data update
- generic thread_data inherits from base_thread_data
- thread_data reworked to support dynamic expansion
- base_thread_data updated to invoke private_instance() function
- thread_data<optional<T>> uses stable_vector aligned to cache line width
- thread_data<identity<T>> uses stable_vector aligned to cache line width
- thread_data for optional and identity provide private private_instance function + friend to base_thread_data
- component_bundle_cache<T> is now thread_data<component_bundle_cache_impl<T>>
* causal update
- thread_data<T>::instances -> thread_data<T>::instance(construct_on_thread{ ... })
- loop over max_supported_threads (constexpr) -> loop over thread_info::get_peak_num_threads()
- tim::get_combined_hash_id -> tim::get_hash_id
- update progress_bundle usage to new thread_data API
* backtrace/backtrace_metrics component update
- backtrace_metrics update
- update to new thead_data API
- add thread CPU time row in perfetto
- fix potential bug when rusage categories are disabled
- fix bug in operator-= not subtracting cpu time of rhs
- backtrace update
- skip all child call-stack below 'tim::openmp::' if sampling_keep_internal = false
* pthread_gotcha component update
- pthread_gotcha::shutdown() invokes pthread_create_gotcha::shutdown()
* pthread_create_gotcha component update
- minor tweak to {start,stop}_bundle functions: pass in thread id
- update to new thread_data API
- track native handles of internal threads
- implement system with pthread_kill to stop dangling bundles
* rocprofiler/roctracer component update
- update to new thread_data API
- loop over max_supported_threads (constexpr) -> loop over thread_info::get_peak_num_threads()
* critical trace (library) update
- update to new thread_data API
- tim::get_combined_hash_id -> tim::get_hash_id
* coverage update
- update to new thread_data API
* tasking update
- update to new thread_data API
- loop over max_supported_threads (constexpr) -> loop over thread_info::get_peak_num_threads()
* roctracer update
- update to new thread_data API
- loop over max_supported_threads (constexpr) -> loop over thread_info::get_peak_num_threads()
* rocm_smi update
- update to new thread_data API
* runtime.cpp update
- update to new thread_data API
* sampling.cpp update
- update to new thread_data API
- loop over max_supported_threads (constexpr) -> loop over thread_info::get_peak_num_threads()
* ompt.cpp update
- invoke pthread_gotcha::shutdown before invoking OMPT finalize function
- this prevents signals from being delivered to OpenMP threads
* tracing.hpp and tracing.cpp update
- replace get_timemory_hash_{ids,aliases} functions with copy_timemory_hash_ids function
- update to new thread_data API
- loop over max_supported_threads (constexpr) -> loop over thread_info::get_peak_num_threads()
- tim::get_combined_hash_id -> tim::get_hash_id
- improvements to + error checking in thread_init function
* library.cpp update
- move copying timemory hash id/aliases to tracing.cpp
- update to new thread_data API
- loop over max_supported_threads (constexpr) -> loop over thread_info::get_peak_num_threads()
* Update BuildSettings.cmake
- add -Wno-interference-size to suppress warning about use of std::hardware_destructive_interference
* Update fork example
- improve scheme for waiting on child processes via waitpid instead of wait
- support running main routine multiple times
- push/pop regions in child process
* Update lib/common/defines.h.in
- allow use to specify misc values via -D <name>=<value>
- OMNITRACE_CACHELINE_SIZE
- OMNITRACE_CACHELINE_SIZE_MIN
- OMNITRACE_ROCM_MAX_COUNTERS
- remove unused defines
- OMNITRACE_ROCM_LOOK_AHEAD
- OMNITRACE_MAX_ROCM_QUEUES
* Update rocprofiler.hpp
- OMNITRACE_MAX_ROCM_COUNTERS -> OMNITRACE_ROCM_MAX_COUNTERS
* Update aligned_static_vector
- set cacheline_align_v from max of OMNITRACE_CACHELINE_SIZE and OMNITRACE_CACHELINE_SIZE_MIN
* Update tracing.cpp
- acquire locks for updating main hash ids/aliases
- only propagate ids/aliases when finalizing
* Update pthread_create_gotcha.cpp
- make sure hash for "start_thread" exists on main thread
* Update causal end to end tests
- if OMNITRACE_BUILD_NUMBER is 1, set OMNITRACE_VERBOSE=0
[ROCm/rocprofiler-systems commit: 518c83e0f9]
jamesxu2
ajanicijamd
David Galiffi
dependabot[bot]
Jeffrey Novotny
Jonathan R. Madsen
Ben Richard
Tal Ben-Nun