* Updates:
- [CLI] Updated --showmemuse:
-> Add VRAM%, provide better context as "GPU Allocated Memory (VRAM%)"
-> Update "GPU memory use (%)" as
"GPU Memory Read/Write Activity(%)"
- [CLI] Updated --showmaxpower and rocm-smi (no arg)
-> Rounding was inconsistent with values past decimal.
This provides the floor value of the device
Change-Id: Ib76dea2cb8483a1d7f53df675b0a94d8d01c81b9
Signed-off-by: Charis Poag <Charis.Poag@amd.com>
* Add ToolsApiTable
Add ToolsApiTable wrapping for
scratch memory tracking
* Add initial support for scratch memory tracking
Buffering is implemented
* cmake formatting (cmake-format) (#525)
Co-authored-by: MythreyaK <MythreyaK@users.noreply.github.com>
* source formatting (clang-format v11) (#524)
Co-authored-by: MythreyaK <MythreyaK@users.noreply.github.com>
* Add callback tracing for scratch
Fixed the error where scratch tracking init was called irrespective of whether any client requested for it
* Apply suggestions from code review
Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
* Fix tools api copy/update
Table were saved/updated incorrectly in previous
commit. Also adds passing user data through the callback
* Fix OpKind sequence for scratch tracking
Previously scratch was using OpKind from rocprofiler-sdk, but
templates were instantiated using API ID. These differ by 1
* Integration tests for scratch reporting
Added buffer and callback integration tests for scratch reporting
* source formatting (clang-format v11) (#550)
Co-authored-by: MythreyaK <26112391+MythreyaK@users.noreply.github.com>
* cmake formatting (cmake-format) (#551)
Co-authored-by: MythreyaK <26112391+MythreyaK@users.noreply.github.com>
* python formatting (black) (#549)
Co-authored-by: MythreyaK <26112391+MythreyaK@users.noreply.github.com>
* CI fixes
* source formatting (clang-format v11) (#554)
Co-authored-by: MythreyaK <26112391+MythreyaK@users.noreply.github.com>
* Update api
Rebase on main and updates based on PR feedback
* Update scratch reporting and address PR comments
- Added agent id to buffer records
- Updated `test_internal_correlation_ids` - Is almost identical to
one in async-copy
- Updated scratch test to check for agent id
- Updated queue id serialization in callback records (prints
handle as nested key)
- Remove `marker_api_traces` from scratch `test_internal_correlation_ids`
validation test
- Rename `amd_tools_api` to `scratch_memory`
- Added doxygen comments
- Remove scratch callback from `tool.cpp`
- Replace assert with `LOF_IF` in `scratch_memory.cpp`
* Update tools table
Changed to match up with changes to hsa tables in main branch
* Rework scratch memory structure
* Update tests
- Added suggestions from PR review, and updated tests accordingly
* Misc cleanup
* Update scratch test
As of Apr 4th, `hsa_amd_agent_set_async_scratch_limit` is disabled.
Note,
> This API: `hsa_amd_agent_set_async_scratch_limit` is currently
> disabled. We need some changes in CP firmware to be able to do this
> and these changes are not ready yet.
> With the current code, you will also not get notifications for
> alternate-scratch allocations because this feature has been disabled
> while CP firmware is making additional changes
> We are hoping to have that feature enabled by ROCm-6.3
* Minor update to lib/rocprofiler-sdk/internal_threading.*
- delay destruction of shared_ptrs of the tasks to prevent rare (but possible) data race on the destruction of the shared_ptr
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: MythreyaK <MythreyaK@users.noreply.github.com>
Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
support four modes:
(0) - no coloring and no loglevel delimiters
(1) - colored loglevel delimiters
(2) - non-colored loglevel delimiters
(3) - fully colored messages for all levels besides INFO
Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
* Add debug printing for write interceptor injected packets
Adds debug printing for write interceptor injected
packets. All packets that pass through the write
intercepter while enabled will be printed.
Only executes/prints when the environment variable
GLOG_v is set to 2 or higher (otherwise it is a no-op
and the expression is not evaluated).
* source formatting (clang-format v11) (#675)
Co-authored-by: bwelton <1683479+bwelton@users.noreply.github.com>
* Changes on fmt location
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: bwelton <1683479+bwelton@users.noreply.github.com>
This reverts commit 5c520f4544c654e5f18e05cabd1c63d64473cfab.
Reason for revert: This patch is introducing a synchronization related bug in Unit_hipGetSetDevice_MultiThreaded testcase.
Change-Id: I367e4d4f1d75b21658ac1127c58982894a97cedb
* Wait for all mem copies to complete before destroying.
* Update source/lib/rocprofiler-sdk/hsa/async_copy.cpp
Co-authored-by: Ammar ELWazir <ammar.elwazir@amd.com>
* Update async_copy.cpp
---------
Co-authored-by: Ammar ELWazir <ammar.elwazir@amd.com>
- update lib/rocprofiler-sdk/internal_threading.*
- use PTL::TaskManager instead of PTL::TaskGroup
- easier to handle for our needs
- eliminate data race in rocprofiler_flush_buffer
- combine memory management of TaskManager and ThreadPool
The build tree was missing a level of nesting, causing diversions based
on in-tree/out-of-tree use.
KR: Also fixed kfdtest paths
Change-Id: I8638b6d6227daabddd8eaa2aa387ba578b8dfab8
Signed-off-by: Stella Laurenzo <stellaraccident@gmail.com>
Signed-off-by: Kent Russell <kent.russell@amd.com>
Gopesh Bhardwaj
Charis Poag
Mythreya
Ammar ELWazir
WhiskyAKM
Junyi Hou
Daniel Martinez
Galantsev, Dmitrii
Karl W Schulz
Benjamin Welton
Giovanni LB
Shweta Khatri
Jonathan R. Madsen
dependabot[bot]
Benjamin Welton
Stella Laurenzo
coleramos425