Each thread has a thread-local record_pair_stack. The stack is
dynamically allocated on first use, but is not detroyed when the
thread exits.
Replaced record_pair_stack pointers with record_pair_stack instances,
the intances are constructed on first odr-use, and destructed when the
thread exits.
Also, converted the cb_journal and act_journal to instances.
Change-Id: I186ac29da477f194880a1ab599f4be5715a23063
Improve the roctx markers performance when the tracer is not engaged
(the application is not running with rocprof).
The performance of roctx push/pop, measured with:
-----------------------------------------------------------------------
auto start = std::chrono::steady_clock::now();
for (int i = 0; i < 10000000; ++i) {
roctxRangePush ("A");
roctxRangePop ();
}
auto end = std::chrono::steady_clock::now();
std::cout << "ns = " << std::chrono::nanoseconds(end - start).count()
/ 10000000 << std::endl;
-----------------------------------------------------------------------
w/o rocprof | with rocprof | commit
92ns | 770ns | 0d6e132: Cleanup CallbackTable::Get
28ns | 712ns | 6421bd5: Cleanup ROCTX's implementation
20ns | 664ns | 7f0e5e5: Remove the roctx range message...
6ns | 665ns | this commit
Change-Id: Id679dcbd0fb190a3179be98a9b2c1db151efee3d
The range message stack is mirrored in case ranges are pushed or popped
while tracing is stopped (by the tracer tool?). When a stop event is
reported, the tracer tool emits RangePop events by unwinding the stack,
then when the start event is reported, it emits RangePush events again
by unwinding the stack. The issue is that the RangePush events should
be emitted in reverse order.
For example:
RangePush(M1); RangePush(M2); \
TracerStop; RangePop; RangePop; \
...; \
TracerStart; RangePush(M2); RangePush(M1); \ <- In the wrong order
RangePop; RangePop;
It could be fixed by reversing the stack in RangeStackIterate but is it
worth it? The roctx range markers are supposed to be unintrusive so that
they can be left in the application even when it isn't being traced.
Simplifying the roctx API and reducing its added latency by removing
the range message stack mirroring seems like the better choise.
TODO: A future change should make roctx events immune to tracer start
and tracer stop requests. Or simply remove roctracer_start/stop.
Change-Id: Ie4d76afb5ce8d263848dcf1b599af394db56ddab
Remove thread_data_init. The C++ standard guarantees that the thread
local variable is initialized before its first odr-use and destructed
when the thread exits. Use a global initializer to set the reference
from the message stack instance in the map.
Remove roctracer_error_string. This does not belong to this library.
ROCTX does not expose errors to the application. The only functions
returning errors are returning -1 (Push/Pop).
Remove memory leaks due to strdup on the ranges messages. The memory
for the messages is guaranteed to be valid for the duration of the
callback, and it is the application's responsibility to strdup the
strings if it needs to extend the message's lifetime.
Add a lock to the RegisterApiCallback implementation. Iterating the
message stack map must be synchronized as a new thread could be adding
a new value to the map.
Change-Id: Iaf5b07ebc9efe4061cb01327d4c7034888727816
Make CallbackTable::Get return the callback_function/user_arg pair
as an actual return value instead of returning it through arguments
pointers.
Change-Id: Ia2dfcdad8c237a09620518ad67af94add47220da
At the end of the test, the tracer tool is unloaded and the active
memory pools are flushed. In the flush callback, to get the activity
operation string, the RocpLoader instance is neeeded, and if the
RocpLoader is not already loaded, it attempts to dlopen the rocprofiler
library.
Calling dlopen from a global destructor hangs because the dynamic
loader lock is already owned (e.g. by dlclose).
To temporarily work around the issue, instanciate the RocpLoader when
the activities needing it are enabled.
Change-Id: I712c66d88c43694fe53a95d6a61d7b22abb75262
System clock timestamps should only come from a single source:
util::timestamp_ns(). Externally, this function is exposed as
roctracer_get_timestamp() (used by the tracer tool).
Removed the now unused HSA Runtime Utilities which were never part
of the ROCtracer API.
Change-Id: I044b7f4da60fd8fdb771b0c877622a3143f0e815
KFDQMTest.BasicCuMaskingEven was previously in TEMP_GFX10_BLACKLIST and
was later added to BLACKLIST_GFX10_NV2X. shows this test
also has problems on NV1X, so move to BLACKLIST_GFX10 to blacklist for
all GFX10 until resolved.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I75426780a52172ed0f2468f0297241902da5002a
* Parallel overhead example with locks
* Support tracing mutex locking + more
- support wrapping pthread_mutex_lock
- support wrapping pthread_mutex_unlock
- support wrapping pthread_mutex_trylock
- get_perfetto_combined_traces setting
- OMNITRACE_TRACE_THREAD_LOCKS option
- ThreadState
- critical trace includes queue id
- enabled/disabled settings in timemory
- fix OMNITRACE_TIMEMORY_COMPONENTS
- fix reading config
- fix setting categories
- applied ThreadState::Internal in various places
- utility::get_filled_array
- utility::get_reserved_vector
- utility::get_thread_index
- fork_gotcha messages about forks
- split out some pthread_gotcha functionality into pthread_create_gotcha
- handle queue id in roctracer callbacks
* Update timemory and PTL submodules
* Misc CMake updates
- Includes fix to omnitrace-static-lib{gcc,stdcxx}
* Misc cleanup to pthread_mutex_gotcha and backtrace
* Fix to duplicate field in module_function json
* Improvement to debug messages
* omnitrace-dl and common improvements
- tweak to delimit
- common::ignore message
- common::join quoting of strings
- omnitrace_set_env ignores if inited and active
- omnitrace_set_mpi ignores if inited and active
* nsync for transpose example
* Fix to thread_deleter<void> functor invoke
* Fix thread state and HIP stream enums
showclocks/showclkfrq does not display pp_dpm_pcie values
in sriov. This fix adds pcie clocks to rsmi_clk_type_t
where rest of the clocks are present.
Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: I6d129ae412623b369c14456ae9781b2dbceb2139
This is consistent with KFD and has significantly better latency.
KFD is taking this as the definition of the SystemClockCounter.
Change-Id: I4c1b3bc58c738206265c55ebefd41356c013bfe5
Add BLACKLIST_GFX10 and BLACKLIST_GFX10_NV2X to reduce redundant
individual blacklists. Remove KFDDBGTest from kfdtest.exclude and
CMakeLists.txt as it is deprecated.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I5ffebb012ff8ef4bba5aadfeb3c7478fe43c65f1
DeviceHdpFlush was part of the prior TEMP_GFX10_BLACKLIST and is
soft-hanging after being re-added to the test list. Temporarily move
back to gfx10 blacklists until root-caused ().
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: Id37b7105566e8243465e4b21ebc2e2e8c4c79923
Add a new option --enable_pci_id to Prometheus plugin, which will map
the GPU index to the PCI Device Identifier.
Change-Id: I38a2a7e4841975da095391002397d4515ffb8e0d
This patch adds the following 4 missing GPU blocks to the SMI LIB:
-RSMI_GPU_BLOCK_MMHUB
-RSMI_GPU_BLOCK_PCIE_BIF
-RSMI_GPU_BLOCK_HDP
-RSMI_GPU_BLOCK_XGMI_WAFL
Signed-off-by: Ori Messinger <Ori.Messinger@amd.com>
Change-Id: Ia1ec6f53e195f4bf7b8f073d6bed4fdb6572e546
In a future change, the tracer API library (libroctracer64.so) will be
automatically registered as a tool library. Until then, explicitly
register it by adding it to the HSA_TOOLS_LIB environment variable.
Change-Id: I44d78ac38608e6da5edf04b498a73485f5609d06
hsa_rsrc_factory was only used to enumerate the agents types and pools.
The pools don't seem to be used by bin/mem_manager.py, so I only
ported the agent enumeration using hsa_iterate_agents.
Change-Id: Idd586aa13db303cf92962a6392771b7bf38b758f
In DeviceHdpFlush, isaBuffer was accidentally used instead of isaBuffer0
during LLVM re-work--revert.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I98d2322e772f821a39505bb336ceb4e6cd8722ef
Compilers doesn't see assert as a usage of the variables, I added [[maybe_unused]] to the variables that are used only in assert to make sure that the compiler is skipping them in the check. Note: [[maybe_unused]] is introduced in C++17
Change-Id: I96bb53cb2ab55ee7120681c2d279271c0075095d
1) The Entry's state was published after making the record avaiable,
so a thread flushing the records could see an unitialized record.
2) data_ and write_pointer_ could become out of sync. write_pointer_
could be indexing into another buffer than what data_ was pointing
to.
3) GetEntry could get a nullptr free_buffer_ because multiple threads
could acquire the work_mutex_ before the work_thread_ could wake up,
or between allocate_worker's loop iterations.
Change-Id: I6f0a015557888eeeaa75a8bce7fde8de276d11dd
A trace buffer is used to efficiently store synchronous event records
so that they can be processed later, possibly in a different thread,
when the buffer is flushed. This helps reduce the latency added by
tracing API calls.
The API does not need to use trace buffers as synchronous events are
directly reported to the client with callbacks, and asynchronous events
(activities) are saved in memory pools.
The implentation of HSA asynchronous memory copy activities was using
a trace buffer shared with the tracer tool to write the records to a
file (async_copy_trace.txt), instead of using a memory pool and
reporting the activity to the client.
Removed the asynchronous memory copies trace buffer, and updated
hsa_async_copy_handler to use the pool specified when the activity
was enabled.
Updated the tracer tool to read HSA_OP_ID_COPY records out of the
default memory pool and write them to async_copy_trace.txt.
Move trace_buffer.h to test/tool as tracer_tool.cpp is now the only
file using it.
Change-Id: Ida95aba2eaf3c3f2a979ed6c2b060374017b7424
Fixing the RPATH skip & Removed the export line from the build.sh as we have find_library with giving it a path to /opt/rocm & easy to use build.sh
Change-Id: I1ac5b51eafb54ef0359bf6fb55f2fe2d39a6cafa
Ammar ELWazir
Jonathan R. Madsen
Laurent Morichetti
Sean Keely
Graham Sider
Ranjith Ramakrishnan
Divya Shikre
Hao Zhou
Bill(Shuzhou) Liu
Ori Messinger
David Yat Sin