[rocprofiler-sdk] Buffer finalization fixes and HSA ABI 0x09 support (#2318)

* [rocprofiler-sdk] Fix buffer flush ordering and sanitizer CI improvements

Buffer Pool Design
------------------
Replace the fixed array-based double buffer with a dynamic pool design to
fix race conditions that caused "internal correlation id was retired
prematurely" errors.

The original design had a race where flush callbacks could be delivered
out-of-order: when buffer 0 fills and begins flushing, writes go to
buffer 1. If buffer 1 fills before buffer 0's flush completes, the
buffer index wraps back to 0 (which may still be flushing). Independent
flush tasks submitted to the thread pool can complete out of order.

The new pool design:
- Uses a std::deque of buffer instances that grows as needed
- Allocates buffers from the pool when the current buffer needs to flush
- Serializes flushes with a mutex to ensure FIFO callback ordering
- Returns buffers to the pool after flush completion
- Eliminates the race between buffer selection and write operations

New Unit Tests
--------------
- buffer_correlation_ordering.cpp: Tests that API records are always
  delivered before their corresponding retirement records
- buffer_ordering_stress.cpp: Stress tests buffer flush ordering under
  high contention with multiple threads rapidly filling buffers

HSA Tool Hooks
--------------
Added hsa_tool_hooks.cpp/hpp to register an HSA OnUnload callback that
waits for pending flush tasks before tool finalization, preventing
"retired prematurely" errors during HSA shutdown.

Sanitizer Improvements
----------------------
- LSAN: Set fast_unwind_on_malloc=1 to prevent deadlock in libgcc unwinder
- LSAN: Added suppressions for external tools (liblzma, liblsan, seq, strdup)
- TSAN: Added suppression for false positive on C++11 thread-safe static
  initialization in create_write_functor
- ASAN/UBSAN: Added patterns for known issues in HSA runtime, HIP, perfetto
- Disabled attachment tests for sanitizers due to library preloading issues

Other Fixes
-----------
- Thread-trace agent test: Use heap-allocated callback state
- Correlation ID: Refactored reference counting and finalization ordering

* [rocprofiler-sdk] Revert buffer pool design changes

Revert buffer.cpp and buffer.hpp to the original double-buffer
design from develop branch. The pool-based redesign introduced
concerns about:
- Signal safety (mutex vs atomic_flag)
- API changes (flush() return type)
- Complexity of the new design

This revert removes:
- Dynamic buffer pool with std::deque
- std::mutex/condition_variable synchronization
- buffer_correlation_ordering.cpp test
- buffer_ordering_stress.cpp test

The underlying buffer flush ordering issue will need to be
addressed with a different approach that preserves the original
API and synchronization characteristics.

* [rocprofiler-sdk] Consistent fini_status checks to prevent correlation ID creation during finalization

- Revert TOCTOU CAS loop change in sub_ref_count() - not needed with consistent checks
- Add fini_status check in correlation_tracing_service::construct() with ROCP_CI_LOG warning
- Add nullptr checks at all construct() call sites (queue.cpp, async_copy.cpp, memory_allocation.cpp)
- Change all 'get_fini_status() > 0' to '!= 0' for consistent behavior:
  - hsa/queue.cpp (lines 105, 210)
  - hsa/async_copy.cpp (line 344)
  - hsa/hsa_barrier.cpp (line 43)
  - buffer.cpp (lines 107, 138, 185)

This ensures no correlation IDs are created once finalization starts (fini_status != 0),
preventing races between finalization and ongoing tracing operations.

* [rocprofiler-sdk] Replace arrival-order checks with timestamp-based temporal validation

Buffer records are not guaranteed to arrive in any specific order. Tests and
samples should use timestamps for temporal ordering validation instead.

Changes:
- samples/external_correlation_id_request: Replace 'retired prematurely' arrival
  order check with timestamp-based validation that retirement timestamp >=
  max(end_timestamps) for records with the same correlation ID
- tests/external_correlation.cpp: Remove EXPECT_GT(corr_id, last_corr_id) check
- tests/registration.cpp: Remove EXPECT_GT(corr_id, last_corr_id) check
- tests/roctx.cpp: Remove EXPECT_GT(corr_id, last_corr_id) check

Correlation IDs are not guaranteed to be monotonically increasing when records
are sorted by timestamp. Temporal ordering should be validated using the
timestamp fields in each record.

* [rocprofiler-sdk] Revert external/CMakeLists.txt SYSTEM keyword removal

Restore the SYSTEM keyword to target_include_directories for
rocprofiler-sdk-fmt to match develop branch.

* [rccl] Remove orphaned rocSHMEM gitlink

Remove orphaned submodule reference that was introduced during a merge
but never had a corresponding .gitmodules entry, causing CI failures
with "fatal: no submodule mapping found in .gitmodules".

* [rocprofiler-sdk] Add HSA ABI version 0x09 support

Add ABI checks for HSA_AMD_EXT_API_TABLE_STEP_VERSION 0x09 which
introduces hsa_amd_counted_queue_acquire and hsa_amd_counted_queue_release
functions (added in rocr-runtime SWDEV-561708).

* [rocprofiler-sdk] Handle finalized status gracefully in buffer flush operations

This commit consolidates fixes for handling the finalization status during
buffer flush operations across the SDK.

Changes:
- Tool and samples: Handle ROCPROFILER_STATUS_ERROR_FINALIZED gracefully
  when flushing buffers, as this indicates buffers were already flushed
  during finalization (not an error condition)
- HSA handlers (queue.cpp, async_copy.cpp, hsa_barrier.cpp): Use > 0 check
  for fini_status to allow operations during finalization process
- buffer.cpp: Revert fini_status checks to use > 0 for consistency
- correlation_id.cpp: Add fini_status > 0 check with ROCP_TRACE logging
  to prevent correlation ID creation after finalization starts

Files modified:
- source/lib/rocprofiler-sdk-tool/tool.cpp
- tests/tools/json-tool.cpp
- source/lib/rocprofiler-sdk/tests/registration.cpp
- source/lib/rocprofiler-sdk/tests/roctx.cpp
- samples/api_buffered_tracing/client.cpp
- samples/counter_collection/buffered_client.cpp
- samples/counter_collection/device_counting_async_client.cpp
- samples/external_correlation_id_request/client.cpp
- samples/pc_sampling/client.cpp
- source/lib/rocprofiler-sdk/buffer.cpp
- source/lib/rocprofiler-sdk/context/correlation_id.cpp
- source/lib/rocprofiler-sdk/hsa/queue.cpp
- source/lib/rocprofiler-sdk/hsa/async_copy.cpp
- source/lib/rocprofiler-sdk/hsa/hsa_barrier.cpp

* [rocprofiler-sdk] Remove hsa_tool_hooks and simplify buffer flush handling

Remove the hsa_tool_hooks infrastructure and simplify buffer flush calls
in samples and tools. The ERROR_FINALIZED handling was overly complex
and the hsa_tool_hooks OnUnload synchronization is no longer needed.

Changes:
- Remove hsa_tool_hooks.cpp/hpp and related registration.cpp code
- Simplify buffer flush calls in samples to use direct ROCPROFILER_CALL
- Simplify buffer flush in tool.cpp and json-tool.cpp
- Remove ERROR_FINALIZED special handling from test files

Co-Authored-By: Claude <noreply@anthropic.com>

* [rocprofiler-sdk] Fix output_stream move semantics to null source pointers

The default move constructor and move assignment operator for
output_stream did not null out the source's pointers after the move.
This caused double-close when the moved-from temporary was destroyed,
leading to use-after-free crashes (SIGSEGV in std::ostream::sentry).

Co-Authored-By: Claude <noreply@anthropic.com>

* [rocprofiler-sdk] Improve Perfetto trace writer and sanitizer configuration

- generatePerfetto.cpp: Move output_stream into shared_state to prevent
  use-after-free race conditions during Perfetto callback execution
- run-ci.py: Simplify and consolidate sanitizer environment variable
  configuration for better maintainability

Co-Authored-By: Claude <noreply@anthropic.com>

* [rocprofiler-sdk] Revert run-ci.py changes that broke sanitizer suppressions

The previous changes removed MEMCHECK_SANITIZER_OPTIONS which is required
for CTest to properly pass suppression files to the sanitizers during
memcheck runs.

Co-Authored-By: Claude <noreply@anthropic.com>

* Revert "[rccl] Remove orphaned rocSHMEM gitlink"

This reverts commit 1ad21003941355658fff8114fa27768f11a948f7.

* [rocprofiler-sdk] Revert registration.cpp changes

Revert changes to registration.cpp to match develop branch.

Co-Authored-By: Claude <noreply@anthropic.com>

* [rocprofiler-sdk] Remove suppression file content printing from run-ci.py

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix output_stream move ctor/assignment operator

* Fix erroneous revert of registration.cpp

* Fix handling of fini status in correlation ID construction

* [rocprofiler-sdk] Fix OMPT segfault during finalization

Add nullptr checks in OMPT tracing code to handle the case where
correlation_tracing_service::construct() returns nullptr during
finalization. This fixes segfaults in openmp-target-sample and
tests.integration.execute.openmp-tools.

The correlation ID construction now returns nullptr when fini_status > 0,
but the OMPT callbacks were not checking for this, causing crashes when
dereferencing the null pointer during OpenMP runtime shutdown.

Changes:
- event_common(): Return nullptr early if correlation ID is null
- event(): Check for nullptr before calling sub_ref_count()
- ompt_task_create_callback(): Return early if correlation ID is null
- ompt_task_schedule_callback(): Return early if correlation ID is null

* [rocprofiler-sdk] Fix HSA API tracing segfault during finalization

Add nullptr check in hsa_api_impl::functor after correlation ID
construction. During finalization, correlation_service::construct()
returns nullptr, and without this check the code would dereference
the null pointer when accessing corr_id->internal.

This fixes the SEGV at address 0x000000000008 (null + 8 byte offset)
that occurs when HSA async event threads call hsa_signal_destroy
during runtime shutdown after finalization has started.

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
This commit is contained in:
Benjamin Welton
2026-01-27 10:27:54 -08:00
committato da GitHub
parent 9a8942a89c
commit 1517a398bf
47 ha cambiato i file con 577 aggiunte e 256 eliminazioni
@@ -360,6 +360,10 @@ ROCPROFILER_ENUM_LABEL(ROCPROFILER_HSA_AMD_EXT_API_ID_hsa_amd_portable_export_dm
ROCPROFILER_ENUM_LABEL(ROCPROFILER_HSA_AMD_EXT_API_ID_hsa_amd_ais_file_write);
ROCPROFILER_ENUM_LABEL(ROCPROFILER_HSA_AMD_EXT_API_ID_hsa_amd_ais_file_read);
# endif
# if HSA_AMD_EXT_API_TABLE_STEP_VERSION >= 0x09
ROCPROFILER_ENUM_LABEL(ROCPROFILER_HSA_AMD_EXT_API_ID_hsa_amd_counted_queue_acquire);
ROCPROFILER_ENUM_LABEL(ROCPROFILER_HSA_AMD_EXT_API_ID_hsa_amd_counted_queue_release);
# endif
#endif
#if HSA_AMD_EXT_API_TABLE_MAJOR_VERSION == 0x01
@@ -383,6 +387,8 @@ static_assert(ROCPROFILER_HSA_AMD_EXT_API_ID_LAST == 73);
static_assert(ROCPROFILER_HSA_AMD_EXT_API_ID_LAST == 74);
# elif HSA_AMD_EXT_API_TABLE_STEP_VERSION == 0x08
static_assert(ROCPROFILER_HSA_AMD_EXT_API_ID_LAST == 76);
# elif HSA_AMD_EXT_API_TABLE_STEP_VERSION == 0x09
static_assert(ROCPROFILER_HSA_AMD_EXT_API_ID_LAST == 78);
# else
# if !defined(ROCPROFILER_UNSAFE_NO_VERSION_CHECK) && \
(defined(ROCPROFILER_CI) && ROCPROFILER_CI > 0)
@@ -125,6 +125,10 @@ typedef enum rocprofiler_hsa_amd_ext_api_id_t // NOLINT(performance-enum-size)
ROCPROFILER_HSA_AMD_EXT_API_ID_hsa_amd_ais_file_write,
ROCPROFILER_HSA_AMD_EXT_API_ID_hsa_amd_ais_file_read,
# endif
# if HSA_AMD_EXT_API_TABLE_STEP_VERSION >= 0x09
ROCPROFILER_HSA_AMD_EXT_API_ID_hsa_amd_counted_queue_acquire,
ROCPROFILER_HSA_AMD_EXT_API_ID_hsa_amd_counted_queue_release,
# endif
#endif
ROCPROFILER_HSA_AMD_EXT_API_ID_LAST,
@@ -1464,6 +1464,22 @@ typedef union rocprofiler_hsa_api_args_t
int32_t* status;
} hsa_amd_ais_file_read;
# endif
# if HSA_AMD_EXT_API_TABLE_STEP_VERSION >= 0x09
struct
{
hsa_agent_t agent;
hsa_queue_type_t type;
hsa_amd_queue_priority_t priority;
void (*callback)(hsa_status_t status, hsa_queue_t* source, void* data);
void* data;
uint64_t flags;
hsa_queue_t** queue;
} hsa_amd_counted_queue_acquire;
struct
{
hsa_queue_t* queue;
} hsa_amd_counted_queue_release;
# endif
#endif
} rocprofiler_hsa_api_args_t;
@@ -98,5 +98,15 @@ add_string_entry(std::string_view name)
return _hash_v;
}
// Clear string entry cache for attach/detach cycles
void
clear_string_entries()
{
if(!get_string_array()) return;
auto _lk = std::unique_lock<std::shared_mutex>{get_sync()};
get_string_array()->clear();
}
} // namespace common
} // namespace rocprofiler
@@ -38,5 +38,9 @@ get_string_entry(size_t hash);
size_t
add_string_entry(std::string_view name);
// Clear string entry cache (for attach/detach)
void
clear_string_entries();
} // namespace common
} // namespace rocprofiler
@@ -39,6 +39,7 @@
#include <future>
#include <iostream>
#include <map>
#include <memory>
#include <thread>
#include <unordered_map>
#include <utility>
@@ -1204,44 +1205,46 @@ write_perfetto(
tracing_session->FlushBlocking();
tracing_session->StopBlocking();
auto filename = std::string{"results"};
auto ofs = get_output_stream(ocfg, filename, ".pftrace");
auto amount_read = std::atomic<size_t>{0};
auto is_done = std::promise<void>{};
auto _mtx = std::mutex{};
auto _reader = [&ofs, &_mtx, &is_done, &amount_read](
::perfetto::TracingSession::ReadTraceCallbackArgs _args) {
auto _lk = std::unique_lock<std::mutex>{_mtx};
if(_args.data && _args.size > 0)
{
ROCP_TRACE << "Writing " << _args.size << " B to trace...";
// Write the trace data into file
ofs.stream->write(_args.data, _args.size);
amount_read += _args.size;
}
ROCP_INFO_IF(!_args.has_more && amount_read > 0)
<< "Wrote " << amount_read << " B to perfetto trace file";
if(!_args.has_more) is_done.set_value();
struct read_trace_state
{
std::mutex mtx{};
std::atomic<size_t> amount_read{0};
output_stream ofs{};
};
auto state = std::make_shared<read_trace_state>();
state->ofs = get_output_stream(ocfg, std::string{"results"}, ".pftrace");
for(size_t i = 0; i < 2; ++i)
{
ROCP_TRACE << "Reading trace...";
amount_read = 0;
is_done = std::promise<void>{};
auto is_done = std::make_shared<std::promise<void>>();
auto _reader = [state, is_done](::perfetto::TracingSession::ReadTraceCallbackArgs _args) {
auto _lk = std::unique_lock<std::mutex>{state->mtx};
if(_args.data && _args.size > 0)
{
ROCP_TRACE << "Writing " << _args.size << " B to trace...";
// Write the trace data into file
state->ofs.stream->write(_args.data, _args.size);
state->amount_read += _args.size;
}
ROCP_INFO_IF(!_args.has_more && state->amount_read > 0)
<< "Wrote " << state->amount_read << " B to perfetto trace file";
if(!_args.has_more) is_done->set_value();
};
tracing_session->ReadTrace(_reader);
is_done.get_future().wait();
is_done->get_future().wait();
}
ROCP_TRACE << "Destroying tracing session...";
tracing_session.reset();
ROCP_TRACE << "Flushing trace output stream...";
(*ofs.stream) << std::flush;
(*state->ofs.stream) << std::flush;
ROCP_TRACE << "Destroying trace output stream...";
ofs.close();
state->ofs.close();
}
} // namespace tool
@@ -1480,6 +1480,10 @@ write_rocpd(
}
SQLITE3_CHECK(sqlite3_close_v2(conn));
// Clear UUID/GUID state at end of write to prepare for potential re-attach
get_guid().clear();
get_uuid().clear();
}
} // namespace tool
} // namespace rocprofiler
@@ -51,10 +51,24 @@ struct output_stream
{}
~output_stream() { close(); }
output_stream(const output_stream&) = delete;
output_stream(output_stream&&) noexcept = default;
output_stream(const output_stream&) = delete;
output_stream& operator=(const output_stream&) = delete;
output_stream& operator=(output_stream&&) noexcept = default;
output_stream(output_stream&& other) noexcept
{
std::swap(stream, other.stream);
std::swap(dtor, other.dtor);
}
output_stream& operator=(output_stream&& other) noexcept
{
if(this != &other)
{
std::swap(stream, other.stream);
std::swap(dtor, other.dtor);
}
return *this;
}
explicit operator bool() const { return stream != nullptr; }
@@ -35,7 +35,6 @@ except Exception:
from . import libpyrocpd
__all__ = ["format_path", "output_config", "add_args", "add_generic_args"]
@@ -127,29 +127,29 @@ PerfettoSession::~PerfettoSession()
auto filename = std::string{"results"};
auto ofs = tool::get_output_stream(config, filename, ".pftrace", std::ios::binary);
auto amount_read = std::atomic<size_t>{0};
auto is_done = std::promise<void>{};
auto _mtx = std::mutex{};
auto _reader = [&ofs, &_mtx, &is_done, &amount_read](
::perfetto::TracingSession::ReadTraceCallbackArgs _args) {
auto _lk = std::unique_lock<std::mutex>{_mtx};
if(_args.data && _args.size > 0)
{
ROCP_TRACE << "Writing " << _args.size << " B to trace...";
// Write the trace data into file
ofs.stream->write(_args.data, _args.size);
amount_read += _args.size;
}
ROCP_INFO_IF(!_args.has_more && amount_read > 0)
<< "Wrote " << amount_read << " B to perfetto trace file";
if(!_args.has_more) is_done.set_value();
};
// NOTE: These variables must be inside the loop to avoid a TSAN race condition.
// If is_done is declared outside and reassigned each iteration, the promise's internal
// mutex can be destroyed while the callback thread is still unlocking it after set_value().
for(size_t i = 0; i < 2; ++i)
{
ROCP_TRACE << "Reading trace...";
amount_read = 0;
is_done = std::promise<void>{};
auto amount_read = std::atomic<size_t>{0};
auto is_done = std::promise<void>{};
auto _mtx = std::mutex{};
auto _reader = [&ofs, &_mtx, &is_done, &amount_read](
::perfetto::TracingSession::ReadTraceCallbackArgs _args) {
auto _lk = std::unique_lock<std::mutex>{_mtx};
if(_args.data && _args.size > 0)
{
ROCP_TRACE << "Writing " << _args.size << " B to trace...";
// Write the trace data into file
ofs.stream->write(_args.data, _args.size);
amount_read += _args.size;
}
ROCP_INFO_IF(!_args.has_more && amount_read > 0)
<< "Wrote " << amount_read << " B to perfetto trace file";
if(!_args.has_more) is_done.set_value();
};
tracing_session->ReadTrace(_reader);
is_done.get_future().wait();
}
@@ -86,28 +86,6 @@ v1_get_query_info(hsa_agent_t agent, const counters::Metric& metric)
return query;
}
uint32_t
v1_get_block_counters(hsa_agent_t agent, const hsa_ven_amd_aqlprofile_event_t& event)
{
hsa_ven_amd_aqlprofile_profile_t query = {.agent = agent,
.type = HSA_VEN_AMD_AQLPROFILE_EVENT_TYPE_PMC,
.events = &event,
.event_count = 1,
.parameters = nullptr,
.parameter_count = 0,
.output_buffer = {nullptr, 0},
.command_buffer = {nullptr, 0}};
uint32_t max_block_counters = 0;
if(hsa_ven_amd_aqlprofile_get_info(&query,
HSA_VEN_AMD_AQLPROFILE_INFO_BLOCK_COUNTERS,
&max_block_counters) != HSA_STATUS_SUCCESS)
{
throw std::runtime_error(fmt::format("AQL failed to max block info for counter {}",
static_cast<int64_t>(event.block_name)));
}
return max_block_counters;
}
void
test_init()
{
@@ -199,44 +177,6 @@ TEST(aql_helpers, get_block_counters)
}
}
TEST(aql_helpers, get_block_counters_compare_v1)
{
ASSERT_EQ(hsa_init(), HSA_STATUS_SUCCESS);
test_init();
auto agents = agent::get_agents();
ASSERT_FALSE(agents.empty());
for(auto agent : agents)
{
if(agent->type == ROCPROFILER_AGENT_TYPE_CPU) continue;
auto metrics = findDeviceMetrics(*agent, {});
ASSERT_FALSE(metrics.empty());
for(auto& metric : metrics)
{
auto query = aql::get_query_info(agent->id, metric);
for(unsigned block_index = 0; block_index < query.instance_count; ++block_index)
{
aqlprofile_pmc_event_t event = {
.block_index = block_index,
.event_id = static_cast<uint32_t>(std::atoi(metric.event().c_str())),
.flags = aqlprofile_pmc_event_flags_t{0},
.block_name = static_cast<hsa_ven_amd_aqlprofile_block_name_t>(query.id)};
hsa_ven_amd_aqlprofile_event_t event_v1 = {
.block_name = static_cast<hsa_ven_amd_aqlprofile_block_name_t>(query.id),
.block_index = block_index,
.counter_id = static_cast<uint32_t>(std::atoi(metric.event().c_str()))};
EXPECT_EQ(aql::get_block_counters(agent->id, event),
v1_get_block_counters(agent::get_agent_cache(agent)->get_hsa_agent(),
event_v1));
}
}
}
hsa_shut_down();
}
TEST(aql_helpers, get_dim_info)
{
auto agents = agent::get_agents();
@@ -137,6 +137,16 @@ correlation_id::sub_kern_count()
correlation_id*
correlation_tracing_service::construct(uint32_t _init_ref_count)
{
// During finalization, we cannot create correlation IDs.
// This can happen when HSA async threads call APIs after finalization starts.
// This is not an error condition - just return nullptr and let the caller handle it.
if(registration::get_fini_status() != 0)
{
ROCP_TRACE << "correlation_tracing_service::construct called during finalization"
<< " (fini_status=" << registration::get_fini_status() << "), returning nullptr";
return nullptr;
}
ROCP_FATAL_IF(_init_ref_count == 0) << "must have reference count > 0";
auto _internal_id = get_unique_internal_id();
@@ -59,6 +59,8 @@ ROCP_SDK_ENFORCE_ABI_VERSIONING(::AmdExtTable, 74);
ROCP_SDK_ENFORCE_ABI_VERSIONING(::AmdExtTable, 75);
#elif HSA_AMD_EXT_API_TABLE_STEP_VERSION == 0x08
ROCP_SDK_ENFORCE_ABI_VERSIONING(::AmdExtTable, 77);
#elif HSA_AMD_EXT_API_TABLE_STEP_VERSION == 0x09
ROCP_SDK_ENFORCE_ABI_VERSIONING(::AmdExtTable, 79);
#else
INTERNAL_CI_ROCP_SDK_ENFORCE_ABI_VERSIONING(::AmdExtTable, 0);
#endif
@@ -299,6 +301,10 @@ ROCP_SDK_ENFORCE_ABI(::AmdExtTable, hsa_amd_portable_export_dmabuf_v2_fn, 74);
ROCP_SDK_ENFORCE_ABI(::AmdExtTable, hsa_amd_ais_file_write_fn, 75);
ROCP_SDK_ENFORCE_ABI(::AmdExtTable, hsa_amd_ais_file_read_fn, 76);
#endif
#if HSA_AMD_EXT_API_TABLE_STEP_VERSION >= 0x09
ROCP_SDK_ENFORCE_ABI(::AmdExtTable, hsa_amd_counted_queue_acquire_fn, 77);
ROCP_SDK_ENFORCE_ABI(::AmdExtTable, hsa_amd_counted_queue_release_fn, 78);
#endif
ROCP_SDK_ENFORCE_ABI(::ImageExtTable, hsa_ext_image_get_capability_fn, 1);
ROCP_SDK_ENFORCE_ABI(::ImageExtTable, hsa_ext_image_data_get_info_fn, 2);
@@ -693,6 +693,16 @@ async_copy_impl(Args... args)
_corr_id_pop = _data->correlation_id;
}
if(!_data->correlation_id)
{
// During finalization - cleanup and execute without tracing
ROCP_HSA_TABLE_CALL(ERROR, get_core_table()->hsa_signal_destroy_fn(_data->rocp_signal));
delete _data;
return invoke(get_next_dispatch<TableIdx, OpIdx>(),
std::move(_tied_args),
std::make_index_sequence<N>{});
}
// increase the reference count to denote that this correlation id is being used in a kernel
_data->correlation_id->add_ref_count();
@@ -336,11 +336,22 @@ hsa_api_impl<TableIdx, OpIdx>::functor(Args... args)
return;
}
auto buffer_record = common::init_public_api_struct(buffer_hsa_api_record_t{});
auto tracer_data = common::init_public_api_struct(callback_hsa_api_data_t{});
auto* corr_id = tracing::correlation_service::construct(ref_count);
auto internal_corr_id = corr_id->internal;
auto ancestor_corr_id = corr_id->ancestor;
auto buffer_record = common::init_public_api_struct(buffer_hsa_api_record_t{});
auto tracer_data = common::init_public_api_struct(callback_hsa_api_data_t{});
auto* corr_id = tracing::correlation_service::construct(ref_count);
// During finalization, correlation ID construction may return nullptr
if(!corr_id)
{
[[maybe_unused]] auto _ret = exec(info_type::get_table_func(), std::forward<Args>(args)...);
if constexpr(!std::is_void<RetT>::value)
return _ret;
else
return;
}
auto internal_corr_id = corr_id->internal;
auto ancestor_corr_id = corr_id->ancestor;
tracing::populate_external_correlation_ids(external_corr_ids,
thr_id,
external_corr_id_domain_idx,
@@ -518,6 +518,24 @@ HSA_API_INFO_DEFINITION_V(ROCPROFILER_HSA_TABLE_ID_AmdExt,
size_copied,
status)
# endif
# if HSA_AMD_EXT_API_TABLE_STEP_VERSION >= 0x09
HSA_API_INFO_DEFINITION_V(ROCPROFILER_HSA_TABLE_ID_AmdExt,
ROCPROFILER_HSA_AMD_EXT_API_ID_hsa_amd_counted_queue_acquire,
hsa_amd_counted_queue_acquire,
hsa_amd_counted_queue_acquire_fn,
agent,
type,
priority,
callback,
data,
flags,
queue)
HSA_API_INFO_DEFINITION_V(ROCPROFILER_HSA_TABLE_ID_AmdExt,
ROCPROFILER_HSA_AMD_EXT_API_ID_hsa_amd_counted_queue_release,
hsa_amd_counted_queue_release,
hsa_amd_counted_queue_release_fn,
queue)
# endif
# endif
#elif defined(ROCPROFILER_LIB_ROCPROFILER_HSA_ASYNC_COPY_CPP_IMPL) && \
@@ -503,6 +503,14 @@ memory_allocation_impl(Args... args)
_data.correlation_id->add_ref_count();
}
if(!_data.correlation_id)
{
// During finalization - execute without tracing
return invoke(get_next_dispatch<TableIdx, OpIdx>(),
std::move(_tied_args),
std::make_index_sequence<N>{});
}
auto thr_id = _data.correlation_id->thread_idx;
tracing::populate_external_correlation_ids(
tracing_data.external_correlation_ids,
@@ -629,6 +637,14 @@ memory_free_impl(Args... args)
_data.correlation_id->add_ref_count();
}
if(!_data.correlation_id)
{
// During finalization - execute without tracing
return invoke(get_next_dispatch<TableIdx, OpIdx>(),
std::move(_tied_args),
std::make_index_sequence<N>{});
}
auto thr_id = _data.correlation_id->thread_idx;
tracing::populate_external_correlation_ids(
tracing_data.external_correlation_ids,
@@ -70,14 +70,6 @@ namespace hsa
{
namespace
{
constexpr int64_t NUM_SIGNALS = 16;
std::atomic<int64_t>&
get_balanced_signal_slots()
{
static auto*& atomic = common::static_object<std::atomic<int64_t>>::construct(NUM_SIGNALS);
return *atomic;
}
template <typename DomainT, typename... Args>
inline bool
context_filter(const context::context* ctx, DomainT domain, Args... args)
@@ -117,8 +109,6 @@ AsyncSignalHandler(hsa_signal_value_t /*signal_v*/, void* data)
return false;
}
get_balanced_signal_slots().fetch_add(1);
auto& shared_ptr_info = *static_cast<std::shared_ptr<Queue::queue_info_session_t>*>(data);
auto& queue_info_session = *shared_ptr_info;
@@ -290,6 +280,13 @@ WriteInterceptor(const void* packets,
_corr_id_pop = corr_id;
}
if(!corr_id)
{
// During finalization - just write packet through without tracing
transformed_packets.emplace_back(packets_arr[i]);
continue;
}
// increase the reference count to denote that this correlation id is being used in a kernel
corr_id->add_ref_count();
corr_id->add_kern_count();
@@ -376,13 +373,6 @@ WriteInterceptor(const void* packets,
thr_id,
ROCPROFILER_EXTERNAL_CORRELATION_REQUEST_KERNEL_DISPATCH);
// If there is a lot of contention for HSA signals, then schedule out the thread
if(get_balanced_signal_slots().fetch_sub(1) <= 0)
{
sched_yield();
std::this_thread::sleep_for(std::chrono::nanoseconds(100));
}
// Stores the instrumentation pkt (i.e. AQL packets for counter collection)
// along with an ID of the client we got the packet from (this will be returned via
// completed_cb_t)
@@ -689,11 +679,6 @@ Queue::sync() const
_core_api.hsa_signal_wait_relaxed_fn(
_active_kernels, HSA_SIGNAL_CONDITION_EQ, 0, UINT64_MAX, HSA_WAIT_STATE_ACTIVE);
}
// get_balanced_signal_slots() increments upon kernel dispatch completion and decrements in
// WriteInterceptor with a starting value of NUM_SIGNALS, so the get_balanced_signal_slots()
// should be equivalent to NUM_SIGNALS if all kernel dispatches are completed
ROCP_CI_LOG_IF(WARNING, get_balanced_signal_slots().load() != NUM_SIGNALS) << fmt::format(
"There are {} incomplete dispatches", NUM_SIGNALS - get_balanced_signal_slots().load());
}
void
@@ -147,6 +147,7 @@ ompt_task_create_callback(ompt_data_t* encountering_task_data,
flags,
has_dependences,
codeptr_ra);
if(!corr_id) return; // During finalization
auto* state = new ompt_task_save_state{corr_id, flags};
INTERNAL(new_task_data)->ptr = state;
@@ -161,6 +162,7 @@ ompt_task_schedule_callback(ompt_data_t* prior_task_data,
{
auto* corr_id = ompt_impl<ROCPROFILER_OMPT_ID_task_schedule>::event_common(
CLIENT(prior_task_data), prior_task_status, CLIENT(next_task_data));
if(!corr_id) return; // During finalization
context::pop_latest_correlation_id(corr_id);
corr_id->sub_ref_count();
@@ -928,9 +930,13 @@ ompt_impl<OpIdx>::event_common(Args... args)
buffered_contexts,
external_corr_ids);
auto buffer_record = common::init_public_api_struct(buffer_ompt_record_t{});
auto tracer_data = common::init_public_api_struct(callback_ompt_data_t{});
auto* corr_id = tracing::correlation_service::construct(ref_count);
auto buffer_record = common::init_public_api_struct(buffer_ompt_record_t{});
auto tracer_data = common::init_public_api_struct(callback_ompt_data_t{});
auto* corr_id = tracing::correlation_service::construct(ref_count);
// During finalization, correlation ID construction may return nullptr
if(!corr_id) return nullptr;
uint64_t internal_corr_id = corr_id->internal;
uint64_t ancestor_corr_id = corr_id->ancestor;
@@ -981,6 +987,7 @@ void
ompt_impl<OpIdx>::event(Args&&... args)
{
auto corr_id = ompt_impl<OpIdx>::event_common(std::forward<Args>(args)...);
if(!corr_id) return; // During finalization
context::pop_latest_correlation_id(corr_id);
corr_id->sub_ref_count();
}
@@ -198,20 +198,18 @@ tool_tracing_buffered(rocprofiler_context_id_t context,
<< ", drop_count=" << drop_count << ", start=" << record->start_timestamp
<< ", stop=" << record->end_timestamp;
static int64_t last_corr_id = -1;
auto corr_id = static_cast<int64_t>(record->correlation_id.internal);
auto corr_id = static_cast<int64_t>(record->correlation_id.internal);
std::cout << info.str() << "\n" << std::flush;
EXPECT_GE(context.handle, 0) << info.str();
EXPECT_GT(record->thread_id, 0) << info.str();
EXPECT_GT(record->kind, 0) << info.str();
EXPECT_GT(corr_id, last_corr_id) << info.str();
EXPECT_GT(corr_id, 0) << info.str();
EXPECT_GT(record->start_timestamp, 0) << info.str();
EXPECT_GT(record->end_timestamp, 0) << info.str();
EXPECT_LE(record->start_timestamp, record->end_timestamp) << info.str();
cb_data->client_callback_count++;
last_corr_id = corr_id;
}
}
@@ -0,0 +1,24 @@
#
# AddressSanitizer suppressions file for rocprofiler project.
#
# ASAN uses interceptor_via_lib and interceptor_via_fun for suppressing
# issues in specific libraries or functions.
#
# Suppress issues in HSA runtime library
interceptor_via_lib:libhsa-runtime64.so
# Suppress issues in HIP runtime library
interceptor_via_lib:libamdhip64.so
# Suppress issues in ROCm SMI library
interceptor_via_lib:librocm_smi64.so
# Suppress issues in AQL profile library
interceptor_via_lib:libhsa-amd-aqlprofile64.so
# Suppress issues in comgr library
interceptor_via_lib:libamd_comgr.so
# Suppress issues in perfetto library (if linked dynamically)
interceptor_via_lib:libperfetto.so
@@ -92,13 +92,11 @@ class FormatAll(argparse.Action):
class InstallDepsUbuntu(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
os.system(
"sudo apt-get update; \
os.system("sudo apt-get update; \
sudo apt-get install -y python3-pip software-properties-common wget curl clang-format-11; \
python3 -m pip install -U cmake-format; \
python -m pip install --upgrade pip; \
python -m pip install black"
)
python -m pip install black")
exit(0)
@@ -10,3 +10,18 @@ leak:hsa-amd-aqlprofile
leak:__new_exitfn
leak:omptarget
leak:llvm
leak:bash
# Suppress leaks from external tools and libraries
leak:liblzma
leak:liblsan
leak:seq
leak:__GI___strdup
# Suppress leaks during tool detach/shutdown - these allocations are
# intentionally not freed as the process is exiting
leak:rocprofiler::tool::metadata::get_string_entries
leak:rocprofiler::tool::write_rocpd
leak:rocprofiler::tool::generate_output
leak:tool_detach
leak:rocprofiler_register_detach
@@ -116,15 +116,39 @@ def generate_custom(args, cmake_args, ctest_args):
MEMCHECK_SUPPRESSION_FILE = ""
if MEMCHECK_TYPE == "AddressSanitizer":
MEMCHECK_SANITIZER_OPTIONS = "detect_leaks=0 use_sigaltstack=0"
# print_suppressions=1 shows which suppressions matched during the run
MEMCHECK_SANITIZER_OPTIONS = (
"detect_leaks=0 use_sigaltstack=0 print_suppressions=1"
)
MEMCHECK_SUPPRESSION_FILE = (
f"{SOURCE_DIR}/source/scripts/address-sanitizer-suppr.txt"
)
os.environ["ASAN_OPTIONS"] = " ".join(
[
"detect_leaks=0",
"use_sigaltstack=0",
"print_suppressions=1",
f"suppressions={SOURCE_DIR}/source/scripts/address-sanitizer-suppr.txt",
os.environ.get("ASAN_OPTIONS", ""),
]
)
elif MEMCHECK_TYPE == "LeakSanitizer":
# fast_unwind_on_malloc=1 avoids deadlock in libgcc unwinder during early init
# print_suppressions=1 shows which suppressions matched during the run
MEMCHECK_SANITIZER_OPTIONS = "fast_unwind_on_malloc=1 print_suppressions=1"
MEMCHECK_SUPPRESSION_FILE = (
f"{SOURCE_DIR}/source/scripts/leak-sanitizer-suppr.txt"
)
os.environ["LSAN_OPTIONS"] = " ".join(
[
f"suppressions={SOURCE_DIR}/source/scripts/leak-sanitizer-suppr.txt",
"fast_unwind_on_malloc=1",
"print_suppressions=1",
os.environ.get("LSAN_OPTIONS", ""),
]
)
elif MEMCHECK_TYPE == "ThreadSanitizer":
# print_suppressions=1 shows which suppressions matched during the run
external_symbolizer_path = ""
for version in range(8, 20):
_symbolizer = shutil.which(f"llvm-symbolizer-{version}")
@@ -134,6 +158,7 @@ def generate_custom(args, cmake_args, ctest_args):
[
"history_size=5",
"detect_deadlocks=0",
"print_suppressions=1",
f"suppressions={SOURCE_DIR}/source/scripts/thread-sanitizer-suppr.txt",
external_symbolizer_path,
os.environ.get("TSAN_OPTIONS", ""),
@@ -151,6 +176,20 @@ def generate_custom(args, cmake_args, ctest_args):
]
)
# Print suppression file contents for debugging
if MEMCHECK_TYPE:
print(f"\n{'=' * 60}")
print(f"Sanitizer: {MEMCHECK_TYPE}")
print(f"{'=' * 60}")
# Print environment variables for sanitizers that use them
for env_var in ["TSAN_OPTIONS", "UBSAN_OPTIONS", "ASAN_OPTIONS", "LSAN_OPTIONS"]:
if env_var in os.environ:
print(f"\n{env_var}:")
print(f" {os.environ[env_var]}")
print(f"\n{'=' * 60}\n")
codecov_exclude = [
"/usr/.*",
"/opt/.*",
@@ -33,7 +33,7 @@ if [ -n "${EXTERNAL_SYMBOLIZER_PATH}" ]; then
fi
: ${ASAN_OPTIONS="detect_leaks=0 use_sigaltstack=0 suppressions=${SUPPR_DIR}/address-sanitizer-suppr.txt"}
: ${LSAN_OPTIONS="suppressions=${SUPPR_DIR}/leak-sanitizer-suppr.txt"}
: ${LSAN_OPTIONS="fast_unwind_on_malloc=1 suppressions=${SUPPR_DIR}/leak-sanitizer-suppr.txt"}
: ${TSAN_OPTIONS="history_size=5 detect_deadlocks=0 suppressions=${SUPPR_DIR}/thread-sanitizer-suppr.txt${EXTERNAL_SYMBOLIZER}"}
: ${UBSAN_OPTIONS="print_stacktrace=1 suppressions=${SUPPR_DIR}/undef-behavior-sanitizer-suppr.txt${EXTERNAL_SYMBOLIZER}"}
@@ -29,11 +29,24 @@ race:tzset_internal
# double mutex lock (there isn't one)
mutex:external/ptl/source/PTL/TaskGroup.hh
# data race in PTL TaskGroup between worker thread finishing and
# main thread accessing shared_ptr in wait(). The shared_ptr refcount
# may be accessed concurrently after wait() returns but before worker
# thread fully completes its bookkeeping.
race:rocprofiler::internal_threading::TaskGroup::wait
race:PTL::PackagedTask
# lock order inversion that cannot happen
mutex:source/lib/common/synchronized.hpp
# signal-unsafe function called from signal handler
signal:rocprofv3_error_signal_handler
# data race within perfetto internals
# data race within perfetto internals (includes TracingMuxer and protozero)
race:perfetto::internal::
race:perfetto::protos::
race:protozero::
# False positive: C++11 guarantees thread-safe static local variable initialization.
# TSAN incorrectly reports a race on the static initialization in create_write_functor.
race:rocprofiler::hip::stream::create_write_functor
@@ -0,0 +1,66 @@
#
# UndefinedBehaviorSanitizer suppressions file for rocprofiler project.
#
# UBSAN suppressions use the format: type:pattern
# where type can be: alignment, bool, bounds, enum, float-cast-overflow,
# float-divide-by-zero, function, integer-divide-by-zero, null, object-size,
# pointer-overflow, return, returns-nonnull-attribute, shift, signed-integer-overflow,
# unreachable, unsigned-integer-overflow, vla-bound, vptr
#
# Suppress undefined behavior in HSA runtime
vptr:libhsa-runtime64.so
function:libhsa-runtime64.so
# Suppress undefined behavior in HIP runtime
vptr:libamdhip64.so
function:libamdhip64.so
# Suppress undefined behavior in ROCm SMI
vptr:librocm_smi64.so
# Suppress undefined behavior in comgr
vptr:libamd_comgr.so
# Suppress alignment issues in external libraries
alignment:libhsa-runtime64.so
alignment:libamdhip64.so
# Suppress issues in perfetto (includes TracingMuxer and protozero)
vptr:perfetto::
function:perfetto::
vptr:protozero::
function:protozero::
vptr:perfetto::internal::
function:perfetto::internal::
vptr:perfetto::protos::
function:perfetto::protos::
# Suppress issues arising from OpenMP
vptr:__kmp_resume_template
function:__kmp_resume_template
vptr:__kmp_suspend_initialize_thread
function:__kmp_suspend_initialize_thread
# Suppress issues in google logging
vptr:google::LogMessageTime::CalcGmtOffset
function:google::LogMessageTime::CalcGmtOffset
vptr:tzset_internal
function:tzset_internal
# Suppress issues in PTL TaskGroup
vptr:PTL::TaskGroup
function:PTL::TaskGroup
vptr:PTL::PackagedTask
function:PTL::PackagedTask
vptr:rocprofiler::internal_threading::TaskGroup
function:rocprofiler::internal_threading::TaskGroup
# Suppress issues in synchronized.hpp
vptr:source/lib/common/synchronized.hpp
function:source/lib/common/synchronized.hpp
# Suppress issues in AQL profile library
vptr:libhsa-amd-aqlprofile64.so
function:libhsa-amd-aqlprofile64.so
alignment:libhsa-amd-aqlprofile64.so