Code object, kernel dispatch, and memory copy tracing (#177)

* Update samples/api_buffered_tracing - external correlation id - support ROCPROFILER_BUFFER_TRACING_KERNEL_DISPATCH * Update lib/rocprofiler/context.cpp - update alternative get_active_contexts paradigm * Update lib/rocprofiler/external_correlation.cpp - inherit correlation id from main thread * Update lib/rocprofiler/hsa/queue.* - typedef changes - rocprofiler_packet union - modify Queue::queue_info_session_t - use rocprofiler_packet - add thread id - add kernel id - add correlation id - out of line definitions - AsyncSignalHandler function update - handle kernel dispatch tracing - Move CreateBarrierPacket and AddVendorSpecificPacket to lambdas - handle contexts * Update lib/rocprofiler/hsa/hsa.cpp - remove unnecessary log function - use new get_active_contexts paradigm - use new correlation id updates * Update AgentCache and kernel dispatch record - include const rocprofiler_agent_t* in rocprofiler_buffer_tracing_kernel_dispatch_record_t - AgentCache::get_rocp_agent returns const pointer * Replace ROCPROFILER_SERVICE_ with ROCPROFILER_ * source formatting * Code Object Tracing - include/rocprofiler/callback_tracing.h - remove rocprofiler_callback_tracing_code_object_unload_data_t - remove rocprofiler_callback_tracing_code_object_kernel_symbol_register_data_t - include/rocprofiler/fwd.h - remove ROCPROFILER_CALLBACK_TRACING_CODE_OBJECT_UNLOAD - remove ROCPROFILER_CALLBACK_TRACING_CODE_OBJECT_DEVICE_KERNEL_SYMBOL_UNREGISTER - lib/common/utility.hpp - assert_public_api_struct_properties() - init_public_api_struct(...) - lib/rocprofiler/registration.cpp - invoke hsa::code_object_init - lib/rocprofiler/hsa/CMakeLists.txt - compile code_object code - lib/rocprofiler/hsa/code_object.{hpp,cpp} - tracing code object load/unload - lib/rocprofiler/hsa/queue.cpp - get_kernel_id * Update lib/rocprofiler/hsa/hsa.cpp - fix should_wrap_functor logic (which was not handling callback_tracer + buffered_tracer properly) * Update lib/rocprofiler/hsa/queue.cpp - fix rocprofiler_buffer_tracing_kernel_dispatch_record_t construction * Update samples/api_buffered_tracing/client.cpp - print kernel names * Move samples/apps to tests/apps * Update lib/rocprofiler/hsa/code_object.cpp - ensure unload callbacks when application is exiting - support user data in between load/unload callbacks * Update lib/rocprofiler/hsa/queue.{hpp,cpp} - store contexts and external correlation ids in queue_info_session - reduce signal_limiter to 96 to fix hangs - fix support for kernel tracing and async memory copies * Add lib/common/scope_destructor.hpp - similar to static_cleanup_wrapper but different * Update include/rocprofiler/buffer_tracing.h - update rocprofiler_buffer_tracing_memory_copy_record_t - remove operation: user can figure that out from correlation id - add kernel id - add rocprofiler agent id * Update include/rocprofiler/callback_tracing.h - fix data type of load_delta field in code object - remove rocp_agent from kernel_symbol_register_data_t (known via code_object_id) * Add samples/code_object_tracing - sample demonstrating code object tracing * Update samples - minor tweak to print_call_stack * Update lib/rocprofiler/hsa/code_object.cpp - flip ordering of unload callbacks for code object unloading and kernel symbol deregistering * clang-tidy fixes * Update lib/rocprofiler/hsa/code_object.cpp - fix heap-use-after-free issue with code object * Update include/rocprofiler/external_correlation.h - update documentation to include info about default value of external correlation value * Use common::container::small_vector for contexts - small_vector<const context*> is an ideal data structure for array of active contexts * Update context handling for code object unload - code object unload is only called for contexts which received the load callback * Update samples - improve ROCPROFILER_CALL macro to include status string - api_buffered_tracing handles ROCPROFILER_STATUS_ERROR_BUFFER_BUSY * Code object shutdown - ensure code object callbacks are invoked prior to finalizing * Update lib/common (memory allocators) - added lib/common/memory folder with allocators * Add lib/rocprofiler/allocator.* - rocprofiler::allocator::static_data_allocator - special allocator for static data which finalizes before any data gets destroyed - rocprofiler::allocator::unique_static_ptr_t - unique_ptr that uses static data deleter (ensure finalize is called) * Update lib/rocprofiler/buffer.cpp - flush checks fini status - use unique_static_ptr_t * Update lib/rocprofiler/internal_threading.* - change meaning of thread_pool_t and task_group_t - improve finalization to prevent data races and heap-use-after-free * Update lib/rocprofiler/registration.cpp - use static_data_allocator for client_library vector * Update lib/rocprofiler/context/context.* - use allocator::unique_static_ptr_t * Update lib/rocprofiler/allocator.cpp - avoid deadlock in deleter<static_data>::operator() * Update lib/rocprofiler/registration.cpp - avoid deadlock in rocprofiler::registration::finalize() * Update lib/rocprofiler/hsa/code_object.cpp - suppress duplicate reporting of code-object/kernel-symbol load/unload * Update leak sanitizer suppressions - __new_exitfn (via stdlib/cxa_atexit.c leaks
2023-11-13 22:30:15 -06:00
@@ -26,6 +26,7 @@

 #include "lib/common/container/stable_vector.hpp"
 #include "lib/common/utility.hpp"
+#include "lib/rocprofiler/allocator.hpp"
 #include "lib/rocprofiler/buffer.hpp"
 #include "lib/rocprofiler/context/context.hpp"
 #include "lib/rocprofiler/internal_threading.hpp"
@@ -41,6 +42,22 @@ namespace rocprofiler
 {
 namespace internal_threading
 {
+using thread_pool_vec_t = std::vector<std::shared_ptr<thread_pool_t>>;
+// Note: task_group maintains a shared_ptr copy to thread_pool to ensure it is not destroyed
+// before the task can be sync'd.
+using task_group_vec_t = std::vector<allocator::unique_static_ptr_t<task_group_t>>;
+
+TaskGroup::TaskGroup(std::shared_ptr<thread_pool_t> pool)
+: parent_type{static_cast<PTL::ThreadPool*>(pool.get())}
+, m_pool{std::move(pool)}
+{}
+
+ThreadPool::ThreadPool(const parent_type::Config& cfg)
+: parent_type{cfg}
+{}
+
+ThreadPool::~ThreadPool() { parent_type::destroy_threadpool(); }
+
 namespace
 {
 template <rocprofiler_runtime_library_t... Idx>
@@ -152,15 +169,10 @@ get_thread_pools()
    return _v;
 }

-auto&
+auto*&
 get_task_groups()
 {
-    static auto _v = task_group_vec_t([](auto& data) {
-        for(auto& itr : data)
-            itr.first->join();
-        data.clear();
-    });
-
+    static auto* _v = new task_group_vec_t{};
    return _v;
 }
 }  // namespace
@@ -184,8 +196,14 @@ finalize()
 {
    // PLT::ThreadPool::f_thread_ids() is not destruction order safe
    // if it does become safe, these two calls could be removed.
-    get_task_groups().destroy();
-    get_thread_pools().clear();
+    if(get_task_groups())
+    {
+        for(auto& itr : *get_task_groups())
+            itr->join();
+        get_task_groups()->clear();
+        delete get_task_groups();
+        get_task_groups() = nullptr;
+    }
 }

 void
@@ -209,13 +227,13 @@ create_callback_thread()
    // this will be index after emplace_back
    auto idx = get_thread_pools().size();

-    auto& thr_pool = get_thread_pools().emplace_back(std::make_shared<thread_pool_cleanup_t>(
-        std::make_unique<thread_pool_t>(thread_pool_config_t{.pool_size = 1}),
-        [](auto& tp) { tp->destroy_threadpool(); }));
+    auto& thr_pool = get_thread_pools().emplace_back(
+        std::make_shared<thread_pool_t>(thread_pool_config_t{.pool_size = 1}));
+
+    if(!get_task_groups()) get_task_groups() = new task_group_vec_t{};

    // construct the task group to use the newly created thread pool
-    get_task_groups().get().emplace_back(std::make_unique<task_group_t>(thr_pool->get().get()),
-                                         thr_pool);
+    get_task_groups()->emplace_back(allocator::make_unique_static<task_group_t>(thr_pool));

    // notify that rocprofiler library finished creating an internal thread
    notify_post_internal_thread_create(ROCPROFILER_LIBRARY);
@@ -227,9 +245,8 @@ create_callback_thread()
 task_group_t*
 get_task_group(rocprofiler_callback_thread_t cb_tid)
 {
-    return (!get_task_groups().get().empty())
-               ? get_task_groups().get().at(cb_tid.handle).first.get()
-               : nullptr;
+    if(!get_task_groups() || get_task_groups()->empty()) return nullptr;
+    return get_task_groups()->at(cb_tid.handle).get();
 }
 }  // namespace internal_threading
 }  // namespace rocprofiler
@@ -269,7 +286,10 @@ rocprofiler_status_t ROCPROFILER_API
 rocprofiler_assign_callback_thread(rocprofiler_buffer_id_t       buffer_id,
                                   rocprofiler_callback_thread_t cb_thread_id)
 {
-    if(cb_thread_id.handle >= rocprofiler::internal_threading::get_task_groups().get().size())
+    if(!rocprofiler::internal_threading::get_task_groups())
+        return ROCPROFILER_STATUS_ERROR_THREAD_NOT_FOUND;
+
+    if(cb_thread_id.handle >= rocprofiler::internal_threading::get_task_groups()->size())
        return ROCPROFILER_STATUS_ERROR_THREAD_NOT_FOUND;

    for(auto& bitr : rocprofiler::buffer::get_buffers())