Add support for scratch reporting (#523)

* Add ToolsApiTable

Add ToolsApiTable wrapping for
scratch memory tracking

* Add initial support for scratch memory tracking

Buffering is implemented

* cmake formatting (cmake-format) (#525)

Co-authored-by: MythreyaK <MythreyaK@users.noreply.github.com>

* source formatting (clang-format v11) (#524)

Co-authored-by: MythreyaK <MythreyaK@users.noreply.github.com>

* Add callback tracing for scratch

Fixed the error where scratch tracking init was called irrespective of whether any client requested for it

* Apply suggestions from code review

Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>

* Fix tools api copy/update

Table were saved/updated incorrectly in previous
commit. Also adds passing user data through the callback

* Fix OpKind sequence for scratch tracking

Previously scratch was using OpKind from rocprofiler-sdk, but
templates were instantiated using API ID. These differ by 1

* Integration tests for scratch reporting

Added buffer and callback integration tests for scratch reporting

* source formatting (clang-format v11) (#550)

Co-authored-by: MythreyaK <26112391+MythreyaK@users.noreply.github.com>

* cmake formatting (cmake-format) (#551)

Co-authored-by: MythreyaK <26112391+MythreyaK@users.noreply.github.com>

* python formatting (black) (#549)

Co-authored-by: MythreyaK <26112391+MythreyaK@users.noreply.github.com>

* CI fixes

* source formatting (clang-format v11) (#554)

Co-authored-by: MythreyaK <26112391+MythreyaK@users.noreply.github.com>

* Update api

Rebase on main and updates based on PR feedback

* Update scratch reporting and address PR comments

- Added agent id to buffer records
- Updated `test_internal_correlation_ids` - Is almost identical to
  one in async-copy
- Updated scratch test to check for agent id
- Updated queue id serialization in callback records (prints
  handle as nested key)
- Remove `marker_api_traces` from scratch `test_internal_correlation_ids`
  validation test
- Rename `amd_tools_api` to `scratch_memory`
- Added doxygen comments
- Remove scratch callback from `tool.cpp`
- Replace assert with `LOF_IF` in `scratch_memory.cpp`

* Update tools table

Changed to match up with changes to hsa tables in main branch

* Rework scratch memory structure

* Update tests

- Added suggestions from PR review, and updated tests accordingly

* Misc cleanup

* Update scratch test

As of Apr 4th, `hsa_amd_agent_set_async_scratch_limit` is disabled.

Note,
> This API: `hsa_amd_agent_set_async_scratch_limit` is currently
> disabled. We need some changes in CP firmware to be able to do this
> and these changes are not ready yet.
> With the current code, you will also not get notifications for
> alternate-scratch allocations because this feature has been disabled
> while CP firmware is making additional changes
> We are hoping to have that feature enabled by ROCm-6.3

* Minor update to lib/rocprofiler-sdk/internal_threading.*

- delay destruction of shared_ptrs of the tasks to prevent rare (but possible) data race on the destruction of the shared_ptr

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: MythreyaK <MythreyaK@users.noreply.github.com>
Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>

source/lib/rocprofiler-sdk/hsa/hsa.cpp

@@ -27,6 +27,7 @@
 #include "lib/rocprofiler-sdk/buffer.hpp"
 #include "lib/rocprofiler-sdk/context/context.hpp"
 #include "lib/rocprofiler-sdk/hsa/details/ostream.hpp"
+#include "lib/rocprofiler-sdk/hsa/scratch_memory.hpp"
 #include "lib/rocprofiler-sdk/hsa/types.hpp"
 #include "lib/rocprofiler-sdk/hsa/utils.hpp"
 #include "lib/rocprofiler-sdk/registration.hpp"
@@ -104,6 +105,7 @@ DEFINE_TABLE_VERSION(core, CORE_API)
 DEFINE_TABLE_VERSION(amd_ext, AMD_EXT_API)
 DEFINE_TABLE_VERSION(fini_ext, FINALIZER_API)
 DEFINE_TABLE_VERSION(img_ext, IMAGE_API)
+DEFINE_TABLE_VERSION(amd_tool, TOOLS_API)
 
 #undef DEFINE_TABLE_VERSION
 #undef DEFINE_TABLE_VERSION_IMPL
@@ -153,6 +155,13 @@ get_tracing_img_ext_table()
     return val;
 }
 
+hsa_amd_tool_table_t*
+get_tracing_amd_tool_table()  // table is never traced
+{
+    static auto*& val = GET_TABLE_IMPL(amd_tool, tracing_table);
+    return val;
+}
+
 hsa_table_version_t
 get_table_version()
 {
@@ -187,6 +196,13 @@ get_img_ext_table()
     return val;
 }
 
+hsa_amd_tool_table_t*
+get_amd_tool_table()
+{
+    static auto*& val = GET_TABLE_IMPL(amd_tool, internal_table);
+    return val;
+}
+
 #undef GET_TABLE_IMPL
 
 hsa_api_table_t&
@@ -196,7 +212,8 @@ get_table()
                                       .core_          = get_core_table(),
                                       .amd_ext_       = get_amd_ext_table(),
                                       .finalizer_ext_ = get_fini_ext_table(),
-                                      .image_ext_     = get_img_ext_table()};
+                                      .image_ext_     = get_img_ext_table(),
+                                      .tools_         = get_amd_tool_table()};
     return tbl;
 }
 
@@ -773,6 +790,20 @@ INSTANTIATE_HSA_TABLE_FUNC(hsa_amd_ext_table_t, ROCPROFILER_HSA_TABLE_ID_AmdExt)
 INSTANTIATE_HSA_TABLE_FUNC(hsa_img_ext_table_t, ROCPROFILER_HSA_TABLE_ID_ImageExt)
 INSTANTIATE_HSA_TABLE_FUNC(hsa_fini_ext_table_t, ROCPROFILER_HSA_TABLE_ID_FinalizeExt)
 
+template <>
+void
+copy_table<hsa_amd_tool_table_t>(hsa_amd_tool_table_t* _tbl, uint64_t _instv)
+{
+    scratch_memory::copy_table(_tbl, _instv);
+}
+
+template <>
+void
+update_table<hsa_amd_tool_table_t>(hsa_amd_tool_table_t* _tbl, uint64_t _instv)
+{
+    scratch_memory::update_table(_tbl, _instv);
+}
+
 #undef INSTANTIATE_HSA_TABLE_FUNC
 }  // namespace hsa
 }  // namespace rocprofiler