AI/rocm-systems
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
803 Commits 3 Branches 0 Tags
3aaffc42daae3f4c31e8d2f4f567847315528f02
Commit Graph

6 Commits

Author SHA1 Message Date
Welton, Benjamin 2c4e20b951 [SWDEV-516561][1/2] Add MARKER_RANGE_EXTENT to capture ROCTX ranges (#363) 
* [SWDEV-516561][1/2] Add MARKER_RANGE_EXTENT to capture ROCTX ranges

Range extent to capture all work between roctxpush/pop operations. Entry callback takes place during roxtxpush and exit callback takes place in roctxpop. This is primarily to allow us to keep an ancestor id on the ancestor stack such that all operations that take place within the push/pop context can be annotated as being apart of this range. With the current setup (where push and pop are two separate operations that need to be combined externally), we cannot keep an ancestor id on the stack and thus cannot tie tracing events to particular ranges.

Correlation id information is inherited from the push operation. Ancestor id needs to be added in a future commit that also outputs this ancestor to CSV.

Output:

```
[ctest] {'size': 64, 'kind': 7, 'operation': 1, 'correlation_id': {'internal': 1525, 'external': 0, 'ancestor': 1524}, 'start_timestamp': 2932551479402642, 'end_timestamp': 2932551491178449, 'thread_id': 3254861}
[ctest] {'size': 64, 'kind': 8, 'operation': 2, 'correlation_id': {'internal': 1525, 'external': 0, 'ancestor': 1524}, 'start_timestamp': 2932551479405878, 'end_timestamp': 2932551491181214, 'thread_id': 3254861}
```

Note: Kind 8 = range extent op.

* Merge fix

Revert several changes

source/lib/rocprofiler-sdk/marker/range_marker.*

- separate out range marker implementation for standard marker implementation

Update public API with marker core range

Support marker core range in sdk (source/lib/rocprofiler-sdk)

Transition rocprofiler-sdk-tool and output lib to use marker core range

Misc fixes for tests

Fix logic in lib/output/generate{CSV,Stats}.cpp

Update tests/rocprofv3/tracing-hip-in-libraries (marker validation)

Fix test_otf2_data

* Test fixes

---------

Co-authored-by: Benjamin Welton <bewelton@amd.com>
 2025-07-08 23:41:22 -07:00
Rawat, Swati 97b7a6315d update copyright date to 2025 (#102) 
* Update LICENSE

* Update conf.py

* Update copyright year

* [fix] Update copyright year

* Update copyright year "ROCm Developer Tools"

* Add license headers to c++ files

* Add license to *.py

* Update licenses in rocdecode sources

---------

Co-authored-by: srawat <120587655+SwRaw@users.noreply.github.com>
Co-authored-by: Mythreya <mythreya.kuricheti@amd.com>
Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
 2025-01-22 19:11:20 -06:00
Jonathan R. Madsen 56030018dc Callback tracing for kernel dispatches + External correlation ID request service (#682) 
* Support ROCPROFILER_CALLBACK_TRACING_KERNEL_DISPATCH

* Fix doxygen

* Update callback tracing

- temporary hacks for kind operation name and iterate kind operations

* Update source/include/rocprofiler-sdk

- introduce sequence id for kernel dispatches

* Update lib/rocprofiler-sdk (seq id)

- support sequence id passing

* Update tests (seq id)

- testing for sequence ids

* Cleanup include/rocprofiler-sdk/fwd.h

* Misc cleanup

* External Correlation ID Request Service (#699)

* External correlation ID request service

- callback requesting an external correlation ID instead of fetching from top of pushed external correlation ID stack

* Update external correlation id request support

- pass internal correlation ID in callback
- async copy generates a correlation ID if none already exists
- added external correlation ID request support for scratch memory tracing
- updated scratch memory tracing to use tracing:: functions

* Update hsa/queue.hpp

- new line at EOF

* Misc tweaks

- remove unnecessary logging in agent.cpp
- correlation_id::add_ref_count check for retirement
- finalization check in HSA queue AsyncSignalHandler

* Improve assertion failure logging in misc tests

* Update include/rocprofiler-sdk/fwd.h

- remove rocprofiler_record_counter_header_t

* Move lib/rocprofiler-sdk/tracing.hpp into lib/rocprofiler-sdk/tracing/ folder

* Update lib/rocprofiler-sdk/hsa/*

- hsa::get_hsa_status_string
- queue_info_session.hpp header
- rocprofiler_packet.hpp

* Update lib/rocprofiler-sdk/{counters,hip,marker}

- execute_phase_exit_callbacks tweaks
- queue_info_session tweaks

* Move rocprofiler_kernel_dispatch_operation_t to include/rocprofiler-sdk/fwd.h

* Update rocprofiler_buffer_tracing_kernel_dispatch_record_t

- add operation field and thread_id field

* Add lib/rocprofiler-sdk/kernel_dispatch

- enum <-> string mapping for kernel dispatch
- tracing implementations

* Update lib/rocprofiler-sdk/CMakeLists.txt

- tracing and kernel dispatch sub-directories

* Update lib/rocprofiler-sdk/{buffer,callback}_tracing.cpp

- invoke rocprofiler::kernel_tracing functions

* Update tests/common/serialization.hpp

- support operation and thread_id fields for rocprofiler_buffer_tracing_kernel_dispatch_record_t

* Update tests/tools/json-tool.cpp

- use external correlation id request service

* Rename sequence_id to dispatch_id
 2024-04-11 19:49:49 -05:00
Jonathan R. Madsen 3f39339926 API Tracing Overhaul (#437) 
* Update include/rocprofiler-sdk/hsa/*

- split HSA API IDs into separate enumerations
- add support for finalize ext table

* Update include/rocprofiler-sdk/hip/*

- remove compiler_api_args.h
- rocprofiler_hip_api_args_t contains all for HIP runtime and HIP compiler
- ROCPROFILER_HIP_API_ID_ -> ROCPROFILER_HIP_RUNTIME_API_ID_

* Update include/rocprofiler-sdk/marker/table_api_id.h

- ROCPROFILER_MARKER_API_TABLE_ID_ -> ROCPROFILER_MARKER_TABLE_ID_

* Update include/rocprofiler-sdk/*/table_api_id.h

- table_api_id.h -> table_id.h

* Update include/rocprofiler-sdk/*/table_api_id.h

- table_api_id.h -> table_id.h

* Update include/rocprofiler-sdk/fwd.h

- ROCPROFILER_CALLBACK_TRACING_HSA_API split into 4 enum values:
  - ROCPROFILER_CALLBACK_TRACING_HSA_CORE_API
  - ROCPROFILER_CALLBACK_TRACING_HSA_AMD_EXT_API
  - ROCPROFILER_CALLBACK_TRACING_HSA_IMAGE_EXT_API
  - ROCPROFILER_CALLBACK_TRACING_HSA_FINALIZE_EXT_API
- ROCPROFILER_BUFFER_TRACING_HSA_API split into 4 enum values:
  - ROCPROFILER_BUFFER_TRACING_HSA_CORE_API
  - ROCPROFILER_BUFFER_TRACING_HSA_AMD_EXT_API
  - ROCPROFILER_BUFFER_TRACING_HSA_IMAGE_EXT_API
  - ROCPROFILER_BUFFER_TRACING_HSA_FINALIZE_EXT_API
- rocprofiler_callback_tracing_code_object_operation_t renamed to rocprofiler_code_object_operation_t (more consistent)
- doxygen updates

* Update include/rocprofiler-sdk/buffer_tracing.h

- improved doxygen comments
- removed unused rocprofiler_buffer_tracing_queue_scheduling_record_t
- removed unused rocprofiler_buffer_tracing_correlation_record_t

* Update include/rocprofiler-sdk/callback_tracing.h

- removed rocprofiler_callback_tracing_hip_compiler_api_data_t
  - rocprofiler_hip_api_args_t and rocprofiler_hip_compiler_api_args_t were combined
  - rocprofiler_hsa_api_retval_t and rocprofiler_hsa_compiler_api_retval_t were combined

* Update lib/rocprofiler-sdk/hsa/*

- utils.hpp
  - formatters for hsa_ext_program_t and hsa_ext_control_directives_t
- defines.hpp
  - removed variadic macros from lib/common/defines.hpp
  - HSA_API_META_DEFINITION, HSA_API_INFO_DEFINITION_0, HSA_API_INFO_DEFINITION_V specialize on table id
- async_copy.cpp
  - ROCPROFILER_HSA_API_ID_* -> ROCPROFILER_HSA_AMD_EXT_API_ID_*
  - add table id to templates
  - improve async_copy_fini
- hsa.hpp
  - add hsa_table_id_lookup
  - add hsa_domain_info
  - add table id to templates
  - add copy_table function
- hsa.cpp
  - add table id to templates
  - require hsa tables to be trivial and standard layout
  - remove set_data_args specialization for hsa_amd_memory_async_copy_rect
  - implement copy_table function
- hsa.def.cpp
  - update enums

* Update lib/rocprofiler-sdk/hip/*

- defines.hpp
  - use lib/common/defines.hpp
  - add hip_table_id_lookup to HIP_API_TABLE_LOOKUP_DEFINITION
- hip.hpp
  - hip_table_id_lookup
  - template iterate_args on table id
  - templated copy_table and update_table
- hip.cpp
  - replaced api_id_bounds with hip_domain_info
  - templated iterate_args on table id
  - templated copy_table and update_table

* Update lib/rocprofiler-sdk/marker/*

- defines.hpp
  - use lib/common/defines.hpp
- marker.cpp
  - updated enums
- marker.def.cpp
  - updated enums

* Update lib/rocprofiler-sdk/tests

- common.hpp
  - ROCPROFILER_CALL_EXPECT
  - callback_data_ext
  - update get_callback_tracing_names with new enums
  - update get_buffer_tracing_names with new enums
- external_correlation.cpp
  - support new HSA API enums
- intercept_table.cpp
  - use test/common.hpp
  - update to new HSA API enums
- registration.cpp
  - support new HSA API enums
- naming.cpp
  - validation for all get_ids(), get_names(), name_by_id(), id_by_name(), etc.

* Update lib/common

- defines.hpp
  - Move IMPL_DETAIL_FOR_EACH_NARG, GET_ADDR_MEMBER_FIELDS, and GET_NAMED_MEMBER_FIELDS here
    - used by HSA, HIP, and Marker
- static_object.hpp
  - is_trivial_standard_layout static constexpr member function
  - suppress register_static_dtor when is_trivial_standard_layout

* Update lib/rocprofiler-sdk/hsa/code_object.*

- name_by_id
- id_by_name
- get_names
- get_ids

* Update lib/rocprofiler-sdk/registration.cpp

- Update rocprofiler_set_api_table for HSA

* Update lib/rocprofiler-sdk/callback_tracing.cpp

- Update for new HSA enums
- Rework to use switch statement
  - rocprofiler_query_callback_tracing_kind_operation_name
  - rocprofiler_iterate_callback_tracing_kind_operations
  - rocprofiler_iterate_callback_tracing_kind_operation_args

* Update lib/rocprofiler-sdk/buffer_tracing.cpp

- Update for new HSA enums
- Rework to use switch statement
  - rocprofiler_query_buffer_tracing_kind_operation_name
  - rocprofiler_iterate_buffer_tracing_kind_operations

* Update lib/rocprofiler-sdk-tool

- helper.cpp
  - update get_buffer_id_names with new enums
  - update get_callback_id_names with new enums
- tools.cpp
  - update to use new HSA enums

* Update samples/common

- added call_stack.hpp
  - source_location struct
  - call_stack_t alias
  - print_call_stack function
- added name_info.hpp
  - utils for getting buffer/callback domain and operation names

* Update samples/api_buffered_tracing/client.cpp

- use samples/common/call_stack.hpp
- use samples/common/name_info.hpp
- update for new HSA enums

* Update samples/api_callback_tracing/client.cpp

- use samples/common/call_stack.hpp
- use samples/common/name_info.hpp
- update for new HSA enums

* Update tests/tools/json-tool.cpp

- update for new HSA enums

* Update tests/rocprofv3/tracing/validate.py

- update for new HSA domain names

* Update samples/counter_collection/main.cpp

- reduce number of kernels to 50,000 since 200,000 causes issues with thread sanitizer
 2024-01-30 12:14:26 -06:00
Jonathan R. Madsen 9efafc4d23 Split ROCTx API tables and update intercept table API (#421) 
* Update include/rocprofiler-sdk

- buffer_tracing.h
  - fix doxygen for rocprofiler_buffer_tracing_hip_api_record_t
  - update doxygen for rocprofiler_buffer_tracing_marker_api_record_t
    - remove unused marker_id field
- fwd.h
  - Split ROCPROFILER_CALLBACK_TRACING_MARKER_API into ROCPROFILER_CALLBACK_TRACING_MARKER_{CORE,CONTROL,NAME}_API
  - Split ROCPROFILER_BUFFER_TRACING_MARKER_API into ROCPROFILER_BUFFER_TRACING_MARKER_{CORE,CONTROL,NAME}_API
  - split rocprofiler_runtime_library_t into rocprofiler_runtime_library_t and rocprofiler_intercept_table_t
    - after split of ROCTx into 3 tables, specifying rocprofiler_at_internal_thread_create became confusing

* Update include/rocprofiler-sdk-roctx/api_trace.h

- Split into three tables: core, control, and name
  - core: what it sounds like
  - control: functions for controling the profiler
  - name: functions for giving resources names

* Update lib/rocprofiler-sdk-roctx/roctx.cpp

- modifications following split into multiple tables

* Update lib/rocprofiler-sdk/marker/*

- modifications following split of ROCTx API into multiple intercept tables

* Update lib/rocprofiler-sdk/tests

- common.hpp
  - add enums to get_callback_tracing_names() and get_buffer_tracing_names()
- intercept_table.cpp
  - update test to use rocprofiler_intercept_table_t (and enums) instead of rocproifler_runtime_library_t
  - update OR combos tested
- roctx.cpp
  - updates following split of ROCTx API table into multiple tables
  - use simplified specification of control API

* Update lib/rocprofiler-sdk

- buffer_tracing.cpp
  - Updates for ROCPROFILER_BUFFER_TRACING_MARKER_{CORE,CONTROL,NAME}_API enum values
- callback_tracing.cpp
  - Updates for ROCPROFILER_CALLBACK_TRACING_MARKER_{CORE,CONTROL,NAME}_API enum values
- intercept_table.hpp
  - notify_runtime_api_registration -> notify_intercept_table_registration
- intercept_table.cpp
  - updates for new rocprofiler_intercept_table_t enum and new ROCTx tables
- registration.cpp
  - updates for new rocprofiler_intercept_table_t enum and new ROCTx tables
  - updates for notify_runtime_api_registration -> notify_intercept_table_registration

* Update lib/rocprofiler-sdk-tool

- helper.cpp
  - Updates for new enums in get_callback_id_names() and get_buffer_id_names()
- tool.cpp
  - migrate to new enums for split ROCTx tables
  - use simplified split for control table vs. core+name tables

* Update samples/{api_callback_tracing,intercept_table}

- intercept_table/client.cpp
  - rocprofiler_runtime_library_t -> rocprofiler_intercept_table_t
- api_callback_tracing/client.cpp
  - Updates for new enums in get_callback_id_names()
  - use simplified split for control table vs. core+name tables
  - migrate to new enums for split ROCTx tables

* Update tests

- rocprofv3/tracing/validate.py
  - handle new marker domain names
- tools/json-tool.cpp
  - Updates for new enums in get_callback_id_names() and get_buffer_id_names()
  - use simplified split for control table vs. core+name tables
  - migrate to new enums for split ROCTx tables

* Update tests/rocprofv3/tracing/CMakeLists.txt

- fix FAIL_REGULAR_EXPRESSION for rocprofv3-test-trace-execute

* Update lib/rocprofiler-sdk-tool/{output_file,tool}.*

- logging in output_file dtor
- support stdout/stderr

* Update lib/common/container/record_header_buffer.hpp

- reduce probability of is_empty() returning true while emplace is happening

* Update lib/rocprofiler-sdk-tool/tool.cpp

- logging for buffered_tracing_callback
- counter collection uses CSV encoder

* Update bin/rocprofv3

- remove -i flag from help menu
 2024-01-26 13:56:15 -06:00
Jonathan R. Madsen 21dd088c8e ROCTx Library Tracing (#390) 
* Update include/rocprofiler-sdk/marker/*

- Update rocprofiler_marker_api_args_t for all API functions
- Add ROCPROFILER_MARKER_API_ID_roctxGetThreadId to rocprofiler_marker_api_id_t

* Update include/rocprofiler-sdk/marker/api_args.h

- fix include

* Update lib/common/mpl.hpp

- is_pair
- is_type_complete_v

* Update include/rocprofiler-sdk/marker/*

- fix rocprofiler_marker_api_retval_t
- add roctxGetThreadId to rocprofiler_marker_api_args_t
- fix type in enum: HsaDevice -> HsaAgent
- add table_api_id.h

* Update include/rocprofiler-sdk/marker.h

- include marker/table_api_id.h

* Update include/rocprofiler-sdk/buffer_tracing.h

- Buffer marker tracer records have begin and end timestamp

* Add lib/rocprofiler-sdk/marker

- tracing implementation for marker (roctx) library

* Update include/rocprofiler-sdk/{buffer_tracing,marker/table_api_id}.h

- rocprofiler_buffer_tracing_marker_record_t -> rocprofiler_buffer_tracing_marker_api_record_t

* Update lib/rocprofiler-sdk/buffer_tracing.cpp

- support for ROCPROFILER_BUFFER_TRACING_MARKER_API

* Update lib/rocprofiler-sdk/callback_tracing.cpp

- support for ROCPROFILER_CALLBACK_TRACING_MARKER_API

* Update lib/rocprofiler-sdk/intercept_table.cpp

- template instantiation for notify_runtime_api_registration

* Update lib/rocprofiler-sdk/registration.cpp

- enable roctx in rocprofiler_set_api_table

* Update lib/rocprofiler-sdk/marker/marker.cpp

- rocprofiler_buffer_tracing_marker_record_t -> rocprofiler_buffer_tracing_marker_api_record_t

* Update lib/rocprofiler/tests for roctx testing

- add roctx.cpp
  - unit tests for roctx callback and buffer tracing
- support marker API in get_{buffer,callback}_tracing_names()

* Update lib/common/logging.cpp

- logging initialized message mentions env variable

* Update lib/common/mpl.hpp

- NOLINT for misc-definitions-in-headers

* Update lib/rocprofiler-sdk/tests/CMakeLists.txt

- include LD_LIBRARY_PATH in rocprofiler-lib-tests-shared tests

* Update lib/rocprofiler-sdk/registration.cpp

- client_library_vec_t is now vector of option<client_library>
  - enables resetting the client_library after finalization
- removed acquiring registration lock when invoke_client_finalizers called via atexit
  - this was causing some lock-order-inversion warnings (potential deadlock)

* Update lib/rocprofiler-sdk/agent.cpp

- model name for agent supports spaces

* Update tests/common/serialization.hpp

- add serialization support for marker tracing data structures

* Update tests/apps

- Add ROCTx markers into reproducible-runtime and transpose

* Update tests/tools/json-tools.cpp

- add marker tracing support
- remove strdup (no longer necessary)

* Update tests/kernel-tracing/validate.py

- validate marker API tracing data

* Update tests/async-copy-tracing/validate.py

- validate marker API tracing data

* Update cmake for load path resolution during testing

* Update tests/async-copy-tracing/CMakeLists.txt

- fix test LD_LIBRARY_PATH

* Update cmake/Templates/rocprofiler-sdk-roctx/config.cmake.in

- fix constructing rocprofiler-sdk-roctx::rocprofiler-sdk-roctx
 2024-01-18 09:48:06 -06:00