Commit Graph

12066 Commits

Author SHA1 Message Date
Jonathan R. Madsen b570ff5273 Update samples/intercept_table (#792)
- install function wrappers around HIP runtime API
  - easily correlated to the executable
  - safer than HSA runtime due to potential for HSA to get invoked after main returns
2024-04-18 05:30:34 -05:00
Jonathan R. Madsen 12c836f95f Async memory copy callback tracing + memory copy size (#791)
* Async memory copy tracing update

- rocprofiler_buffer_tracing_memory_copy_record_t: thread_id and bytes
- support ROCPROFILER_CALLBACK_TRACING_MEMORY_COPY
- init_public_api_struct can fully construct

* Testing for callback async copy tracing
2024-04-18 04:31:59 -05:00
Benjamin Welton edb1883a05 Modified hipMalloc size for main.cpp in sample (#786)
* Modified hipMalloc size for main.cpp in sample

* Update samples/counter_collection/main.cpp

---------

Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
2024-04-18 04:28:56 -05:00
Jonathan R. Madsen 32bc339789 Simplify json-tool JSON schema for callback records (#790)
- formerly, the rocprofiler_callback_tracing_record_t data was stored in itr["record"], e.g. itr["record"]["correlation_id"]
  - dropped "record" key, e.g. itr["correlation_id"]
2024-04-18 03:58:10 -05:00
Jonathan R. Madsen 39dd5a81df PyTest utilities (#789)
- tests/pytest-packages/pytest_utils containing utilities for pytest validation tests
- example:

```python
from rocprofiler_sdk.pytest_utils.dotdict import dotdict
```
2024-04-18 03:09:01 -05:00
Jonathan R. Madsen 93136ec042 Fix kernel dispatch tracing operation names (#787)
- "ENQUEUE" -> "KERNEL_DISPATCH_ENQUEUE"
- "COMPLETE" -> "KERNEL_DISPATCH_COMPLETE"
2024-04-18 01:06:39 -05:00
Jonathan R. Madsen b7b2c294e0 Update lib/rocprofiler-sdk/hsa/queue.cpp (#788)
- fix heap-use-after-free
2024-04-18 00:05:55 -05:00
Giovanni LB 97c6b0b867 Re-enabling ATT tests
Change-Id: Ied23410d40eecbbfb8972d36ed065289f74aaa0c
2024-04-17 18:49:55 -03:00
Giovanni LB e64e15aba4 Re-enabling ATT tests
Change-Id: Ied23410d40eecbbfb8972d36ed065289f74aaa0c


[ROCm/rocprofiler commit: 97c6b0b867]
2024-04-17 18:49:55 -03:00
Benjamin Welton 81afb310ff Remove duplicate entries from derived counters (#782) 2024-04-17 13:47:02 -07:00
Jonathan R. Madsen 8cc28ae51d Enable HSA packet write interception for callback kernel tracing (#780) 2024-04-17 14:55:04 -05:00
Benjamin Welton d173dd5c04 Small error printout on test (#781) 2024-04-17 08:54:56 -07:00
Shweta.Khatri bc9cac97fe Fixing compilation errors related to MUSL libc
Fix Musl libc NULL errors and unsupported pthread funcs for compatibility.
Also ensures cleanup and error handling irrespective of CPU affinity override.

Fix submitted by github dev - AngryLoki
https://github.com/ROCm/ROCR-Runtime/issues/181

Change-Id: Ia487315e504112be5d3370756f23f6e23b9ae4be
2024-04-17 07:14:15 -04:00
Ammar ELWazir 7ace7c5303 Update ci_pc_sampling.yml (#744) 2024-04-16 23:24:33 -05:00
Vladimir Indic 200da0f274 pcs: API updated (#770) 2024-04-16 22:51:16 -05:00
Ammar ELWazir 9ace9d5450 Update continuous_integration.yml 2024-04-16 22:24:06 -05:00
Ammar ELWazir 27f3c36b41 Create scan.yaml 2024-04-16 22:23:50 -05:00
Ammar ELWazir f6eb84a83e Fixing CI 2024-04-16 22:16:46 -05:00
Ammar ELWazir fa882dbebb Update continuous_integration.yml (#779) 2024-04-16 22:09:46 -05:00
Ammar ELWazir 658bc57c10 Branches CI (#778)
* Update formatting.yml

* Update docs.yml
2024-04-16 21:49:55 -05:00
Giovanni Lenzi Baraldi fed296daed Removed unused field (#774)
* Removed unused field

* Formatting
2024-04-16 23:34:47 -03:00
Ammar ELWazir 87d549c8a9 Adding Keyword search pattern (#768)
* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Adding the scan as a script

* clean up

* Update continuous_integration.yml
2024-04-16 21:08:32 -05:00
Ammar ELWazir e13c4b2f31 Adding libdw-dev for pc sampling ci (#776) 2024-04-16 20:59:45 -05:00
Benjamin Welton 7bb9ea39d6 fix cmake formatting issue 2024-04-16 23:51:11 +00:00
Benjamin Welton 95c25acfee Remove unnecessary unknown gfx 2024-04-16 23:43:59 +00:00
Jonathan R. Madsen 2d5523b420 Fix drm include for OpenSUSE (#767)
* Fix drm include for OpenSUSE

- uses libdrm/drm.h instead of drm/drm.h

* Fix "List Files" step in CI workflows

* Fix "List Files" step in CI workflows
2024-04-16 23:16:26 +00:00
Jonathan R. Madsen d766034c06 Fix Findlibdw.cmake and Findlibelf.cmake (#764)
- improve pkg-config and explicit search fallback
2024-04-15 15:35:07 -05:00
Ammar ELWazir 3ec628e004 Fixing Markdown and Docs (#759)
* Update README.md

* Update index.md

* Update installation.md

* Update rocprofv3.md

* Update tool_library_overview.md

* Update CMakeLists.txt

* Update upload-image-to-github.py

* Update upload-image-to-github.py

---------

Co-authored-by: Gopesh Bhardwaj <gopesh.bhardwaj@amd.com>
2024-04-14 14:35:00 -05:00
Benjamin Welton c2f659ab5c Removal of HSA from counter collection (#697)
* Minor fix

Removal of HSA from counter collection

Tests for AQL

Updated counter collection client to build profiles in tool init

* Rebased

* Debug printing

* Formatting

* More format

* fix shadowing

---------

Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
2024-04-12 18:46:10 -07:00
Jonathan R. Madsen 07537b6231 rocprofiler_kernel_dispatch_info_t + header record for buffered counter collection (#758)
* Update include/rocprofiler-sdk

- defines.h
  - ROCPROFILER_VERSION_10_0 -> ROCPROFILER_SDK_VERSION_0_0
- fwd.h
  - rocprofiler_counter_record_kind_t
  - rocprofiler_kernel_dispatch_info_t
  - rocprofiler_record_counter_t
    - has dispatch id instead of correlation id
  - rocprofiler_counter_info_v0_t
    - added rocprofiler_counter_id_t field
    - added is_constant field
    - reordered better packing
- dispatch_profile.h
  - added rocprofiler_profile_counting_dispatch_record_t for use as a header record for rocprofiler_profile_counting_dispatch_data_t
- callback_tracing.h
  - rocprofiler_callback_tracing_kernel_dispatch_data_t uses rocprofiler_kernel_dispatch_info_t
- buffer_tracing.h
  - rocprofiler_buffer_tracing_kernel_dispatch_record_t uses rocprofiler_kernel_dispatch_info_t

* Update lib/rocprofiler-sdk/*

- transition to rocprofiler_kernel_dispatch_info_t
- set id and is_constant values for rocprofiler_counter_info_v0_t in rocprofiler_query_counter_info

* Update lib/rocprofiler-sdk-tool

- transition to rocprofiler_kernel_dispatch_info_t

* Update lib/rocprofiler-sdk/counters/tests/core.cpp

- transition to rocprofiler_kernel_dispatch_info_t

* Update samples

- transition to rocprofiler_kernel_dispatch_info_t
- transition to rocprofiler_counter_record_kind_t

* Update tests

- transition to rocprofiler_kernel_dispatch_info_t
- transition to rocprofiler_counter_record_kind_t
- improve integration test validation for counter-collection
- update serialization for new/additional types

* Fix tests/counter-collection/validate.py

- loosen restrictions on the length of counter description

* Update include/rocprofiler-sdk/buffer_tracing.h

- remove accidental packed attribute

* Update lib/rocprofiler-sdk/counters/xml/derived_counters.xml

- Add description for TCC_TAG_STALL_sum (reference: https://rocm.docs.amd.com/en/develop/conceptual/gpu-arch/mi300-mi200-performance-counters.html)

* Update tests/page-migration/validate.py
2024-04-12 17:30:34 -05:00
Mythreya fd3d97287c Page migration reporting (#651)
* Page migration reporting support

* Page migration: Update parser and reporting

Container does not lave latest KFD header, so CI might fail

* Add kfd_ioctl.h

* Formatting

* Update get_key

- get key was not used (and shouldn't be), so delete it

* clang-tidy fixes

* Tests for page migration

* Apply suggestions from code review

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Update tests/bin/page-migration/CMakeLists.txt

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Update page-migration test app

- add hipHostRegister to register mmap'ed allocation with HIP
- misc cleanup and reorg
- remove HSA_XNACK=1 from test env

* Update lib/rocprofiler-sdk/tests/page_migration.cpp

- fix compilation error

* Minor updates (reorg, rename)

* Page migration reporting support

* Page migration: Update parser and reporting

Container does not lave latest KFD header, so CI might fail

* Update page migration tests, fix trigger types

* Page Migration Tracing Support Refactoring (#753)

* Reorganization

* Update page migration init/fini

* Formatting

* Update page_migration.cpp

- change logging severity

* Skip test if KFD does not support page migration reporting

* Rework skipping test if KFD does not support page migration

* Fix event trigger enum values

* Fix clang-diagnostic-unused-const-variable

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
2024-04-12 15:51:44 -05:00
Jonathan R. Madsen 0e83f48cd5 Update docs workflow (#760)
* Update docs workflow

- Switch from s-weigand/setup-conda to conda-incubator/setup-miniconda

* Update docs workflow

- Update shell for steps using conda
- Add build-docs-from-source job

* Update docs workflow

- fix run command for build-docs-from-source

* Update docs workflow

- fix run command for build-docs-from-source

* Update docs workflow

- configure, build, install, and package

* Update continuous_integration.yml workflow

- Disable building the docs in the core workflow
2024-04-12 14:55:13 -05:00
Jonathan R. Madsen d6bb50cae1 Minor fixes + correlation id files + compute_runtime_sizeof (#757)
* Update lib/rocprofiler-sdk/context/*

- create correlation_id.{hpp,cpp} and moved implementation into these files instead of in context.{hpp,cpp}

* Update lib/rocprofiler-sdk/thread_trace/att_core.hpp

- fixed header includes

* Update lib/common/utility.hpp (runtime sizeof)

- added compute_runtime_sizeof<T>() function to set the "size" field to be the offset of the "reserved_padding" field if one exists

* Fix to compute_runtime_sizeof
2024-04-12 12:34:00 -05:00
Jonathan R. Madsen 3eaa678054 CTest Environment Update (#756)
* Update test/tools/json-tool.cpp

- push/pop ppid as external correlation id instead of pid

* Update environment variables for tests and samples

* Revert to old CDash dashboard in run-ci.py

* Revert to new CDash dashboard in run-ci.py
2024-04-12 08:40:00 -05:00
lancesix 066e659d6e rocprofiler-sdk-codeobj: use pkg-config to find libdw / libelf (#749)
* rocprofiler-sdk-codeobj: use pkg-config to find libdw / libelf

The current version of source/lib/rocprofiler-sdk-codeobj/CMakeLists.txt
adds -ldw and -lelf to target_link_libraries. However, on a system where
libdw-dev / libelf-dev is missing, the cmake configuration phase will
run properly and a compile time error will eventually be raised.

This patch changes the CMakelists.txt to search for libelf libdw and
configures the target as needed.  Systems missing the required support
should report an error when running cmake instead of in the middle of
the compilation.

* Use INTERFACE targets

* Resolve issues with Findlib{dw,elf}

---------

Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
2024-04-12 03:48:35 -05:00
Jonathan R. Madsen 87490d0018 Update lib/rocprofiler-sdk-tool (#755)
- questionable data race within std::regex in CI
- simplify rocprofiler::tool::format
- set config::tmp_directory to default to output_path
- fs::create_directories for tmp_file
- rework get_file_name(...) and compose_tmp_file_name(...) in tool.cpp
2024-04-12 03:48:02 -05:00
Jonathan R. Madsen 7ef9e1ea8c Bump version to 0.3.0 (#754) 2024-04-12 02:56:12 -05:00
Jonathan R. Madsen 56030018dc Callback tracing for kernel dispatches + External correlation ID request service (#682)
* Support ROCPROFILER_CALLBACK_TRACING_KERNEL_DISPATCH

* Fix doxygen

* Update callback tracing

- temporary hacks for kind operation name and iterate kind operations

* Update source/include/rocprofiler-sdk

- introduce sequence id for kernel dispatches

* Update lib/rocprofiler-sdk (seq id)

- support sequence id passing

* Update tests (seq id)

- testing for sequence ids

* Cleanup include/rocprofiler-sdk/fwd.h

* Misc cleanup

* External Correlation ID Request Service (#699)

* External correlation ID request service

- callback requesting an external correlation ID instead of fetching from top of pushed external correlation ID stack

* Update external correlation id request support

- pass internal correlation ID in callback
- async copy generates a correlation ID if none already exists
- added external correlation ID request support for scratch memory tracing
- updated scratch memory tracing to use tracing:: functions

* Update hsa/queue.hpp

- new line at EOF

* Misc tweaks

- remove unnecessary logging in agent.cpp
- correlation_id::add_ref_count check for retirement
- finalization check in HSA queue AsyncSignalHandler

* Improve assertion failure logging in misc tests

* Update include/rocprofiler-sdk/fwd.h

- remove rocprofiler_record_counter_header_t

* Move lib/rocprofiler-sdk/tracing.hpp into lib/rocprofiler-sdk/tracing/ folder

* Update lib/rocprofiler-sdk/hsa/*

- hsa::get_hsa_status_string
- queue_info_session.hpp header
- rocprofiler_packet.hpp

* Update lib/rocprofiler-sdk/{counters,hip,marker}

- execute_phase_exit_callbacks tweaks
- queue_info_session tweaks

* Move rocprofiler_kernel_dispatch_operation_t to include/rocprofiler-sdk/fwd.h

* Update rocprofiler_buffer_tracing_kernel_dispatch_record_t

- add operation field and thread_id field

* Add lib/rocprofiler-sdk/kernel_dispatch

- enum <-> string mapping for kernel dispatch
- tracing implementations

* Update lib/rocprofiler-sdk/CMakeLists.txt

- tracing and kernel dispatch sub-directories

* Update lib/rocprofiler-sdk/{buffer,callback}_tracing.cpp

- invoke rocprofiler::kernel_tracing functions

* Update tests/common/serialization.hpp

- support operation and thread_id fields for rocprofiler_buffer_tracing_kernel_dispatch_record_t

* Update tests/tools/json-tool.cpp

- use external correlation id request service

* Rename sequence_id to dispatch_id
2024-04-11 19:49:49 -05:00
Jonathan R. Madsen 0f5c575435 Fix code_object_operation_t and memory_copy_operation_t enums (#751)
- enums for operations should not contain callback/buffer tracing categorization
- e.g. ROCPROFILER_CALLBACK_TRACING_CODE_OBJECT_LOAD should be ROCPROIFLER_CODE_OBJECT_LOAD
2024-04-11 18:52:13 -05:00
Jonathan R. Madsen 3c005b81b1 Add support for hsa_amd_queue_get_info (#752)
* Add support for hsa_amd_queue_get_info

- HSA_AMD_EXT_API_TABLE_STEP_VERSION == 0x02

* Suppress unused-but-set-parameter warnings
2024-04-11 18:29:22 -05:00
David Yat Sin d6d5786051 Adding queue information queries
New hsa_amd_queue_get_info API to support:

- HSA_AMD_QUEUE_INFO_AGENT: Agent that owns the underlying HW queue

- HSA_AMD_QUEUE_INFO_DOORBELL_ID: KFD doorbell ID of the queue
completion signal.

Change-Id: I98842131bcbdd08552649791a5d43e578a615808
2024-04-11 12:53:48 -04:00
David Yat Sin 3443fdf665 PC Sampling: Disable coredump when sessions active
When doing a coredump, we try to park the wave and save its PC in
ttmp7/ttmp11, but these registers will be overwritten by PC Sampling
requests.

Change-Id: I60fb734eb3bed4ee3cc8d8bba9ec4a527fff9671
2024-04-11 12:53:43 -04:00
David Yat Sin 49e56ce782 PC Sampling: Convert timestamps to system time
Convert timestamps inside samples to system time

Change-Id: I5fad9a6887fa27c0ded9aa9b5f251cba2868f88f
2024-04-11 12:53:37 -04:00
David Yat Sin 547c9cb143 PC Sampling: Implement lost sample count
Change-Id: Idfdfbac71c1813dd7a97c301619cf8ce83713c53
2024-04-11 12:53:31 -04:00
David Yat Sin 8abbf9475b PC Sampling: Implement flush
Flush is used by the client to retrieve data that are currently stored
in the buffers. This is used by the client to retrieve current data when
the buffers are not full.

Change-Id: Ib8304dcdfb2797cb060ec72df4970d95cf6be348
2024-04-11 12:53:24 -04:00
David Yat Sin 5177d17f5d PC Sampling: Push data to PC Sampling client
Each time there is enough data to fill the client session buffer,
callback the client data ready function to transfer the buffer contents
to the client.

Change-Id: Id79775426fa6d22e00dc2ef6f55c439eacb9b2af
2024-04-11 12:53:17 -04:00
David Yat Sin 855e454671 PC Sampling: Retrieve data from trap handler
Retrieve data from the buffers previously set in the 2nd level trap
handler TMA. We use a double buffering mechanism to allow the 2nd level
trap handler to write to one buffer while we are copying data from the
other.

Co-authored by: Joseph Greathouse <Joseph.Greathouse@amd.com>
Co-authored by: James Zhu <James.Zhu@amd.com>

Change-Id: I252c381ea06b8cf927c4f9af6ea59dedc3717fbb
2024-04-11 12:53:12 -04:00
David Yat Sin efdb72fd71 PC Sampling: Update 2nd level trap handler
Update 2nd level trap handler when PC Sampling is enabled

Change-Id: I95bf2bca8057d2f8313923c7f012f033e12ccc3a
2024-04-11 12:53:06 -04:00
David Yat Sin 8d666dea01 PC Sampling: Allocate resources to retrieve data from trap handler
Allocate required device and host buffers to be able to interact with
the 2nd level trap handler.

Change-Id: If99de5aacf956ca57ecafc7b04b797be9c9decaa
2024-04-11 12:53:00 -04:00
Joseph Greathouse 431a70471e PC Sampling: Add gfx9 2nd trap handler for PC Sampling
Code is valid for gfx9 GPUs excluding gfx94x.

1st level trap handler will use TTMP13[22] to indicate host trap and
TTMP13[21] to indicate stochastic trap.

For each PC sampling method (hosttrap and stochastic), we use a double
buffering mechanism to transfer data between GPU and host.
The GPU will dump data into one buffer while CPU may be reading data
from the other buffer. There are 2 separate signals, one for each
buffer.
When signal != 0, the buffer belongs to the GPU and the GPU can write
to it. Once the buffer has reached the high watermark, the GPU will
set the signal to 0 to wake up the host and so that the host can try
to switch the buffers and read the data.

Co-authored-by: David Yat Sin <David.YatSin@amd.com>
Change-Id: If3eb0913e52fb4788059a71e5feca334612f3d5d
2024-04-11 12:52:54 -04:00