Commit Graph

8920 Commits

Author SHA1 Message Date
Karl W Schulz 42f06ca6bf update artifact name to include matrix info
Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
2024-04-19 09:39:05 -05:00
David Galiffi b09dc36e71 Create CODEOWNERS
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
2024-04-19 07:57:37 -05:00
Benjamin Welton 29bc84ec0c Add default values for kernel struct (#798)
* Add default values for kernel struct

* Update hsa-queue-dependency app

- default initializers
- check HSA_AMD_MEMORY_POOL_INFO_RUNTIME_ALLOC_ALLOWED for memory pools
- clang-tidy fixes (member -> static, etc.)

* Update run-ci.py

- add --progress --output-on-failure -V if no other options regarding verbosity are passed
- improve the ability to control the stages

---------

Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
2024-04-18 21:07:20 -05:00
Benjamin Welton c668028781 clang-18 warning fixes (part 1) (#785)
* clang-18 warning fixes (part 1)

* format fixes

* More format fixes

* More format fixes
2024-04-18 18:28:32 -05:00
Ammar ELWazir 21170b870d Enabling Mi200 & Mi300 Testing (#797)
* Enabling Mi200 & Mi300 Testing

* Update continuous_integration.yml
2024-04-18 15:47:52 -05:00
Ammar ELWazir ae2abb584b Stabilizing the machines used for testing (#771)
* Stabilizing the machines used for testing

* Update .github/workflows/continuous_integration.yml

* Update continuous_integration.yml

* Delete .github/workflows/ci_pc_sampling.yml

* Update continuous_integration.yml

- add mi300
- use if conditions for whether to run "extended" core tests

* Consistency in matrix for each job

* Update continuous_integration.yml

- include runner in core CDash name

* Update continuous_integration.yml

- remove mi300

* Update continuous_integration.yml

- add mi300

* Update CI workflow

- tweak "Install requirements" step

* Update CI workflow

- timeout on install requirements

* Update CI workflow

- revert sanitizers to gcc-12

* Update CI workflow

- remove core installation of clang-tidy (handled by python pip)

* Update CI workflow

- disable fast-fail

* Update CI workflow

- add runner to all build names
- remove mi200 and mi300

* Update CI workflow

- code coverage runs on navi3

* Update CI workflow

- add runner to all build names

* Tweak to CI sanitizer jobs

* Update CI workflow

- Handle excluded tests

* Update CI workflow

- Handle excluded tests (part 2)

* Update CI workflow

- Handle excluded tests (fix quotations)

---------

Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
2024-04-18 13:48:57 -05:00
Jonathan R. Madsen b953774580 Improve testing stability (#796)
- Update tests/bin/reproducible-runtime
  - tweak algorithm for sleeping
- Update lib/rocprofiler-sdk/hsa/async_copy
  - handle egregious skews for async copy times
- Update kernel tracing
  - handle egregious skews for kernel dispatch times
- Update lib/rocprofiler-sdk/hsa/code_object
  - use static object wrappers for code object info
- Update lib/rocprofiler-sdk-tool/config.cpp
  - fix data race in output_keys / get_local_datetime
2024-04-18 11:42:51 -05:00
Giovanni Lenzi Baraldi b6c0b50d3e Removed duplicated code (#784)
* Removed duplicated code

* Clang format

* Removing union from pc_sampling API

* Clang format
2024-04-18 07:42:26 -03:00
Jonathan R. Madsen b570ff5273 Update samples/intercept_table (#792)
- install function wrappers around HIP runtime API
  - easily correlated to the executable
  - safer than HSA runtime due to potential for HSA to get invoked after main returns
2024-04-18 05:30:34 -05:00
Jonathan R. Madsen 12c836f95f Async memory copy callback tracing + memory copy size (#791)
* Async memory copy tracing update

- rocprofiler_buffer_tracing_memory_copy_record_t: thread_id and bytes
- support ROCPROFILER_CALLBACK_TRACING_MEMORY_COPY
- init_public_api_struct can fully construct

* Testing for callback async copy tracing
2024-04-18 04:31:59 -05:00
Benjamin Welton edb1883a05 Modified hipMalloc size for main.cpp in sample (#786)
* Modified hipMalloc size for main.cpp in sample

* Update samples/counter_collection/main.cpp

---------

Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
2024-04-18 04:28:56 -05:00
Jonathan R. Madsen 32bc339789 Simplify json-tool JSON schema for callback records (#790)
- formerly, the rocprofiler_callback_tracing_record_t data was stored in itr["record"], e.g. itr["record"]["correlation_id"]
  - dropped "record" key, e.g. itr["correlation_id"]
2024-04-18 03:58:10 -05:00
Jonathan R. Madsen 39dd5a81df PyTest utilities (#789)
- tests/pytest-packages/pytest_utils containing utilities for pytest validation tests
- example:

```python
from rocprofiler_sdk.pytest_utils.dotdict import dotdict
```
2024-04-18 03:09:01 -05:00
Jonathan R. Madsen 93136ec042 Fix kernel dispatch tracing operation names (#787)
- "ENQUEUE" -> "KERNEL_DISPATCH_ENQUEUE"
- "COMPLETE" -> "KERNEL_DISPATCH_COMPLETE"
2024-04-18 01:06:39 -05:00
Jonathan R. Madsen b7b2c294e0 Update lib/rocprofiler-sdk/hsa/queue.cpp (#788)
- fix heap-use-after-free
2024-04-18 00:05:55 -05:00
Giovanni LB 97c6b0b867 Re-enabling ATT tests
Change-Id: Ied23410d40eecbbfb8972d36ed065289f74aaa0c
2024-04-17 18:49:55 -03:00
Benjamin Welton 81afb310ff Remove duplicate entries from derived counters (#782) 2024-04-17 13:47:02 -07:00
Jonathan R. Madsen 8cc28ae51d Enable HSA packet write interception for callback kernel tracing (#780) 2024-04-17 14:55:04 -05:00
Benjamin Welton d173dd5c04 Small error printout on test (#781) 2024-04-17 08:54:56 -07:00
Shweta.Khatri bc9cac97fe Fixing compilation errors related to MUSL libc
Fix Musl libc NULL errors and unsupported pthread funcs for compatibility.
Also ensures cleanup and error handling irrespective of CPU affinity override.

Fix submitted by github dev - AngryLoki
https://github.com/ROCm/ROCR-Runtime/issues/181

Change-Id: Ia487315e504112be5d3370756f23f6e23b9ae4be
2024-04-17 07:14:15 -04:00
Ammar ELWazir 7ace7c5303 Update ci_pc_sampling.yml (#744) 2024-04-16 23:24:33 -05:00
Vladimir Indic 200da0f274 pcs: API updated (#770) 2024-04-16 22:51:16 -05:00
Ammar ELWazir 9ace9d5450 Update continuous_integration.yml 2024-04-16 22:24:06 -05:00
Ammar ELWazir 27f3c36b41 Create scan.yaml 2024-04-16 22:23:50 -05:00
Ammar ELWazir f6eb84a83e Fixing CI 2024-04-16 22:16:46 -05:00
Ammar ELWazir fa882dbebb Update continuous_integration.yml (#779) 2024-04-16 22:09:46 -05:00
Ammar ELWazir 658bc57c10 Branches CI (#778)
* Update formatting.yml

* Update docs.yml
2024-04-16 21:49:55 -05:00
Giovanni Lenzi Baraldi fed296daed Removed unused field (#774)
* Removed unused field

* Formatting
2024-04-16 23:34:47 -03:00
Ammar ELWazir 87d549c8a9 Adding Keyword search pattern (#768)
* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Update continuous_integration.yml

* Adding the scan as a script

* clean up

* Update continuous_integration.yml
2024-04-16 21:08:32 -05:00
Ammar ELWazir e13c4b2f31 Adding libdw-dev for pc sampling ci (#776) 2024-04-16 20:59:45 -05:00
Benjamin Welton 7bb9ea39d6 fix cmake formatting issue 2024-04-16 23:51:11 +00:00
Benjamin Welton 95c25acfee Remove unnecessary unknown gfx 2024-04-16 23:43:59 +00:00
Jonathan R. Madsen 2d5523b420 Fix drm include for OpenSUSE (#767)
* Fix drm include for OpenSUSE

- uses libdrm/drm.h instead of drm/drm.h

* Fix "List Files" step in CI workflows

* Fix "List Files" step in CI workflows
2024-04-16 23:16:26 +00:00
Jonathan R. Madsen d766034c06 Fix Findlibdw.cmake and Findlibelf.cmake (#764)
- improve pkg-config and explicit search fallback
2024-04-15 15:35:07 -05:00
Ammar ELWazir 3ec628e004 Fixing Markdown and Docs (#759)
* Update README.md

* Update index.md

* Update installation.md

* Update rocprofv3.md

* Update tool_library_overview.md

* Update CMakeLists.txt

* Update upload-image-to-github.py

* Update upload-image-to-github.py

---------

Co-authored-by: Gopesh Bhardwaj <gopesh.bhardwaj@amd.com>
2024-04-14 14:35:00 -05:00
Benjamin Welton c2f659ab5c Removal of HSA from counter collection (#697)
* Minor fix

Removal of HSA from counter collection

Tests for AQL

Updated counter collection client to build profiles in tool init

* Rebased

* Debug printing

* Formatting

* More format

* fix shadowing

---------

Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
2024-04-12 18:46:10 -07:00
Jonathan R. Madsen 07537b6231 rocprofiler_kernel_dispatch_info_t + header record for buffered counter collection (#758)
* Update include/rocprofiler-sdk

- defines.h
  - ROCPROFILER_VERSION_10_0 -> ROCPROFILER_SDK_VERSION_0_0
- fwd.h
  - rocprofiler_counter_record_kind_t
  - rocprofiler_kernel_dispatch_info_t
  - rocprofiler_record_counter_t
    - has dispatch id instead of correlation id
  - rocprofiler_counter_info_v0_t
    - added rocprofiler_counter_id_t field
    - added is_constant field
    - reordered better packing
- dispatch_profile.h
  - added rocprofiler_profile_counting_dispatch_record_t for use as a header record for rocprofiler_profile_counting_dispatch_data_t
- callback_tracing.h
  - rocprofiler_callback_tracing_kernel_dispatch_data_t uses rocprofiler_kernel_dispatch_info_t
- buffer_tracing.h
  - rocprofiler_buffer_tracing_kernel_dispatch_record_t uses rocprofiler_kernel_dispatch_info_t

* Update lib/rocprofiler-sdk/*

- transition to rocprofiler_kernel_dispatch_info_t
- set id and is_constant values for rocprofiler_counter_info_v0_t in rocprofiler_query_counter_info

* Update lib/rocprofiler-sdk-tool

- transition to rocprofiler_kernel_dispatch_info_t

* Update lib/rocprofiler-sdk/counters/tests/core.cpp

- transition to rocprofiler_kernel_dispatch_info_t

* Update samples

- transition to rocprofiler_kernel_dispatch_info_t
- transition to rocprofiler_counter_record_kind_t

* Update tests

- transition to rocprofiler_kernel_dispatch_info_t
- transition to rocprofiler_counter_record_kind_t
- improve integration test validation for counter-collection
- update serialization for new/additional types

* Fix tests/counter-collection/validate.py

- loosen restrictions on the length of counter description

* Update include/rocprofiler-sdk/buffer_tracing.h

- remove accidental packed attribute

* Update lib/rocprofiler-sdk/counters/xml/derived_counters.xml

- Add description for TCC_TAG_STALL_sum (reference: https://rocm.docs.amd.com/en/develop/conceptual/gpu-arch/mi300-mi200-performance-counters.html)

* Update tests/page-migration/validate.py
2024-04-12 17:30:34 -05:00
Mythreya fd3d97287c Page migration reporting (#651)
* Page migration reporting support

* Page migration: Update parser and reporting

Container does not lave latest KFD header, so CI might fail

* Add kfd_ioctl.h

* Formatting

* Update get_key

- get key was not used (and shouldn't be), so delete it

* clang-tidy fixes

* Tests for page migration

* Apply suggestions from code review

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Update tests/bin/page-migration/CMakeLists.txt

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Update page-migration test app

- add hipHostRegister to register mmap'ed allocation with HIP
- misc cleanup and reorg
- remove HSA_XNACK=1 from test env

* Update lib/rocprofiler-sdk/tests/page_migration.cpp

- fix compilation error

* Minor updates (reorg, rename)

* Page migration reporting support

* Page migration: Update parser and reporting

Container does not lave latest KFD header, so CI might fail

* Update page migration tests, fix trigger types

* Page Migration Tracing Support Refactoring (#753)

* Reorganization

* Update page migration init/fini

* Formatting

* Update page_migration.cpp

- change logging severity

* Skip test if KFD does not support page migration reporting

* Rework skipping test if KFD does not support page migration

* Fix event trigger enum values

* Fix clang-diagnostic-unused-const-variable

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
2024-04-12 15:51:44 -05:00
Jonathan R. Madsen 0e83f48cd5 Update docs workflow (#760)
* Update docs workflow

- Switch from s-weigand/setup-conda to conda-incubator/setup-miniconda

* Update docs workflow

- Update shell for steps using conda
- Add build-docs-from-source job

* Update docs workflow

- fix run command for build-docs-from-source

* Update docs workflow

- fix run command for build-docs-from-source

* Update docs workflow

- configure, build, install, and package

* Update continuous_integration.yml workflow

- Disable building the docs in the core workflow
2024-04-12 14:55:13 -05:00
Jonathan R. Madsen d6bb50cae1 Minor fixes + correlation id files + compute_runtime_sizeof (#757)
* Update lib/rocprofiler-sdk/context/*

- create correlation_id.{hpp,cpp} and moved implementation into these files instead of in context.{hpp,cpp}

* Update lib/rocprofiler-sdk/thread_trace/att_core.hpp

- fixed header includes

* Update lib/common/utility.hpp (runtime sizeof)

- added compute_runtime_sizeof<T>() function to set the "size" field to be the offset of the "reserved_padding" field if one exists

* Fix to compute_runtime_sizeof
2024-04-12 12:34:00 -05:00
Jonathan R. Madsen 3eaa678054 CTest Environment Update (#756)
* Update test/tools/json-tool.cpp

- push/pop ppid as external correlation id instead of pid

* Update environment variables for tests and samples

* Revert to old CDash dashboard in run-ci.py

* Revert to new CDash dashboard in run-ci.py
2024-04-12 08:40:00 -05:00
lancesix 066e659d6e rocprofiler-sdk-codeobj: use pkg-config to find libdw / libelf (#749)
* rocprofiler-sdk-codeobj: use pkg-config to find libdw / libelf

The current version of source/lib/rocprofiler-sdk-codeobj/CMakeLists.txt
adds -ldw and -lelf to target_link_libraries. However, on a system where
libdw-dev / libelf-dev is missing, the cmake configuration phase will
run properly and a compile time error will eventually be raised.

This patch changes the CMakelists.txt to search for libelf libdw and
configures the target as needed.  Systems missing the required support
should report an error when running cmake instead of in the middle of
the compilation.

* Use INTERFACE targets

* Resolve issues with Findlib{dw,elf}

---------

Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
2024-04-12 03:48:35 -05:00
Jonathan R. Madsen 87490d0018 Update lib/rocprofiler-sdk-tool (#755)
- questionable data race within std::regex in CI
- simplify rocprofiler::tool::format
- set config::tmp_directory to default to output_path
- fs::create_directories for tmp_file
- rework get_file_name(...) and compose_tmp_file_name(...) in tool.cpp
2024-04-12 03:48:02 -05:00
Jonathan R. Madsen 7ef9e1ea8c Bump version to 0.3.0 (#754) 2024-04-12 02:56:12 -05:00
Jonathan R. Madsen 56030018dc Callback tracing for kernel dispatches + External correlation ID request service (#682)
* Support ROCPROFILER_CALLBACK_TRACING_KERNEL_DISPATCH

* Fix doxygen

* Update callback tracing

- temporary hacks for kind operation name and iterate kind operations

* Update source/include/rocprofiler-sdk

- introduce sequence id for kernel dispatches

* Update lib/rocprofiler-sdk (seq id)

- support sequence id passing

* Update tests (seq id)

- testing for sequence ids

* Cleanup include/rocprofiler-sdk/fwd.h

* Misc cleanup

* External Correlation ID Request Service (#699)

* External correlation ID request service

- callback requesting an external correlation ID instead of fetching from top of pushed external correlation ID stack

* Update external correlation id request support

- pass internal correlation ID in callback
- async copy generates a correlation ID if none already exists
- added external correlation ID request support for scratch memory tracing
- updated scratch memory tracing to use tracing:: functions

* Update hsa/queue.hpp

- new line at EOF

* Misc tweaks

- remove unnecessary logging in agent.cpp
- correlation_id::add_ref_count check for retirement
- finalization check in HSA queue AsyncSignalHandler

* Improve assertion failure logging in misc tests

* Update include/rocprofiler-sdk/fwd.h

- remove rocprofiler_record_counter_header_t

* Move lib/rocprofiler-sdk/tracing.hpp into lib/rocprofiler-sdk/tracing/ folder

* Update lib/rocprofiler-sdk/hsa/*

- hsa::get_hsa_status_string
- queue_info_session.hpp header
- rocprofiler_packet.hpp

* Update lib/rocprofiler-sdk/{counters,hip,marker}

- execute_phase_exit_callbacks tweaks
- queue_info_session tweaks

* Move rocprofiler_kernel_dispatch_operation_t to include/rocprofiler-sdk/fwd.h

* Update rocprofiler_buffer_tracing_kernel_dispatch_record_t

- add operation field and thread_id field

* Add lib/rocprofiler-sdk/kernel_dispatch

- enum <-> string mapping for kernel dispatch
- tracing implementations

* Update lib/rocprofiler-sdk/CMakeLists.txt

- tracing and kernel dispatch sub-directories

* Update lib/rocprofiler-sdk/{buffer,callback}_tracing.cpp

- invoke rocprofiler::kernel_tracing functions

* Update tests/common/serialization.hpp

- support operation and thread_id fields for rocprofiler_buffer_tracing_kernel_dispatch_record_t

* Update tests/tools/json-tool.cpp

- use external correlation id request service

* Rename sequence_id to dispatch_id
2024-04-11 19:49:49 -05:00
Jonathan R. Madsen 0f5c575435 Fix code_object_operation_t and memory_copy_operation_t enums (#751)
- enums for operations should not contain callback/buffer tracing categorization
- e.g. ROCPROFILER_CALLBACK_TRACING_CODE_OBJECT_LOAD should be ROCPROIFLER_CODE_OBJECT_LOAD
2024-04-11 18:52:13 -05:00
Jonathan R. Madsen 3c005b81b1 Add support for hsa_amd_queue_get_info (#752)
* Add support for hsa_amd_queue_get_info

- HSA_AMD_EXT_API_TABLE_STEP_VERSION == 0x02

* Suppress unused-but-set-parameter warnings
2024-04-11 18:29:22 -05:00
David Yat Sin d6d5786051 Adding queue information queries
New hsa_amd_queue_get_info API to support:

- HSA_AMD_QUEUE_INFO_AGENT: Agent that owns the underlying HW queue

- HSA_AMD_QUEUE_INFO_DOORBELL_ID: KFD doorbell ID of the queue
completion signal.

Change-Id: I98842131bcbdd08552649791a5d43e578a615808
2024-04-11 12:53:48 -04:00
David Yat Sin 3443fdf665 PC Sampling: Disable coredump when sessions active
When doing a coredump, we try to park the wave and save its PC in
ttmp7/ttmp11, but these registers will be overwritten by PC Sampling
requests.

Change-Id: I60fb734eb3bed4ee3cc8d8bba9ec4a527fff9671
2024-04-11 12:53:43 -04:00
David Yat Sin 49e56ce782 PC Sampling: Convert timestamps to system time
Convert timestamps inside samples to system time

Change-Id: I5fad9a6887fa27c0ded9aa9b5f251cba2868f88f
2024-04-11 12:53:37 -04:00