16 Коммитов

Автор SHA1 Сообщение Дата
Jonathan R. Madsen a2288eb50b [rocprofiler-sdk] Install unit tests and helper functions for integration tests (#921)
* [rocprofiler-sdk] Install unit tests and helper functions for integration tests

* Fix rocprofiler-sdk-tests-target export

* Fix handling of cmake policy CMP0174

* Remove -vv from new pytest.ini files

* add unit tests and integration tests.

* add path to ci workflow.

* misc. fixes.

* pc sampling tests.

* bug fixes.

* pc sampling tests fix.

* misc.

* Update CMakeLists.txt

* Update rocprofiler_config_install_tests.cmake, correct license name

* fix units tests install issues.

* fix counters_def file path.

* fix bug, arg shifting.

* vendor pytest-cmake.

* cmake config fix. missing endfunction()

* disable tests, 1.rocprofv3-trace-hip-libs. 2.kernel-tracing. 3.external_correlation 4.rocpd.

* disable buffered-tracing test and remove pytest-cmake from requirements.txt.

* disable hip-graph-tracing test.

* fix building standalone tests to load rocprofiler-sdk cmake package first and then find rocprofiler_sdk_pytest module.

* addressed comments: 1.add local bin path to code cov workflow. 2.add to cmake prefix path local bin. 3.use ROCPROFILER_MEMCHECK_PRELOAD_ENV_VALUE 4.misc. fix

* enabled back tests api_buffered, external_correlation_id, hip-graph, kernel-tracing, rocpd, tracing-hip-in-libraries. and misc fixes(formating, extra fixtures for agent-index tests.)

* cpack to use llvm bin for .hsaco debug symbols.

* psdb tests fixes.

* EOL.

* misc. fixes and Disable api_buffered_tracing, external_correlation_id, hip-graph-tracing, kernel-tracing, rocpd, summary, tracing-hip-libraries, tracing-plus-counter-collection.

* fix incorrect cmakelists file.

* strip smallkernel.bin

* format.

* revert disabled tests commit.

* misc. fix in counter tests.

* misc.

* search codeobj unit test assets in curr bin and install bin.

* refactor newly added rocpd tests.

* modify tests for newly added hip-host-tracing.

* add LD LIB path to units, psdb is failing due to libs not being found.

---------

Co-authored-by: Venkateshwar Reddy Kandula <venkateshwar.kandula1306@gmail.com>
Co-authored-by: Venkateshwar Reddy Kandula <Venkateshwarreddy.Kandula@amd.com>
Co-authored-by: JeniferC99 <150404595+JeniferC99@users.noreply.github.com>
2025-11-21 08:06:56 -06:00
Madsen, Jonathan a6831b7b5b [SDK] Fix double buffer data race (#394)
* Fix double buffer data race

- fixes relatively rare data race in double buffering scheme

In `rocprofiler::buffer::instance::emplace`, the `container::record_header_buffer::get_record_headers()` function returned a `std::vector<rocprofiler_record_header_t*>` and then invoked callback to tool. It was possible for that callback to still be executing while the buffer was being updated. This potentially introduced a scenario where the rocprofiler_record_header_t* was modified (or corrupted) before the tool processed the record. In rocprofv3, this would result in a "future" buffer record showing up among "past" buffer records. E.g., correlation id sequence of 1-15 where the buffer flushes after five values, could result in this during processing:
|     |     |     |     |      |
|:---:|:---:|:---:|:---:|:---:|
|  1 |  2 |  3 |  4 | 15 |
|  6 |  7 |  8 |  9 | 10 |
| 11 | 12 | 13 | 14 | 15 |

Because buffer A (of double buffering scheme) originally containing corr ids 1-5 stalled after process corr id 4 (e.g. write to disk), buffer B filled up with 6-10 and started flushing, causing a switch back to buffer A, and buffer A was filled with 11-15 by the time callback accessed what was originally corr id 5 but was now updated to corr id 15.

* Update CHANGELOG

* misc minor cleanup

---------

Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>

[ROCm/rocprofiler-sdk commit: 8a1ee46e47]
2025-05-14 13:19:22 -05:00
Madsen, Jonathan 36f4788ad5 [CI] Miscellaneous Testing Updates (#305)
* Add rocprofiler-sdk-utilities.cmake

- contains cmake function rocprofiler_sdk_get_gfx_architectures

* Update perfetto_reader.py

- fix hash collision

* Update project names in tests folders

- rocprofiler-tests -> rocprofiler-sdk-tests

* Fix incorrect allocation-error handling

* [CI] Disable openmp tests for navi2, navi3, and navi4

* Suppress leaks by omptarget and llvm

---------

Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>

[ROCm/rocprofiler-sdk commit: 2d072f9217]
2025-03-22 18:51:42 -05:00
Madsen, Jonathan e677801859 Undefined behavior warnings caught by ROCPROFILER_DEFAULT_FAIL_REGEX (#23)
* Add regex for undefined behavior to ROCPROFILER_DEFAULT_FAIL_REGEX

- add UBSAN_OPTIONS to setup-sanitizer-env.sh

* Improve ROCPROFILER_DEFAULT_FAIL_REGEX

* Use -fno-sanitize-recover=undefined flag

- this compiler flag causes all undefined behavior errors to exit

* Revert ROCPROFILER_DEFAULT_FAIL_REGEX

* fix for shift overflow

---------

Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
Co-authored-by: Manjunath-Jakaraddi <manjunath.jakaraddi@amd.com>

[ROCm/rocprofiler-sdk commit: e743bf5a93]
2025-02-06 08:55:57 -06:00
Rawat, Swati edb51fc861 update copyright date to 2025 (#102)
* Update LICENSE

* Update conf.py

* Update copyright year

* [fix] Update copyright year

* Update copyright year "ROCm Developer Tools"

* Add license headers to c++ files

* Add license to *.py

* Update licenses in rocdecode sources

---------

Co-authored-by: srawat <120587655+SwRaw@users.noreply.github.com>
Co-authored-by: Mythreya <mythreya.kuricheti@amd.com>
Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>

[ROCm/rocprofiler-sdk commit: 97b7a6315d]
2025-01-22 19:11:20 -06:00
Jonathan R. Madsen 07f698a532 CMake: Consistently name CMake Targets (#1082)
* Change all rocprofiler-X target names to rocprofiler-sdk-X

* Update rocprofiler-sdk-config.cmake

- fix install tree target names
- simplify logic for using find w/ components and find w/o components

* Update rocprofiler-sdk-roctx-config.cmake

- simplify logic for using find w/ components and find w/o components

* Update samples/intercept_table/CMakeLists.txt

- demonstrate/test use of `find_package(rocprofiler-sdk ... COMPONENTS ...)`

[ROCm/rocprofiler-sdk commit: 74facf87a6]
2024-10-25 11:17:34 -05:00
Jonathan R. Madsen 34c35c26ba Fix misaligned stores in buffer (#1063)
* Fix misaligned read/write to buffer

- causes undefined behavior

* Update run-ci.py

- fix spurious CDash submission failure warning

* Improve run-ci.py support for UBSan

* Relax rocprofv3 summary stats count expectation

* Update CHANGELOG

[ROCm/rocprofiler-sdk commit: 37e0d7efce]
2024-09-10 17:08:57 -05:00
Jonathan R. Madsen 12ae4e6ce4 Migrate to rocprofiler-sdk:: namespace in CMake everywhere (#892)
- remove all usage/support for rocprofiler:: namespace

[ROCm/rocprofiler-sdk commit: a76f61a0a3]
2024-05-29 22:28:43 -05:00
Benjamin Welton ee77f548f7 Reduce max memory usage of buffer-parallel test (#280)
[ROCm/rocprofiler-sdk commit: 7ab334b432]
2023-12-06 14:00:22 -08:00
Ammar ELWazir 898cef06f5 Misc updates for distribution (#233)
* Adding tools support

* cmake formatting (cmake-format) (#227)

Co-authored-by: SrirakshaNag <SrirakshaNag@users.noreply.github.com>

* Checking to do rebase

* Adding rocprofv2 script

* cmake formatting (cmake-format) (#229)

Co-authored-by: bgopesh <bgopesh@users.noreply.github.com>

* Fixing build for the tool

* Removing the requirement for rocm_version

* Update rocprofiler_utilities.cmake

* C++ filesystem fixes

- added source/lib/common/filesystem.hpp
  - support older compilers which have <experimental/filesystem> and do not have <filesystem>
- added samples/common/filesystem.hpp
  - samples now depend on "common" library which provides the correct filesystem header
- renamed rocprofiler-stdcxxfs interface target to rocprofiler-cxx-filesystem
  - support old LLVM in addition to GNU
- fix bin/rocprof/rocprof.cpp
  - was using VLA

* Fix rocprofiler-drm include directories

- OpenSUSE only has include/libdrm/drm.h (no include/drm/drm.h)

* Tools fixes

* Fix for the tools

* Fix rocprofv2 script

* Fixing Filesystem Issues

* source formatting (clang-format v11) (#234)

Co-authored-by: ammarwa <ammarwa@users.noreply.github.com>

* Vlaindic/pc sampling api update (#235)

* pcs: updating PC sampling API

* source formatting (clang-format v11) (#232)

Co-authored-by: vlaindic <vlaindic@users.noreply.github.com>

---------

Co-authored-by: vlaindic <vladimir.indic@amd.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: vlaindic <vlaindic@users.noreply.github.com>

* Vlaindic/pc sampling api update for ammar branch (#244)

*Updating the documentation inside pc_sampling.h

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: vlaindic <vlaindic@users.noreply.github.com>

* pcs: use @p in front of params

* pcs: documenting struct fields updated

* Fixing PC Sampling Documentation issues

* Fixing PC Sampling Documentation

* Relocated tools directory to source/lib/rocprofiler-tool

* Fixes/updates to rocprofiler-tool

- updated CMake
- Fixed miscellaneous issues in the code (VLAs, etc.)
- Updated rocprofv2 to reflect some minor env variables changes in rocprofiler-tool
- Fixed clang-tidy warnings

* Update lib/rocprofiler-tool/CMakeLists.txt

- link to atomic library

* Add $ORIGIN/.. RUNPATH to rocprofiler-tool

* Adding readme file for tools

* Renaming the tools readme file

* Update ReadMe.md

* Update ReadMe.md

* Documentation updates

- overview and explanation of design and concepts

* Fix lib/rocprofiler-tool/README.md

- delete ReadMe.md

* Hacks for build

* Update Filesystem

* cmake formatting (cmake-format) (#248)

Co-authored-by: ammarwa <ammarwa@users.noreply.github.com>

* source formatting (clang-format v11) (#249)

Co-authored-by: ammarwa <ammarwa@users.noreply.github.com>

* source formatting (clang-format v11) (#250)

Co-authored-by: ammarwa <ammarwa@users.noreply.github.com>

* Addressing review comments on the tool readme file

* Revert "Hacks for build"

This reverts commit d6688cb3d1226c46fc97e37ced889a5b0d180940.

* Fixes for GCC 7.5 compiler in OpenSUSE 15.4

* Update lib/rocprofiler-tool/CMakeLists.txt

- link to AQL profile library

* Fix lib/rocprofiler-tool/README.md

- fix markdown

* Fix lib/rocprofiler-tool

- fix usage of hsa_ven_amd_loader_query_host_address

* Fix unused variable warnings

- byproduct of variables only used in assert statements

* Update docs

- update about.md
  - more "Important Changes" section here
- update tool_library_overview.md
  - extend "Tool Library Design" section
  - write "Tool Initialization" section
  - write "Tool Finalization" section

* Add ghc::filesystem submodule

* Implement usage of ghc::filesystem

* Add ROCPROFILER_BUILD_GHC_FS option

- option to use external/filesystem (ghc)

* Update samples/counter-collection

- compile flags
- common library
- fixes for warnings

* Update tests/kernel-tracing/CMakeLists.txt

- change install location of kernel-tracing-test-tool and install rpath

* Update samples/common/CMakeLists.txt

- compile features requiring C++17

* Update lib/rocprofiler-tool/tool.cpp

- remove include <filesystem>
- comment out unused variable
- remove unused functions
- move some functions into anonymous namespace

---------

Co-authored-by: Sriraksha Nagaraj <Sriraksha.Nagaraj@amd.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: SrirakshaNag <SrirakshaNag@users.noreply.github.com>
Co-authored-by: gobhardw <gopesh.bhardwaj@amd.com>
Co-authored-by: bgopesh <bgopesh@users.noreply.github.com>
Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
Co-authored-by: ammarwa <ammarwa@users.noreply.github.com>
Co-authored-by: vlaindic <vladimir.indic@amd.com>
Co-authored-by: vlaindic <vlaindic@users.noreply.github.com>
Co-authored-by: Vladimir Indic <139573562+vlaindic@users.noreply.github.com>
Co-authored-by: Benjamin Welton <bewelton@amd.com>
Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>

[ROCm/rocprofiler-sdk commit: fe5d074375]
2023-11-28 10:04:37 -06:00
Jonathan R. Madsen 35c6c82025 Fixes licensing in files (#206)
* Update LICENSE

- fix inconsistencies

* Revert lib/rocprofiler/counters/parser/scanner.cpp

* Update lib/rocprofiler/counters/tests/dimension.cpp

- revert ending curly brace

* Revert missing curly braces

- missing curly braces when file did not end with a new line

[ROCm/rocprofiler-sdk commit: 086218c2eb]
2023-11-14 10:58:33 -06:00
Benjamin Welton 184fff009a Agent, Counters, and AQL (#55)
* Migrate XML counter defs and reader from v1/v2

* Current Working Set

* Modified parser

* Evaluate AST Start

* Update lib/common/xml

- move definitions out of class declaration

* Update lib/rocprofiler/counters/parser

- update build of bison and flex build
  - reproducible generation
- add ROCPROFILER_REGENERATE_COUNTERS_PARSER option
- fix namespacing

* Update lib/rocprofiler/counters/xml

- change location of XML files and install them

* Update lib/rocprofiler/counter/tests

- normalize the test names
- improve test failures (more clear about where failure is)

* Update lib/rocprofiler/counters

- fix namespace
- update to new XML metrics directory

* Update lib/rocprofiler/CMakeLists.txt

- link to object library

* Update lib/rocprofiler/hsa/types.hpp

- reorganize includes

* Add metric loading class/printers

* Agent Implementation

* Queue Implementation (#79)

* Queue Implementation

* API Implementation For Counters (part 1) (#80)

* API Implementation For Counters

* Bewelton/counter collection 3 (#84)

* Added counter sample

* More changes

* More changes

* Update samples/counter_collection

- mostly formatting

* Update include/rocprofiler/counters.h

- formatting

* Add lib.common/synchronized.hpp

- Synchronized struct

* Update lib/rocprofiler/counters/xml/basic_counters.xml

- whitespace

* Update scripts/patch-parser.cmake

- tweaks for consistency

* Update lib/rocprofiler/counters/parser/tests/parser_tests.cpp

- formatting

* Update lib/rocprofiler/counters/parser

- improve consistency in rocprofiler-expr-parser-patch
- update parser.{h,cpp} and scanner.cpp
  - formatting + regenerated

* Update lib/rocprofiler/aql

- formatting
- clang-tidy fixes
- guard against memory pool access errors

* Update lib/rocprofiler/aql/tests

- formatting
- update use of get_val
- normalize test names

* Update lib/rocprofiler/counters/tests

- formatting
- patch basic_counters and derived_counters
- normalize test names

* Update lib/rocprofiler/aql/tests

- set_tests_properties

* Update test labels

- fix minor issue with gtest labels

* Update lib/rocprofiler/counters

- formatting
- clang-tidy fixes

* Update lib/rocprofiler/hsa

- fix includes
- formatting
- clang-tidy fixes
- tweak to queue_controller_init interface

* Update lib/rocprofiler

- include fixes
- namespace fixes
- clang-tidy fixes
- formatting

* Update scripts/run-ci.py

- exclude counters/parser from code coverage (generated files)

* Update include/rocprofiler/counters.h

- fix doxygen comment

* Update lib/rocprofiler/aql/packet_construct.cpp

- guard against HSA_AMD_MEMORY_POOL_ACCESS_DISALLOWED_BY_DEFAULT and HSA_AMD_MEMORY_POOL_ACCESS_NEVER_ALLOWED

* Update lib/rocprofiler/counters/parser/raw_ast.hpp

- clang-tidy fixes

* Update lib/rocprofiler/counters/evaluate_ast.hpp

- clang-tidy fixes

* Update lib/rocprofiler/aql/tests

- disable packet_generation_single and packet_generation_multi tests
  - the entire implementation rocprofiler::get_ext_table() is incorrect

* Minor fixes before cleanup

* More changes

* More fixes

* More fixes

* source formatting (clang-format v11) (#99)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Revert PTL submodule

* Update scripts/run-ci.py

- exclude counters/parser from code coverage (generated files)

* Migrating counters state to context

* Linting

* source formatting (clang-format v11) (#101)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* revert run-ci

* Testing fixes

* More test changes

* Fix minor typo

* Small queue change

* Small queue change

* source formatting (clang-format v11) (#102)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* source formatting (clang-format v11) (#105)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Documentation Change

* More documentation fixes

* source formatting (clang-format v11) (#106)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Threading fixes

* Threading fixes

* source formatting (clang-format v11) (#107)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Threading fixes

* More test fixes

* More agent fixes

* More build fixes

* source formatting (clang-format v11) (#109)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* changed test timeouts

* Build fix

* Build fix

* Updates to agent

* source formatting (clang-format v11) (#114)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* cmake formatting (cmake-format) (#113)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* remove git worktree folder

* Doc update

* testing fix

* Another test fix

* More test changes

* Rebase

* source formatting (clang-format v11) (#116)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Documentation

* source formatting (clang-format v11) (#119)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* PTL Changes

* Minor agent fix for empty labels

* source formatting (clang-format v11) (#120)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Minor agent fix for empty labels

* Refactor read_map

* source formatting (clang-format v11) (#121)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Refactor read_map

* Cache fixes

* source formatting (clang-format v11) (#122)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

---------

Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: bwelton <bwelton@users.noreply.github.com>

[ROCm/rocprofiler-sdk commit: 010693b795]
2023-10-16 15:41:40 -05:00
Jonathan R. Madsen 1bf4cb0893 Agent information w/o using hsa-runtime (#100)
* Agent information w/o using hsa-runtime

- remove lib/rocprofiler/hsa/agent.{hpp,cpp}
- update include/rocprofiler/agent.h
  - basically all possible info from /sys/class/kfd/kfd/topology/nodes/*

* Print topology in rocprofiler_lib.agent test

- hack to help diagnose errors

* Update lib/rocprofiler/tests/details/agent.cpp

- use LOG_IF(WARNING, ...) instead of LOG_IF(FATAL, ...)

* Update lib/rocprofiler/tests/agent.cpp

- print rocminfo at beginning of test
- fix comparison of agent handle
- misc other checks

* Updte lib/rocprofiler/agent.cpp

- handle unreadable /sys/class/kfd/kfd/topology/nodes/<N>/properties file

* Update lib/tests/buffering/CMakeLists.txt

- increase timeout to 120
  - buffering.parallel will timeout when thread sanitizing is enabled

* Update cmake: rocprofiler-drm

- find drm headers and libraries

* Update include/rocprofiler/agent.h

- add family_id field

* Update lib/rocprofiler/agent.cpp

- parse /proc/cpuinfo for name, family, apicid, etc.
- read_topology uses unique pointers to cleanup memory allocations
- implement name and gfxip

* Update lib/rocprofiler/tests/agent.cpp

- improved failure message
- check name/gfxip
- remove check against hsa_agent_t.handle
  - this value is dependent on the address of C++ class

* Update lib/rocprofiler/tests/details/agent.cpp

- tweak gfxip_ variable which is broken for CPU

* Update lib/rocprofiler/agent.cpp

- update string handling for name and gfxip

* Update lib/rocprofiler/tests/agent.cpp

- minor output tweak

* Update lib/rocprofiler/registration.{hpp,cpp}

- registration::init_logging() function

* Update lib/rocprofiler/agent.cpp

- fix hex handling of GFX step version

* Update lib/rocprofiler/tests/details/agent.cpp

- fix format string when nearest CPUs not found

* Update lib/rocprofiler/tests/CMakeLists.txt

- exclude details/agent.cpp from being parsed for gtest tests

* Update include/rocprofiler/fwd.h

- add ROCPROFILER_STATUS_ERROR_INCOMPATIBLE_ABI status

* Update lib/rocprofiler/tests/details/agent.{hpp,cpp}

- replace with slightly modified implementation of rocminfo
  - primary change was not printing

* Update lib/rocprofiler/tests/agent.cpp

- update test to use rocminfo data

* Update lib/rocprofiler/agent.cpp

- add pc_sampling_configs
- return error on incompatible ABI

* Update counters and counters tests

- rename test names for consistency
- fixed incorrect spelling of derived

* Add lib/rocprofiler/tests/{timestamp,version}.cpp

- add timestamp and version tests for rocprofiler_get_timestamp and rocprofiler_get_version, respectively

* Update lib/rocprofiler/tests/agent

- fix double free of name_str from isa_info_t

* Update include/rocprofiler/agent.h

- comments for rocprofiler_agent_mem_bank_t
- add rocprofiler_dim3_t
- comments for rocprofiler_agent_t
- add new fields to rocprofiler_agent_t
    - cu_count
    - workgroup_max_size
    - workgroup_max_dim
    - grid_max_size
    - grid_max_dim
    - vendor_name
    - product_name
- change prototype of rocprofiler_available_agents_cb_t to be const agent**

* Update lib/rocprofiler/agent.cpp

- set size field
- implement:
  - product_name
  - vendor_name
  - workgroup_max_size
  - workgroup_max_dim
  - grid_max_size
  - grid_max_dim
  - cu_count

* Update lib/rocprofiler/tests/agent.cpp

- changes for const agent*

* Update samples/pc_sampling

- updates for const agent*

* Update lib/rocprofiler/agent.cpp

- fix ABI compatibility check
  - return incompatible if tool agent is larger than our agent

* Update include/rocprofiler/agent.h

- doxygen comments
- make size field of rocprofiler_agent_t uint64_t for consistency
- add gpu_id via /sys/class/kfd/kfd/.../<idx>/gpu_id
- add model_name via /sys/class/kfd/kfd/.../<idx>/name

* Update lib/rocprofiler/agent.cpp

- add read_file function (vector of strings)
- support enum in read_property
- assign model_name and gpu_id fields
- remove unique_id

* Update lib/rocprofiler/tests/details/agent.*

- support family id, ucode_version, sdma_ucode_version

* Update lib/rocprofiler/tests/agent.cpp

- Add rocprofiler_lib.agent_abi test
- Verify family_id, ucode_version, sdma_ucode_version

[ROCm/rocprofiler-sdk commit: a798a26227]
2023-10-10 18:10:23 -05:00
Jonathan R. Madsen 77ab79e3e2 Fix set_tests_properties on some unittests (#90)
* Fix set_tests_properties on some unittests

- misspelled variable in two places

* Update samples/api_buffered_tracing/client.cpp

- output to file by default

* Update samples/api_callback_tracing/client.cpp

- output to file by default

* Update lib/rocprofiler/registration.cpp

- improve guards around initialize and finalize

* Update lib/rocprofiler/tests/registration.cpp

- test rocprofiler_iterate_callback_tracing_kind_names
- validate number of kind names and number of HSA operation names

* Update CI workflow and run-ci.py

- change --coverage flag to support all/unittests/samples
  - samples mode excludes lib/common
  - samples mode appends -L samples
  - unittests mode appends -L unittests

* Update samples/api_buffered_tracing/client.cpp

- header include location fix

[ROCm/rocprofiler-sdk commit: 2d533ad91e]
2023-09-27 15:44:46 -05:00
Jonathan R. Madsen 18da0bd49d Contexts, tracing, include reorg, registration, thread-pool (#65)
* Update scripts/update-doxygen.sh

- ensure build-docs folder exists

* Update scripts/run-ci.py

- exclude files in details subdirectory from code coverage

* Update scripts/thread-sanitizer-suppr.txt

- exclude races in glog

* Update docs/rocprofiler.dox.in

- exclude defines in include/rocprofiler/defines.h from doxygen
- Tweak EXCLUDE_PATTERNS and EXAMPLE_PATTERNS

* Update docs workflow

- trigger workflow whenever there is a change to the public headers (which may be doxygen comments)

* Update include/rocprofiler (reorg and overhaul)

- rocprofiler_status_t additions
  - CONTEXT_NOT_FOUND
  - CONTEXT_ERROR
  - INVALID_CONTEXT_ID
  - INVALID_CONTEXT
  - BUFFER_BUSY
- rocprofiler_context_is_active func
- rocprofiler_context_is_valid func
- rocprofiler_service_callback_tracing_kind_t update
  - remove ROCPROFILER_SERVICE_CALLBACK_TRACING_HELPER_THREAD
- Remove rocprofiler_tracing_helper_thread_operation_t
- Remove rocprofiler_helper_thread_callback_tracer_data_t
- Added rocprofiler_internal_thread_library_t
- Added rocprofiler_at_internal_thread_create
- split rocprofiler.h into several smaller headers
- reworked rocprofiler_status_t values
- added doxygen comments for enums
- replaced rocprofiler_trace_record_operation_kind_t with rocprofiler_trace_operation_t
- use @ instead of / in doxygen comment in rocprofiler_plugin.h
- fix ref to ROCPROFILER_SERVICE_CALLBACK_TRACING_MARKER_API
- end group in fwd.h
- remove PROFILE_COUNTING group in dispatch_profile.h
- remove premature group close in callback_tracing.h
- hsa.h: remove rocprofiler_hsa_trace_data_t
- fwd.h: remove rocprofiler_tracer_callback_data_t
- rename rocprofiler_correlation_id_t.handle to rocprofiler_correlation_id_t.id (consistency)
- fwd.h: add rocprofiler_callback_tracing_record_t
- callback_tracing.h: update rocprofiler_hsa_api_callback_tracer_data_t
- callback_tracing.h: add size fields
- simplify rocprofiler_tracer_callback_t
- removed ROCPROFILER_NONNULL from rocprofiler_get_version
- added rocprofiler_get_timestamp
- ROCPROFILER_STATUS_ERROR_CONFIGURATION_LOCKED in rocprofiler_status_t
- add ROCPROFILER_STATUS_ERROR_THREAD_NOT_FOUND rocprofiler_status_t
- add rocprofiler_buffer_category_t
- rocprofiler_trace_operation_t -> rocprofiler_tracing_operation_t
- rocprofiler_user_data_t union
- tweak rocprofiler_callback_tracing_record_t
  - make external_correlation_id non-pointer
  - add rocprofiler_user_data_t data field
- tweak rocprofiler_record_header_t
  - instead of single uint64_t kind field, have union for category + kind (two u32) with u64 hash
- API extensions for kind id <-> kind string
- API extensions for operation id <-> operation string
- rocprofiler_callback_trace_kind_name_cb_t
- rocprofiler_callback_trace_operation_name_cb_t
- rocprofiler_iterate_callback_trace_kind_names
- rocprofiler_iterate_callback_trace_kind_operation_names
- modify rocprofiler_hsa_api_callback_tracer_data_t data members (remove pointers)
- add rocprofiler_callback_trace_operation_args_cb_t function pointer typedef
- add rocprofiler_iterate_callback_trace_operation_args function
- fixed inconsistent use of *_trace_* vs. *_tracing_* (opting for tracing)
- removed rocprofiler_query_callback_trace_kind_name
- removed rocprofiler_query_callback_kind_operation_name
- Add include/rocprofiler/registration.h
  - header dedicated to registering a tool/client with rocprofiler
  - this header is not intended to be included by rocprofiler.h
  - rocprofiler_client_id_t
    - identifier for client tool
  - rocprofiler_client_finalize_t
    - function pointer prototype for tool-initiated finalization
  - rocprofiler_tool_initialize_t
    - function pointer prototype for tool initialization (i.e. configuration)
  - rocprofiler_tool_finalize_t
    - function pointer prototype for tool finalization
  - rocprofiler_tool_configure_result_t
    - struct returned by tool/client to rocprofiler
  - rocprofiler_is_initialized
    - function for querying whether tool-induced initialization is possible
  - rocprofiler_is_finalized
    - function for querying whether rocprofiler has been finalized
  - rocprofiler_configure prototype
    - this is the function tools implement
    - prototype is always marked as having default visibility
    - no implementation in rocprofiler
  - added typedef for rocprofiler_configure function pointer
  - added rocprofiler_force_configure to explicitly invoke rocprofiler_configure instead of relying on lazy init
- made callback typedef names more consistent (_cb_t suffix)
- typedef for rocprofiler_internal_thread_library_cb_t function pointer
- added rocprofiler_at_internal_thread_create function
- added rocprofiler_callback_thread_t struct
- added rocprofiler_create_callback_thread function
- added rocprofiler_assign_callback_thread function
- removed rocprofiler_buffer_tracing_record_header_t in favor of kind and correlation id in each record type
- added rocprofiler_buffer_tracing_kind_name_cb_t typedef
- added rocprofiler_buffer_tracing_operation_name_cb_t typedef
- added rocprofiler_iterate_buffer_tracing_kind_names function
- added rocprofiler_iterate_buffer_tracing_kind_operation_names function
- removed rocprofiler_query_buffer_trace_kind_name function
- removed rocprofiler_query_buffer_kind_operation_name function

* Update lib/common/container/stable_vector.hpp

- include limits header
- reserve_size struct
- overload stable_vector constructor to support reserving as part of construction

* Update lib/common/container/record_header_buffer.{hpp,cpp}

- add emplace member function accepting category and kind (two u32 variables) instead of one u64 kind
- use std::shared_mutex to prevent data-race when reading m_headers
- record_header_buffer is now multiple writer, single reader
- add read_lock member function (shared)
- add read_unlock member function (shared)
- lock member function gets exclusive lock
- unlock member function releases exclusive lock

* Rename "config" to "context" + restructure + implement

- Restructure config files + license
  - move config files into lib/rocprofiler/config subfolder
  - rename some files
  - add license to some files which were missing it
- Rename config/helpers.hpp
  - rename to allocator.hpp
  - remove get_domain_max_ops
- Create config/domain.{hpp,cpp}
  - structures for handling tracing domains and ops
- Update config/config.{hpp,cpp}
  - buffer_instance struct
  - callback_tracing_service struct
  - buffer_tracing_service struct
  - config struct
  - allocate_{config,buffer} func
  - {validate,start,stop}_config funcs
  - get_registered_configs func
  - get_active_configs func
  - get_buffers func
- Update rocprofiler.cpp
  - Implement rocprofiler_create_context
  - Implement rocprofiler_start_context
  - Implement rocprofiler_stop_context
  - Implement rocprofiler_context_is_active
  - Implement rocprofiler_context_is_valid
  - Implement rocprofiler_flush_buffer
  - Implement rocprofiler_destroy_buffer
  - Implement rocprofiler_create_buffer
- Update lib/rocprofiler/hsa
  - use rocprofiler_tracer_activity_domain_t instead of rocprofiler_tracer_activity_domain_t
  - remove ROCPROFILER_TRACER_ACTIVITY_DOMAIN_HSA_API fromHSA_API_INFO_DEFINITION_* macros
- Update lib/rocprofiler/context/domain.*
  - fixes for domain_info (i.e. use correct enums)
  - update rocprofiler_status_t codes
  - fix template instantiations
- Update lib/rocprofiler/context/context.*
  - use rocprofiler_service_callback_tracing_kind_t instead of rocprofiler_tracer_activity_domain_t
  - rename correlation_context to correlation_tracing_service
  - fix domains in callback_tracing_service and buffer_tracing_service
  - unique_ptr for callback_tracer and buffered_tracer in context
- Update lib/rocprofiler/rocprofiler.cpp
  - implement rocprofiler_configure_callback_tracing_service
- Update lib/rocprofiler/hsa/ostream.hpp
  - include rocprofiler.h instead of tracer.hpp
- Update lib/rocprofiler/hsa
  - migration to use rocprofiler_hsa_api_callback_tracer_data_t instead of rocprofiler_hsa_trace_data_t
  - restructure hsa_api_impl<Idx>
    - remove phase_enter and phase_exit
    - add set_data_args (partial replacement for phase_enter)
    - functor handles the contexts
- Update lib/rocprofiler/rocprofiler.cpp
  - implement rocprofiler_get_version
- Update lib/rocprofiler/hsa/hsa.{hpp,cpp}
  - remove hsa_api_ prefix for functions already in hsa namespace
- Update lib/rocprofiler/context/context.{hpp,cpp}
  - add client_idx to context struct (tool identifier)
  - add push_client function to set client_idx before context is allocated
  - add pop_client function to remove client identifier from future context creations
  - implemented {registered,active}_contexts and buffers to use new container::reserve_size overload to stable_vector
  - fix implementation of start_context
  - fix implementation of stop_context
- Update lib/rocprofiler/rocprofiler.cpp
  - prevent context creation, buffer creation, pc sampling config, etc. after initialization
  - add nullptr checks to rocprofiler_context_is_valid
  - fix rocprofiler_configure_callback_tracing_service
    - was checking size of buffers, not registered context
  - implement rocprofiler_iterate_callback_trace_kind_names
  - implement rocprofiler_iterate_callback_trace_kind_operation_names
- Update lib/rocprofiler/CMakeLists.txt
  - add registration.{hpp,cpp} to rocprofiler-library target sources
- Update lib/rocprofiler/hsa/utils.hpp
  - fix using fmt::formt with const char* strings
  - remove join functions (no longer used)
- Update lib/rocprofiler/hsa/hsa.{hpp,cpp}
  - remove args_string function
  - remove named_args_string function
  - update iterate_args function
    - change callback type
    - accept user data
  - rework the hsa_api_impl<Idx>::functor function
    - save the rocprofiler_callback_tracing_record_t between callbacks
  - update update_table function
    - check buffered_tracer domains
  - remove comments
- Update lib/rocprofiler/hsa/defines.hpp
  - remove MEMBER_<N> macros
  - add ADDR_MEMBER_<N> macros
  - remove doxygen comments for GET_MEMBER_FIELDS
  - add GET_ADDR_MEMBER_FIELDS
  - update HSA_API_INFO_DEFINITION_{0,V}
    - rename domain_idx to callback_domain_idx
    - add buffered_domain_idx
    - add as_arg_addr function
- Update lib/rocprofiler/rocprofiler.cpp
  - implement rocprofiler_iterate_callback_trace_operation_args
- Remove lib/rocprofiler/tracing.{hpp,cpp} and lib/rocprofiler/CMakeLists.txt
  - unused
- Update lib/rocprofiler/hsa/hsa.{hpp,cpp}
  - support buffered tracing in hsa_api_impl<Idx>::functor
  - rocprofiler_callback_trace_operation_args_cb_t -> rocprofiler_callback_tracing_operation_args_cb_t
    - i.e. trace -> tracing
- Update lib/rocprofiler/context/context.{hpp,cpp}
  - removed buffer_instance struct
  - removed allocate_buffer function
  - removed get_buffers function
  - changed buffer_tracing_service::buffer_array_t
- Update lib/rocprofiler/hsa: hsa.cpp, ostream.hpp, details folder
  - move ostream.hpp into details folder to prevent from contributing to code coverage
  - update cmake build system for new directory

* Add lib/rocprofiler/registration.{hpp,cpp}

- implements rocprofiler_set_api_table (called by rocprofiler-register)
- miscellaneous functions for client configure/initialize/finalize
- functions for querying the init/fini status
- relocated OnLoad HSA workaround to this file
  - at present, this is used to workaround ROCr not having rocprofiler-register integration yet
- implement rocprofiler_force_configure function
- implement rocprofiler_is_initialized function
- implement rocprofiler_is_finalized function
- ensure configure functions only invoked once
- ensure internal thread creation notification functions are invoked
- get_status is pair of atomics
- fix heap-use-after-free in init_logging
- update finalize
  - invoke hsa_shut_down
  - set all active contexts to null pointers

* Add lib/rocprofiler/buffer_tracing.cpp

- contains implementations of buffer_tracing (i.e. rocprofiler/buffer_tracing.h)
- previous implementation may have been moved out of lib/rocprofiler/rocprofiler.cpp

* Add lib/rocprofiler/buffer.{hpp,cpp}

- contains implementations of buffer (i.e. rocprofiler/buffer.h) and misc internal access functions
- previous implementation may have been moved out of lib/rocprofiler/rocprofiler.cpp and lib/rocprofiler/context/context.{hpp,cpp}

* Add lib/rocprofiler/callback_tracing.cpp

- contains implementations of callback_tracing (i.e. rocprofiler/callback_tracing.h)
- previous implementation may have been moved out of lib/rocprofiler/rocprofiler.cpp

* Add lib/rocprofiler/context.cpp

- contains implementations of context public API functions (i.e. rocprofiler/context.h)
- previous implementation may have been moved out of lib/rocprofiler/rocprofiler.cpp

* Add lib/rocprofiler/internal_threading.{hpp,cpp}

- contains implementations of internal_threading (i.e. rocprofiler/internal_threading.h)
- also contains implementations of internal access functions
- update finalize function
  - join all task groups and destroy all thread pools first, then reset unique_ptr

* Update lib/rocprofiler/rocprofiler.cpp

- rocprofiler_get_version returns status
- implement rocprofiler_get_timestamp
- remove misc implementations that were split into other files

* Update lib/rocprofiler/CMakeLists.txt

- compile new implementation files
  - buffer.cpp
  - buffer_tracing.cpp
  - callback_tracing.cpp
  - context.cpp
  - internal_threading.cpp

* Update lib/tests/buffering/buffering-*.cpp

- update to reflect changes to rocprofiler_record_header_t

* Update CMakeLists.txt

- increase minimum cmake version to 3.21 which added HIP support as a language

* Add samples/apps/transpose

- simple HIP application for testing

* Add samples/api_callback_tracing

- HIP application and tool library
- This effectively demos how to setup HSA API tracing
  - For each function called in tool, it stores the func/file/line and prints it during finalization
- client.hpp and client.cpp are the tool library
- Implement use of rocprofiler_iterate_callback_trace_operation_args
- add demo of using rocprofiler_get_version
- add_test
  - remove PASS_REGULAR_EXPRESSION
    - causing false passes during memcheck
  - add ROCPROFILER_MEMCHECK_PRELOAD_ENV to environment
- check if rocprofiler is initialized before stopping context

* Add samples/api_buffered_tracing

- Sample demonstrating tracing the HSA API via buffering
- demo rocprofiler_record_header_compute_hash
- throw exceptions for unexpected buffer data
- add_test
  - remove PASS_REGULAR_EXPRESSION
    - causing false passes during memcheck
  - add ROCPROFILER_MEMCHECK_PRELOAD_ENV to environment

* Update samples/CMakeLists.txt

- add subdirectory for api_callback_tracing
- add subdirectory api_buffered_tracing

* Update samples/pc_sampling/common.h

- fix processing of headers

* Update lib/rocprofiler/hsa/details/ostream.hpp

- fix data race on HSA_depth_max_cnt and recursion
- HSA_depth_max_cnt and recursion is now thread-local static instead of global static
- replace std::string usage with std::string_view

* Actions update

- add dependabot.yml
- use actions/checkout@v4
- install latest libasan and libtsan in sanitizer containers

* Add PTL (Parallel Tasking Library) submodule

[ROCm/rocprofiler-sdk commit: d3eaacd610]
2023-09-20 19:32:02 -05:00
Jonathan R. Madsen ccd154b74c Buffering: initial implementation and tests (#20)
* Update source/lib/common

- CMakeLists.txt
  - less verbose
  - rocprofiler-common-library uses rocprofiler-headers target
- mpl.hpp
  - metaprogramming header with type_list, size_of, index_of, and is_one_of
- record_header_buffer.{hpp,cpp}
  - wrapper class around atomic_ring_buffer and vector of rocprofiler_record_header_t
- atomic_ring_buffer.{hpp,cpp}
  - request function accepts wrap param when overwritting is not desirable
  - can_clear member function
  - clear member function for rewinding write pointer to start of buffer
- containers/CMakeLists.txt
  - include record_header_buffer.{hpp,cpp} in build target

* Update source/lib/tests: Buffering tests

- Added buffering tests. See comments in code for description

* atomic_ring_buffer -> ring_buffer

- remove ring_buffer implementation
- rename atomic_ring_buffer to ring_buffer

* atomic_ring_buffer -> ring_buffer

- remove ring_buffer implementation
- rename atomic_ring_buffer to ring_buffer

* Update record_header_buffer

- lock, unlock, is_locked, clear, save, and load member functions

* Buffering tests

- add buffer test for save/load capability

* Update rocprofiler_memcheck.cmake

- fix erroneous spaces causing incorrect string evaluation

* Update ring_buffer

- fix exception message

* undef HIP_PROF_API

- make sure HIP_PROF_API is undefined before including hip_runtime.h
- avoid directly including hip/hip_runtime.h

* Update rocprofiler_config_interfaces

- remove stale preprocessor defines that are from old rocprofiler/roctracer
  - HIP_PROF_HIP_API_STRING=1
  - PROF_API_IMPL=1

* Update run-ci.py

- fix paths to suppression files
- improve printing logs to console in github actions

* Update buffering implementation

- remove support for using malloc instead of mmap in ring_buffer
- provide some info functions in record_header_buffer
- improve the testing of the save-load buffer test

* Update run-ci.py

- fix CTEST_CUSTOM_COVERAGE_EXCLUDE

* Update hip/api_args.h

- remove undef HIP_PROF_API

* Update buffering-save-load.cpp

- updated comments

* Update record_header_buffer

- default ctor
- allocate member function
- is_allocated member function

* Update buffering-save-load.cpp

- tweaked usage of record_header_buffer to delay allocation

[ROCm/rocprofiler-sdk commit: b12ef4a75e]
2023-08-30 11:34:03 -05:00