Graphe des révisions

15 Révisions

Auteur SHA1 Message Date
Jonathan R. Madsen cf5e4b4b1b Integration Testing (#211)
* Add external/cereal submodule

- used for integration testing

* Update lib/common/container/small_vector.hpp

- documentation notes

* Update tests/apps

- update transpose app (fix build)
- add reproducible-runtime app

* Update include/rocprofiler/fwd.h

- rocprofiler_service_callback_phase_t -> rocprofiler_callback_phase_t

* Update PTL submodule

- fix for task group: submitting tasks from different thread

* Update lib/rocprofiler/hsa/queue.cpp

- CHECK_NOTNULL(_buffer)

* Update lib/rocprofiler/hsa/hsa.cpp

- use buffer::get_buffer instead of manually looking for buffer

* Update lib/rocprofiler/internal_threading.cpp

- use buffer::get_buffer instead of manually looking for buffer

* Update lib/rocprofiler/buffer.cpp

- offset the buffer id
- properly handle rocprofiler_create_buffer reusing rocprofiler_buffer_id_t on a different context

* Update tests

- kernel tracing library for integration testing

* Add cereal submodule

* Update lib/rocprofiler/registration.*

- OnUnload
- Support ROCP_TOOL_LIBRARIES for python usage
- improve finalize function
- remove calling hsa_shut_down in finalize function

* Update lib/rocprofiler/buffer.*

- allocate_buffer sets the buffer id value
- expose (internally) is_valid_buffer_id
- update test

* Update tests/kernel-tracing

- installation
- better organization of JSON groups
- improved messaging

* Update lib/rocprofiler/registration.cpp

- add workaround for hsa-runtime supporting rocprofiler-register

* Update tests/kernel-tracing/kernel-tracing.cpp

- fix memory leaks

* cereal support for minimal JSON

- update cereal submodule to rocprofiler branch
- change REPO_BRANCH in rocprofiler_checkout_git_submodule for cereal
- update tests/kernel-tracing/kernel-tracing.cpp
  - use minimal json
  - slight tweak putting giving contexts name in storing name + context pointer pair in map

* Update tests/kernel-tracing/kernel-tracing.cpp

- support runtime selection of contexts via KERNEL_TRACING_CONTEXTS environment variable

* Update tests

- tests/CMakeLists.txt
  - find_package(Python3 REQUIRED)
- tests/kernel-tracing
  - pytest validation

* Update CI workflow

- install pytest
- add checks for test labels

* Update scripts/run-ci.py

- change --coverage options
  - replace 'unittests' with 'tests'
- replace test label regex '-L unittests' with '-L tests'

* Update requirements.txt

- this is now an empty file since none of the packages are required for this repo
2023-11-16 03:21:39 -06:00
Jonathan R. Madsen 086218c2eb Fixes licensing in files (#206)
* Update LICENSE

- fix inconsistencies

* Revert lib/rocprofiler/counters/parser/scanner.cpp

* Update lib/rocprofiler/counters/tests/dimension.cpp

- revert ending curly brace

* Revert missing curly braces

- missing curly braces when file did not end with a new line
2023-11-14 10:58:33 -06:00
Jonathan R. Madsen 55f2dabbb3 Generalized updates (#174)
- include/rocprofiler/agent.h
  - move rocprofiler_dim3_t
- include/rocprofiler/buffer_tracing.h
  - size fields
  - update kernel dispatch record
- include/rocprofiler/callback_tracing.h
  - remove rocprofiler_callback_tracing_code_object_unload_data_t
  - remove rocprofiler_callback_tracing_code_object_register_host_kernel_symbol_data_t
- include/rocprofiler/fwd.h
  - added ROCPROFILER_STATUS_ERROR_CONTEXT_CONFLICT
  - remove ROCPROFILER_CALLBACK_TRACING_CODE_OBJECT_UNLOAD
  - remove ROCPROFILER_CALLBACK_TRACING_CODE_OBJECT_DEVICE_KERNEL_SYMBOL_UNREGISTER
  - add rocprofiler_kernel_id_t typedef
  - add rocprofiler_dim3_t (moved from agent.h)
- lib/common/synchronized.hpp
  - rlock/wlock return decltype(auto)
  - separate prototype from definition
- lib/common/utility.{hpp,cpp}
  - timestamp functions replicating HSA implementation(s)
  - init_public_api_struct for setting size field and ensuring certain type traits
  - simplified static_cleanup_wrapper
  - separate prototype from definition in active_capacity_gate
- lib/rocprofiler/agent.cpp
  - tweak get_rocprofiler_agent impl
- lib/rocprofiler/buffer.cpp
  - fix buffer message log level
- lib/rocprofiler/context.cpp
  - use new paradigm for getting active contexts
- lib/rocprofiler/internal_threading.hpp
  - update to simplified static_cleanup_wrapper implementation
- lib/rocprofiler/registration.cpp
  - fix deactivating contexts
- lib/rocprofiler/rocprofiler.cpp
  - status string for context conflict
- lib/rocprofiler/context/context.*
  - correlation_id struct
  - new get_active_contexts paradigm
- lib/rocprofiler/counters/core.*
  - rocprofiler_packet union
  - tweak start/stop context to accept pointer instead of handle
- lib/rocprofiler/counters/dimensions.cpp
  - update to new get_rocp_agent() return type
- lib/rocprofiler/hsa/hsa.*
  - update to new get_active_contexts paradigm
  - update to new correlation id implementation
  - guard against hsa.def.cpp direct compilation
- lib/rocprofiler/hsa/queue_controller.*
  - update to change in get_rocp_agent return type
  - consistent aliases
  - lookup function for getting queue pointer from hsa queue id
- lib/rocprofiler/hsa/queue.*
  - rocprofiler_packet
  - extend queue_info_session_t
- lib/rocprofiler/tests/registration.cpp
  - improve diagnostic on perf check for rocprofiler_lib.callback_registration_lambda_with_result
2023-11-06 21:59:31 -06:00
Saurabh Verma 63775f241a Evaluation portion for metrics (#123)
* EvaluateAST and validation of RawAST

* Adding MetricDimension class and concepts

* set_dimensions() and improved ValidateRawAST()

* source formatting (clang-format v11) (#124)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Addressing 1st round of review comments

* Modified the parser production rules to support the right syntax for REDUCE and SELECT derived metric expressions

* changes to raw_ast.hpp and fmt::format()

* Parser tests updated to support corrected REDUCE and SELECT syntax

* changes to EvaluateAST::set_dimensions() and other dimension related code changes

* Added a test for EvaluateAST::evaluate() to test basic arithmetic on EvaluateAST

* Format source code (via clang-format v11) on sauverma/evaluate-ast (#146)

* source formatting (clang-format v11)

* Add dimension information to counter record

Restructures counter records to have the following design:

rocprofiler_record_id_t which is an int64_t that encodes
both the counter id and dimension information for the
record. The first 16 bits are reserved for the counter id while
the last 48 are split among the dimensions specified in
rocprofiler_dimension_t (currently 8 bits per dimension).
Each of the 8 bits for the dimension stores the dimension
value for that dimension for this record (i.e. a value of 8
on dimension XCC would denote XCC[8] for the counter). The
split among the dimensions will automatically adjust as
dimensions are added or removed.

The record also contains a union of {int64_t hw_counter, double
derived_counter} to specify the value of the record at
rocprofiler_record_id_t. int64_t denotes a physical hardware
counter that has integer types while the double is used for derived
counters (which type this counters values are needs to be queried
separately).

* Integration of new id type + other fixes

---------

Co-authored-by: sauverma93 <sauverma93@users.noreply.github.com>
Co-authored-by: Benjamin Welton <bewelton@amd.com>

* Fixed sissues with reduce() implementation and added a test for reduce()

* Updated parser syntax for reduce() and updated the parser test. Disabled the test for select()

* Build warning fixes

* Modifications to support fetching xcc/etc info from agent

* Initial plumbing working for single counters, cleanup+tests still needed

* Remove string comparison from reduce ops

* source formatting (clang-format v11) (#163)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* cmake formatting (cmake-format) (#164)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* source formatting (clang-format v11) (#171)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Merged with master

* source formatting (clang-format v11) (#172)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* source formatting (clang-format v11) (#173)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Test fix

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: bwelton <bwelton@users.noreply.github.com>
Co-authored-by: sauverma93 <sauverma93@users.noreply.github.com>
Co-authored-by: Benjamin Welton <bewelton@amd.com>
2023-11-03 21:10:40 -07:00
Jonathan R. Madsen 4f2dc896d3 Support Tool Intercept API Tables (#165)
* Update include/rocprofiler

- intercept_table.h header
- generic rocprofiler_runtime_library_t
- rocprofiler_internal_thread_library_t is not typedef for rocprofiler_runtime_library_t
- rocprofiler_at_runtime_api_registration

* Update lib/rocprofiler

- minor tweaks to context.cpp
  - check if none context early
  - disallow stop_context when finalizing
- add intercept_table.hpp and intercept_table.cpp
  - implement rocprofiler_at_runtime_api_registration
  - implement notify_runtime_api_registration
- update registration.cpp
  - invoke notify_runtime_api_registration
  - tweak to fini status when invoking client finalizer

* Update lib/rocprofiler/tests

- add tests for intercept table

* Add samples/intercept_table

- demonstrate how to install custom API function wrappers instead of relying on HSA callback tracing

* Update lib/rocprofiler/tests/intercept_table.cpp

- remove _SERVICE from ROCPROFILER_SERVICE_

* Update include/rocprofiler/intercept_table.h

- Update doxygen comments

* Update lib/rocprofiler/intercept_table.cpp

- return error config locked if already initialized

* Update lib/rocprofiler/intercept_table.cpp

- remove unnecessary alias

* Apply suggestions from code review

Co-authored-by: Tony Tye <Tony.Tye@amd.com>

* Update doxygen comments

- clarify when rocprofiler_at_runtime_api_registration can be invoked

* Use rocprofiler_runtime_library_t for intercept table and internal threading

- remove rocprofiler_intercept_library_t alias to rocprofiler_runtime_library_t
- remove rocprofiler_internal_thread_library_t alias to rocprofiler_runtime_library_t
- move around documentation with regard to rocprofiler_runtime_library_t enumeration
- added some extra doxygen documentation to internal threading functions

---------

Co-authored-by: Tony Tye <Tony.Tye@amd.com>
2023-11-02 19:10:10 -05:00
Jonathan R. Madsen 14373c57be Doxygen Improvements (#170)
* Doxygen updates

- Fix multiple @param where [in]/[out] was misplaced
- Fix @return
- Insert @retval
- Separate out installing conda environment from build docs step
2023-11-01 15:31:15 -05:00
Jonathan R. Madsen 033fd941e0 Remove SERVICE_ from ROCPROFILER_SERVICE_* enum vals (#168)
- these are unnecessary and are inconsistent with the name convention of everything else related to callback tracing
2023-10-31 15:06:03 -05:00
Jonathan R. Madsen cfbea0e5eb Update include/rocprofiler and lib/rocprofiler (#166)
- renamed inconsistent callback tracing types
- updated HIP and Marker API data structures (resemble HSA)
- cleaned up api_args.h and api_id.h headers
- cleaned up hsa.h, hip.h, and marker.h headers
- update to use (more consistent) name changes
- update code object data structs
- ROCPROFILER_SERVICE_CALLBACK_PHASE_{LOAD,UNLOAD} equivalent to ENTER, EXIT respectively
2023-10-31 12:48:24 -05:00
Jonathan R. Madsen 87cc748c3d Query callback and buffered tracing names (#135)
* Update include/rocprofiler/buffer_tracing.h

- add query functions for kind name, and kind operation name
- tweak iterate functions to not be specifically dedicated to names

* Update include/rocprofiler/callback_tracing.h

- add query functions for kind name, and kind operation name
- tweak iterate functions to not be specifically dedicated to names

* Update lib/rocprofiler/callback_tracing.cpp

- implement rocprofiler_query_callback_tracing_kind_name
- implement rocprofiler_query_callback_tracing_kind_name_buf
- implement rocprofiler_query_callback_tracing_kind_operation_name
- implement rocprofiler_query_callback_tracing_kind_operation_name_buf
- implement rocprofiler_iterate_callback_tracing_kinds
- implement rocprofiler_iterate_callback_tracing_kind_operations

* Update lib/rocprofiler/buffer_tracing.cpp

- implement rocprofiler_query_buffer_tracing_kind_name
- implement rocprofiler_query_buffer_tracing_kind_name_buf
- implement rocprofiler_query_buffer_tracing_kind_operation_name
- implement rocprofiler_query_buffer_tracing_kind_operation_name_buf
- implement rocprofiler_iterate_buffer_tracing_kinds
- implement rocprofiler_iterate_buffer_tracing_kind_operations

* Update lib/rocprofiler/tests/registration.cpp

- use new implementation for getting callback/buffer tracing names

* Update samples/api_buffered_tracing

- use new implementation for getting callback/buffer tracing names

* Update samples/api_callback_tracing

- use new implementation for getting callback/buffer tracing names

* Remove buffered query functions

- *_buf variants of the rocprofiler_query_X_tracing_Y functions were removed since we currently have no names requiring these functions

* Rename ROCPROFILER_STATUS_ERROR_DOMAIN_NOT_FOUND

- "DOMAIN" changed to "KIND" since former is more specific tracing whereas kind is used more generically
2023-10-19 15:21:07 -05:00
Jonathan R. Madsen 6a3f79e626 Update correlation id definition + status strings + const active contexts (#127)
* Update include/rocprofiler

- remove rocprofiler_external_correlation_id_t
- redefine rocprofiler_correlation_id_t to include internal id and external user data
- associate rocprofiler_push_external_correlation_id and rocprofiler_pop_external_correlation_id with a context

* Update include/rocprofiler/rocprofiler.h

- rocprofiler_get_status_name
- rocprofiler_get_status_string

* Update lib/rocprofiler/rocprofiler.cpp

- implement rocprofiler_get_status_name and rocprofiler_get_status_string

* Update lib/rocprofiler/tests/status.cpp

- unit test for status string and name

* Update lib/rocprofiler/tests/registration.cpp

- update to new rocprofiler_correlation_id_t

* Update samples

- update to new rocprofiler_correlation_id_t

* Add lib/rocprofiler/external_correlation.cpp

- placeholder for external correlation push/pop

* Update lib/rocprofiler/hsa/agent_cache.cpp

- slight tweak to when HSA_AMD_AGENT_INFO_NEAREST_CPU is defined

* Update context implementation and hsa.cpp

- get_active_contexts is array of const context pointers
- update hsa_api_impl<Idx>::functor to new rocprofiler_correlation_id_t

* Update include/rocprofiler/fwd.h

- add ROCPROFILER_STATUS_ERROR_INVALID_ARGUMENT
- reorder enum for consistency

* Update include/rocprofiler/external_correlation.h

- doxygen comments
- thread id parameter

* Update include/rocprofiler/rocprofiler.h

- add rocprofiler_get_thread_id function (needed for external corr id)

* Update lib/common/synchronized.hpp

- explicit LockedType
- define all copy/move ctor and assignment
- update rlock/wlock/ulock to support arguments and return values
- Support additional template parameter for special case of synchronized instance which is the mapped type of a sychronized map

* Update lib/rocprofiler/external_correlation.cpp

- implement rocprofiler_{push,pop}_external_correlation_id

* Update lib/rocprofiler/CMakeLists.txt

- external_correlation.hpp

* Update lib/rocprofiler/rocprofiler.cpp

- status string for ROCPROFILER_STATUS_ERROR_INVALID_ARGUMENT
- implement rocprofiler_get_thread_id

* Update lib/rocprofiler/tests (external correlation)

- add external_correlation unit tests

* Update include/rocprofiler/callback_tracing.h

- doxygen comments
- callback invoked in callback tracing has user_data pointer passed to it

* Update samples/api_callback_tracing/client.cpp

- add rocprofiler_user_data_t* to tool_tracing_callback

* Update lib/rocprofiler/tests/registration.cpp

- add rocprofiler_user_data_t* to tool_tracing_callback

* Update lib/rocprofiler/context/context.{hpp,cpp}

- update correlation_tracing_service
  - external_correlation instance
  - rename get_unique_record_id to get_unique_internal_id

* Update lib/tests/common/demangling.cpp

- tweak mangled definitions due to changing function get_unique_record_id to get_unique_internal_id

* Update lib/rocprofiler/hsa/hsa.cpp

- handle updates to external correlation id
- handle updates to callback signature in callback tracing

* Update CMakeLists.txt

- CMAKE_BUILD_TYPE=Coverage defines CODECOV=1

* Update samples/api_callback_tracing/client.cpp
2023-10-18 13:59:41 -05:00
Jonathan R. Madsen d1518c65b2 Miscellaneous Updates (const-correctness, logic fixes, etc.) (#126)
* Update lib/rocprofiler/hsa/hsa.cpp

- fix logic for constructing callback_contexts and buffered_contexts arrays

* Update include/rocprofiler/{agent,fwd,pc_sampling}.h

- remove rocprofiler_pc_sampling_config_array_t due to const problems
- update rocprofiler_agent_t to use arrays to const data
- remove redundant rocprofiler_query_pc_sampling_agent_configurations
  - this implementation is quite literally looking up info in the agent struct that was passed

* Update lib/rocprofiler/pc_sampling.cpp

- remove rocprofiler_query_pc_sampling_agent_configurations

* update lib/rocprofiler/agent.cpp

- handle const fields
- make mi200_pc_sampling_config variable static

* Update lib/rocprofiler/tests/agent.cpp

- tweak to pc_sampling_configs offset

* Update samples/pc_sampling

- Update sample to reflect minor tweaks to pc_sampling_configs in rocprofiler_agent_t

* Update CI workflow

- remove 'if: ${{ always() }}'
  - I suspect this is why the jobs do not cancel in progress correctly
2023-10-17 00:39:41 -05:00
Benjamin Welton 010693b795 Agent, Counters, and AQL (#55)
* Migrate XML counter defs and reader from v1/v2

* Current Working Set

* Modified parser

* Evaluate AST Start

* Update lib/common/xml

- move definitions out of class declaration

* Update lib/rocprofiler/counters/parser

- update build of bison and flex build
  - reproducible generation
- add ROCPROFILER_REGENERATE_COUNTERS_PARSER option
- fix namespacing

* Update lib/rocprofiler/counters/xml

- change location of XML files and install them

* Update lib/rocprofiler/counter/tests

- normalize the test names
- improve test failures (more clear about where failure is)

* Update lib/rocprofiler/counters

- fix namespace
- update to new XML metrics directory

* Update lib/rocprofiler/CMakeLists.txt

- link to object library

* Update lib/rocprofiler/hsa/types.hpp

- reorganize includes

* Add metric loading class/printers

* Agent Implementation

* Queue Implementation (#79)

* Queue Implementation

* API Implementation For Counters (part 1) (#80)

* API Implementation For Counters

* Bewelton/counter collection 3 (#84)

* Added counter sample

* More changes

* More changes

* Update samples/counter_collection

- mostly formatting

* Update include/rocprofiler/counters.h

- formatting

* Add lib.common/synchronized.hpp

- Synchronized struct

* Update lib/rocprofiler/counters/xml/basic_counters.xml

- whitespace

* Update scripts/patch-parser.cmake

- tweaks for consistency

* Update lib/rocprofiler/counters/parser/tests/parser_tests.cpp

- formatting

* Update lib/rocprofiler/counters/parser

- improve consistency in rocprofiler-expr-parser-patch
- update parser.{h,cpp} and scanner.cpp
  - formatting + regenerated

* Update lib/rocprofiler/aql

- formatting
- clang-tidy fixes
- guard against memory pool access errors

* Update lib/rocprofiler/aql/tests

- formatting
- update use of get_val
- normalize test names

* Update lib/rocprofiler/counters/tests

- formatting
- patch basic_counters and derived_counters
- normalize test names

* Update lib/rocprofiler/aql/tests

- set_tests_properties

* Update test labels

- fix minor issue with gtest labels

* Update lib/rocprofiler/counters

- formatting
- clang-tidy fixes

* Update lib/rocprofiler/hsa

- fix includes
- formatting
- clang-tidy fixes
- tweak to queue_controller_init interface

* Update lib/rocprofiler

- include fixes
- namespace fixes
- clang-tidy fixes
- formatting

* Update scripts/run-ci.py

- exclude counters/parser from code coverage (generated files)

* Update include/rocprofiler/counters.h

- fix doxygen comment

* Update lib/rocprofiler/aql/packet_construct.cpp

- guard against HSA_AMD_MEMORY_POOL_ACCESS_DISALLOWED_BY_DEFAULT and HSA_AMD_MEMORY_POOL_ACCESS_NEVER_ALLOWED

* Update lib/rocprofiler/counters/parser/raw_ast.hpp

- clang-tidy fixes

* Update lib/rocprofiler/counters/evaluate_ast.hpp

- clang-tidy fixes

* Update lib/rocprofiler/aql/tests

- disable packet_generation_single and packet_generation_multi tests
  - the entire implementation rocprofiler::get_ext_table() is incorrect

* Minor fixes before cleanup

* More changes

* More fixes

* More fixes

* source formatting (clang-format v11) (#99)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Revert PTL submodule

* Update scripts/run-ci.py

- exclude counters/parser from code coverage (generated files)

* Migrating counters state to context

* Linting

* source formatting (clang-format v11) (#101)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* revert run-ci

* Testing fixes

* More test changes

* Fix minor typo

* Small queue change

* Small queue change

* source formatting (clang-format v11) (#102)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* source formatting (clang-format v11) (#105)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Documentation Change

* More documentation fixes

* source formatting (clang-format v11) (#106)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Threading fixes

* Threading fixes

* source formatting (clang-format v11) (#107)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Threading fixes

* More test fixes

* More agent fixes

* More build fixes

* source formatting (clang-format v11) (#109)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* changed test timeouts

* Build fix

* Build fix

* Updates to agent

* source formatting (clang-format v11) (#114)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* cmake formatting (cmake-format) (#113)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* remove git worktree folder

* Doc update

* testing fix

* Another test fix

* More test changes

* Rebase

* source formatting (clang-format v11) (#116)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Documentation

* source formatting (clang-format v11) (#119)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* PTL Changes

* Minor agent fix for empty labels

* source formatting (clang-format v11) (#120)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Minor agent fix for empty labels

* Refactor read_map

* source formatting (clang-format v11) (#121)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Refactor read_map

* Cache fixes

* source formatting (clang-format v11) (#122)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

---------

Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: bwelton <bwelton@users.noreply.github.com>
2023-10-16 15:41:40 -05:00
Jonathan R. Madsen a798a26227 Agent information w/o using hsa-runtime (#100)
* Agent information w/o using hsa-runtime

- remove lib/rocprofiler/hsa/agent.{hpp,cpp}
- update include/rocprofiler/agent.h
  - basically all possible info from /sys/class/kfd/kfd/topology/nodes/*

* Print topology in rocprofiler_lib.agent test

- hack to help diagnose errors

* Update lib/rocprofiler/tests/details/agent.cpp

- use LOG_IF(WARNING, ...) instead of LOG_IF(FATAL, ...)

* Update lib/rocprofiler/tests/agent.cpp

- print rocminfo at beginning of test
- fix comparison of agent handle
- misc other checks

* Updte lib/rocprofiler/agent.cpp

- handle unreadable /sys/class/kfd/kfd/topology/nodes/<N>/properties file

* Update lib/tests/buffering/CMakeLists.txt

- increase timeout to 120
  - buffering.parallel will timeout when thread sanitizing is enabled

* Update cmake: rocprofiler-drm

- find drm headers and libraries

* Update include/rocprofiler/agent.h

- add family_id field

* Update lib/rocprofiler/agent.cpp

- parse /proc/cpuinfo for name, family, apicid, etc.
- read_topology uses unique pointers to cleanup memory allocations
- implement name and gfxip

* Update lib/rocprofiler/tests/agent.cpp

- improved failure message
- check name/gfxip
- remove check against hsa_agent_t.handle
  - this value is dependent on the address of C++ class

* Update lib/rocprofiler/tests/details/agent.cpp

- tweak gfxip_ variable which is broken for CPU

* Update lib/rocprofiler/agent.cpp

- update string handling for name and gfxip

* Update lib/rocprofiler/tests/agent.cpp

- minor output tweak

* Update lib/rocprofiler/registration.{hpp,cpp}

- registration::init_logging() function

* Update lib/rocprofiler/agent.cpp

- fix hex handling of GFX step version

* Update lib/rocprofiler/tests/details/agent.cpp

- fix format string when nearest CPUs not found

* Update lib/rocprofiler/tests/CMakeLists.txt

- exclude details/agent.cpp from being parsed for gtest tests

* Update include/rocprofiler/fwd.h

- add ROCPROFILER_STATUS_ERROR_INCOMPATIBLE_ABI status

* Update lib/rocprofiler/tests/details/agent.{hpp,cpp}

- replace with slightly modified implementation of rocminfo
  - primary change was not printing

* Update lib/rocprofiler/tests/agent.cpp

- update test to use rocminfo data

* Update lib/rocprofiler/agent.cpp

- add pc_sampling_configs
- return error on incompatible ABI

* Update counters and counters tests

- rename test names for consistency
- fixed incorrect spelling of derived

* Add lib/rocprofiler/tests/{timestamp,version}.cpp

- add timestamp and version tests for rocprofiler_get_timestamp and rocprofiler_get_version, respectively

* Update lib/rocprofiler/tests/agent

- fix double free of name_str from isa_info_t

* Update include/rocprofiler/agent.h

- comments for rocprofiler_agent_mem_bank_t
- add rocprofiler_dim3_t
- comments for rocprofiler_agent_t
- add new fields to rocprofiler_agent_t
    - cu_count
    - workgroup_max_size
    - workgroup_max_dim
    - grid_max_size
    - grid_max_dim
    - vendor_name
    - product_name
- change prototype of rocprofiler_available_agents_cb_t to be const agent**

* Update lib/rocprofiler/agent.cpp

- set size field
- implement:
  - product_name
  - vendor_name
  - workgroup_max_size
  - workgroup_max_dim
  - grid_max_size
  - grid_max_dim
  - cu_count

* Update lib/rocprofiler/tests/agent.cpp

- changes for const agent*

* Update samples/pc_sampling

- updates for const agent*

* Update lib/rocprofiler/agent.cpp

- fix ABI compatibility check
  - return incompatible if tool agent is larger than our agent

* Update include/rocprofiler/agent.h

- doxygen comments
- make size field of rocprofiler_agent_t uint64_t for consistency
- add gpu_id via /sys/class/kfd/kfd/.../<idx>/gpu_id
- add model_name via /sys/class/kfd/kfd/.../<idx>/name

* Update lib/rocprofiler/agent.cpp

- add read_file function (vector of strings)
- support enum in read_property
- assign model_name and gpu_id fields
- remove unique_id

* Update lib/rocprofiler/tests/details/agent.*

- support family id, ucode_version, sdma_ucode_version

* Update lib/rocprofiler/tests/agent.cpp

- Add rocprofiler_lib.agent_abi test
- Verify family_id, ucode_version, sdma_ucode_version
2023-10-10 18:10:23 -05:00
Jonathan R. Madsen 5e4e7b41f1 Documentation, sanitizers, and PTL submodule (#71)
* Update scripts/thread-sanitizer-suppr.txt

- ignore data race occasionally triggered by libamdhip64.so

* Update external/CMakeLists.txt

- configure PTL to use locks in task queues

* Update PTL submodule

- tweal to task queues to prevent data race from std::list next pointer

* Add scripts/setup-sanitizer-env.sh

- bash script that exports the {ASAN,LSAN,TSAN}_OPTIONS used by run-ci.py

* Update include/rocprofiler (doxygen)

- fix doxygen grouping

* Update docs workflow

- change concurrency group to be specific to workflow + ref
  - this prevents separate PRs triggering this workflow from cancelling each other
2023-09-20 23:14:25 -05:00
Jonathan R. Madsen d3eaacd610 Contexts, tracing, include reorg, registration, thread-pool (#65)
* Update scripts/update-doxygen.sh

- ensure build-docs folder exists

* Update scripts/run-ci.py

- exclude files in details subdirectory from code coverage

* Update scripts/thread-sanitizer-suppr.txt

- exclude races in glog

* Update docs/rocprofiler.dox.in

- exclude defines in include/rocprofiler/defines.h from doxygen
- Tweak EXCLUDE_PATTERNS and EXAMPLE_PATTERNS

* Update docs workflow

- trigger workflow whenever there is a change to the public headers (which may be doxygen comments)

* Update include/rocprofiler (reorg and overhaul)

- rocprofiler_status_t additions
  - CONTEXT_NOT_FOUND
  - CONTEXT_ERROR
  - INVALID_CONTEXT_ID
  - INVALID_CONTEXT
  - BUFFER_BUSY
- rocprofiler_context_is_active func
- rocprofiler_context_is_valid func
- rocprofiler_service_callback_tracing_kind_t update
  - remove ROCPROFILER_SERVICE_CALLBACK_TRACING_HELPER_THREAD
- Remove rocprofiler_tracing_helper_thread_operation_t
- Remove rocprofiler_helper_thread_callback_tracer_data_t
- Added rocprofiler_internal_thread_library_t
- Added rocprofiler_at_internal_thread_create
- split rocprofiler.h into several smaller headers
- reworked rocprofiler_status_t values
- added doxygen comments for enums
- replaced rocprofiler_trace_record_operation_kind_t with rocprofiler_trace_operation_t
- use @ instead of / in doxygen comment in rocprofiler_plugin.h
- fix ref to ROCPROFILER_SERVICE_CALLBACK_TRACING_MARKER_API
- end group in fwd.h
- remove PROFILE_COUNTING group in dispatch_profile.h
- remove premature group close in callback_tracing.h
- hsa.h: remove rocprofiler_hsa_trace_data_t
- fwd.h: remove rocprofiler_tracer_callback_data_t
- rename rocprofiler_correlation_id_t.handle to rocprofiler_correlation_id_t.id (consistency)
- fwd.h: add rocprofiler_callback_tracing_record_t
- callback_tracing.h: update rocprofiler_hsa_api_callback_tracer_data_t
- callback_tracing.h: add size fields
- simplify rocprofiler_tracer_callback_t
- removed ROCPROFILER_NONNULL from rocprofiler_get_version
- added rocprofiler_get_timestamp
- ROCPROFILER_STATUS_ERROR_CONFIGURATION_LOCKED in rocprofiler_status_t
- add ROCPROFILER_STATUS_ERROR_THREAD_NOT_FOUND rocprofiler_status_t
- add rocprofiler_buffer_category_t
- rocprofiler_trace_operation_t -> rocprofiler_tracing_operation_t
- rocprofiler_user_data_t union
- tweak rocprofiler_callback_tracing_record_t
  - make external_correlation_id non-pointer
  - add rocprofiler_user_data_t data field
- tweak rocprofiler_record_header_t
  - instead of single uint64_t kind field, have union for category + kind (two u32) with u64 hash
- API extensions for kind id <-> kind string
- API extensions for operation id <-> operation string
- rocprofiler_callback_trace_kind_name_cb_t
- rocprofiler_callback_trace_operation_name_cb_t
- rocprofiler_iterate_callback_trace_kind_names
- rocprofiler_iterate_callback_trace_kind_operation_names
- modify rocprofiler_hsa_api_callback_tracer_data_t data members (remove pointers)
- add rocprofiler_callback_trace_operation_args_cb_t function pointer typedef
- add rocprofiler_iterate_callback_trace_operation_args function
- fixed inconsistent use of *_trace_* vs. *_tracing_* (opting for tracing)
- removed rocprofiler_query_callback_trace_kind_name
- removed rocprofiler_query_callback_kind_operation_name
- Add include/rocprofiler/registration.h
  - header dedicated to registering a tool/client with rocprofiler
  - this header is not intended to be included by rocprofiler.h
  - rocprofiler_client_id_t
    - identifier for client tool
  - rocprofiler_client_finalize_t
    - function pointer prototype for tool-initiated finalization
  - rocprofiler_tool_initialize_t
    - function pointer prototype for tool initialization (i.e. configuration)
  - rocprofiler_tool_finalize_t
    - function pointer prototype for tool finalization
  - rocprofiler_tool_configure_result_t
    - struct returned by tool/client to rocprofiler
  - rocprofiler_is_initialized
    - function for querying whether tool-induced initialization is possible
  - rocprofiler_is_finalized
    - function for querying whether rocprofiler has been finalized
  - rocprofiler_configure prototype
    - this is the function tools implement
    - prototype is always marked as having default visibility
    - no implementation in rocprofiler
  - added typedef for rocprofiler_configure function pointer
  - added rocprofiler_force_configure to explicitly invoke rocprofiler_configure instead of relying on lazy init
- made callback typedef names more consistent (_cb_t suffix)
- typedef for rocprofiler_internal_thread_library_cb_t function pointer
- added rocprofiler_at_internal_thread_create function
- added rocprofiler_callback_thread_t struct
- added rocprofiler_create_callback_thread function
- added rocprofiler_assign_callback_thread function
- removed rocprofiler_buffer_tracing_record_header_t in favor of kind and correlation id in each record type
- added rocprofiler_buffer_tracing_kind_name_cb_t typedef
- added rocprofiler_buffer_tracing_operation_name_cb_t typedef
- added rocprofiler_iterate_buffer_tracing_kind_names function
- added rocprofiler_iterate_buffer_tracing_kind_operation_names function
- removed rocprofiler_query_buffer_trace_kind_name function
- removed rocprofiler_query_buffer_kind_operation_name function

* Update lib/common/container/stable_vector.hpp

- include limits header
- reserve_size struct
- overload stable_vector constructor to support reserving as part of construction

* Update lib/common/container/record_header_buffer.{hpp,cpp}

- add emplace member function accepting category and kind (two u32 variables) instead of one u64 kind
- use std::shared_mutex to prevent data-race when reading m_headers
- record_header_buffer is now multiple writer, single reader
- add read_lock member function (shared)
- add read_unlock member function (shared)
- lock member function gets exclusive lock
- unlock member function releases exclusive lock

* Rename "config" to "context" + restructure + implement

- Restructure config files + license
  - move config files into lib/rocprofiler/config subfolder
  - rename some files
  - add license to some files which were missing it
- Rename config/helpers.hpp
  - rename to allocator.hpp
  - remove get_domain_max_ops
- Create config/domain.{hpp,cpp}
  - structures for handling tracing domains and ops
- Update config/config.{hpp,cpp}
  - buffer_instance struct
  - callback_tracing_service struct
  - buffer_tracing_service struct
  - config struct
  - allocate_{config,buffer} func
  - {validate,start,stop}_config funcs
  - get_registered_configs func
  - get_active_configs func
  - get_buffers func
- Update rocprofiler.cpp
  - Implement rocprofiler_create_context
  - Implement rocprofiler_start_context
  - Implement rocprofiler_stop_context
  - Implement rocprofiler_context_is_active
  - Implement rocprofiler_context_is_valid
  - Implement rocprofiler_flush_buffer
  - Implement rocprofiler_destroy_buffer
  - Implement rocprofiler_create_buffer
- Update lib/rocprofiler/hsa
  - use rocprofiler_tracer_activity_domain_t instead of rocprofiler_tracer_activity_domain_t
  - remove ROCPROFILER_TRACER_ACTIVITY_DOMAIN_HSA_API fromHSA_API_INFO_DEFINITION_* macros
- Update lib/rocprofiler/context/domain.*
  - fixes for domain_info (i.e. use correct enums)
  - update rocprofiler_status_t codes
  - fix template instantiations
- Update lib/rocprofiler/context/context.*
  - use rocprofiler_service_callback_tracing_kind_t instead of rocprofiler_tracer_activity_domain_t
  - rename correlation_context to correlation_tracing_service
  - fix domains in callback_tracing_service and buffer_tracing_service
  - unique_ptr for callback_tracer and buffered_tracer in context
- Update lib/rocprofiler/rocprofiler.cpp
  - implement rocprofiler_configure_callback_tracing_service
- Update lib/rocprofiler/hsa/ostream.hpp
  - include rocprofiler.h instead of tracer.hpp
- Update lib/rocprofiler/hsa
  - migration to use rocprofiler_hsa_api_callback_tracer_data_t instead of rocprofiler_hsa_trace_data_t
  - restructure hsa_api_impl<Idx>
    - remove phase_enter and phase_exit
    - add set_data_args (partial replacement for phase_enter)
    - functor handles the contexts
- Update lib/rocprofiler/rocprofiler.cpp
  - implement rocprofiler_get_version
- Update lib/rocprofiler/hsa/hsa.{hpp,cpp}
  - remove hsa_api_ prefix for functions already in hsa namespace
- Update lib/rocprofiler/context/context.{hpp,cpp}
  - add client_idx to context struct (tool identifier)
  - add push_client function to set client_idx before context is allocated
  - add pop_client function to remove client identifier from future context creations
  - implemented {registered,active}_contexts and buffers to use new container::reserve_size overload to stable_vector
  - fix implementation of start_context
  - fix implementation of stop_context
- Update lib/rocprofiler/rocprofiler.cpp
  - prevent context creation, buffer creation, pc sampling config, etc. after initialization
  - add nullptr checks to rocprofiler_context_is_valid
  - fix rocprofiler_configure_callback_tracing_service
    - was checking size of buffers, not registered context
  - implement rocprofiler_iterate_callback_trace_kind_names
  - implement rocprofiler_iterate_callback_trace_kind_operation_names
- Update lib/rocprofiler/CMakeLists.txt
  - add registration.{hpp,cpp} to rocprofiler-library target sources
- Update lib/rocprofiler/hsa/utils.hpp
  - fix using fmt::formt with const char* strings
  - remove join functions (no longer used)
- Update lib/rocprofiler/hsa/hsa.{hpp,cpp}
  - remove args_string function
  - remove named_args_string function
  - update iterate_args function
    - change callback type
    - accept user data
  - rework the hsa_api_impl<Idx>::functor function
    - save the rocprofiler_callback_tracing_record_t between callbacks
  - update update_table function
    - check buffered_tracer domains
  - remove comments
- Update lib/rocprofiler/hsa/defines.hpp
  - remove MEMBER_<N> macros
  - add ADDR_MEMBER_<N> macros
  - remove doxygen comments for GET_MEMBER_FIELDS
  - add GET_ADDR_MEMBER_FIELDS
  - update HSA_API_INFO_DEFINITION_{0,V}
    - rename domain_idx to callback_domain_idx
    - add buffered_domain_idx
    - add as_arg_addr function
- Update lib/rocprofiler/rocprofiler.cpp
  - implement rocprofiler_iterate_callback_trace_operation_args
- Remove lib/rocprofiler/tracing.{hpp,cpp} and lib/rocprofiler/CMakeLists.txt
  - unused
- Update lib/rocprofiler/hsa/hsa.{hpp,cpp}
  - support buffered tracing in hsa_api_impl<Idx>::functor
  - rocprofiler_callback_trace_operation_args_cb_t -> rocprofiler_callback_tracing_operation_args_cb_t
    - i.e. trace -> tracing
- Update lib/rocprofiler/context/context.{hpp,cpp}
  - removed buffer_instance struct
  - removed allocate_buffer function
  - removed get_buffers function
  - changed buffer_tracing_service::buffer_array_t
- Update lib/rocprofiler/hsa: hsa.cpp, ostream.hpp, details folder
  - move ostream.hpp into details folder to prevent from contributing to code coverage
  - update cmake build system for new directory

* Add lib/rocprofiler/registration.{hpp,cpp}

- implements rocprofiler_set_api_table (called by rocprofiler-register)
- miscellaneous functions for client configure/initialize/finalize
- functions for querying the init/fini status
- relocated OnLoad HSA workaround to this file
  - at present, this is used to workaround ROCr not having rocprofiler-register integration yet
- implement rocprofiler_force_configure function
- implement rocprofiler_is_initialized function
- implement rocprofiler_is_finalized function
- ensure configure functions only invoked once
- ensure internal thread creation notification functions are invoked
- get_status is pair of atomics
- fix heap-use-after-free in init_logging
- update finalize
  - invoke hsa_shut_down
  - set all active contexts to null pointers

* Add lib/rocprofiler/buffer_tracing.cpp

- contains implementations of buffer_tracing (i.e. rocprofiler/buffer_tracing.h)
- previous implementation may have been moved out of lib/rocprofiler/rocprofiler.cpp

* Add lib/rocprofiler/buffer.{hpp,cpp}

- contains implementations of buffer (i.e. rocprofiler/buffer.h) and misc internal access functions
- previous implementation may have been moved out of lib/rocprofiler/rocprofiler.cpp and lib/rocprofiler/context/context.{hpp,cpp}

* Add lib/rocprofiler/callback_tracing.cpp

- contains implementations of callback_tracing (i.e. rocprofiler/callback_tracing.h)
- previous implementation may have been moved out of lib/rocprofiler/rocprofiler.cpp

* Add lib/rocprofiler/context.cpp

- contains implementations of context public API functions (i.e. rocprofiler/context.h)
- previous implementation may have been moved out of lib/rocprofiler/rocprofiler.cpp

* Add lib/rocprofiler/internal_threading.{hpp,cpp}

- contains implementations of internal_threading (i.e. rocprofiler/internal_threading.h)
- also contains implementations of internal access functions
- update finalize function
  - join all task groups and destroy all thread pools first, then reset unique_ptr

* Update lib/rocprofiler/rocprofiler.cpp

- rocprofiler_get_version returns status
- implement rocprofiler_get_timestamp
- remove misc implementations that were split into other files

* Update lib/rocprofiler/CMakeLists.txt

- compile new implementation files
  - buffer.cpp
  - buffer_tracing.cpp
  - callback_tracing.cpp
  - context.cpp
  - internal_threading.cpp

* Update lib/tests/buffering/buffering-*.cpp

- update to reflect changes to rocprofiler_record_header_t

* Update CMakeLists.txt

- increase minimum cmake version to 3.21 which added HIP support as a language

* Add samples/apps/transpose

- simple HIP application for testing

* Add samples/api_callback_tracing

- HIP application and tool library
- This effectively demos how to setup HSA API tracing
  - For each function called in tool, it stores the func/file/line and prints it during finalization
- client.hpp and client.cpp are the tool library
- Implement use of rocprofiler_iterate_callback_trace_operation_args
- add demo of using rocprofiler_get_version
- add_test
  - remove PASS_REGULAR_EXPRESSION
    - causing false passes during memcheck
  - add ROCPROFILER_MEMCHECK_PRELOAD_ENV to environment
- check if rocprofiler is initialized before stopping context

* Add samples/api_buffered_tracing

- Sample demonstrating tracing the HSA API via buffering
- demo rocprofiler_record_header_compute_hash
- throw exceptions for unexpected buffer data
- add_test
  - remove PASS_REGULAR_EXPRESSION
    - causing false passes during memcheck
  - add ROCPROFILER_MEMCHECK_PRELOAD_ENV to environment

* Update samples/CMakeLists.txt

- add subdirectory for api_callback_tracing
- add subdirectory api_buffered_tracing

* Update samples/pc_sampling/common.h

- fix processing of headers

* Update lib/rocprofiler/hsa/details/ostream.hpp

- fix data race on HSA_depth_max_cnt and recursion
- HSA_depth_max_cnt and recursion is now thread-local static instead of global static
- replace std::string usage with std::string_view

* Actions update

- add dependabot.yml
- use actions/checkout@v4
- install latest libasan and libtsan in sanitizer containers

* Add PTL (Parallel Tasking Library) submodule
2023-09-20 19:32:02 -05:00