ace34abd113afffc403189da5bd752b217bdb67f
10 İşleme
| Yazar | SHA1 | Mesaj | Tarih | |
|---|---|---|---|---|
 SrirakshaNag
|
ace34abd11 |
kernel iteration filtering for counter collection (#911)
* kernel filtering for counter collection * fixing trace tests * removing print statements * fix CI fail * handling preload and updating docs * minor fix * misc fix * misc fix * Typo fix * Update rocprofv3 + input schema - "application_passes" -> "jobs" - removed nesting in YAML/JSON inputs - improved customAction (now booleanArgAction) - supports --<name> (defaults to true) - supports --<name>=<truth-value> - supports --<name> <truth-value> - added --kernel-iteration-range to command-line - automatically support new command-line options in YAML/JSON input - standardized PMC return from text input to match PMC from YAML/JSON input - added support for --log-level env - updated various input*.(yml|json) to modified schema * Update config.cpp - added recommended code to get_kernel_filter_range * Fixing iteration * misc fix * support only [-] for iteration * bug fix * Fix using-rocprofv3.rst * Update config.cpp - patch get_kernel_filter_range --------- Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com> |
||
 Jonathan R. Madsen
|
957bb7a4e5 |
Adding Perfetto support (#867)
* Perfetto submodule * include/rocprofiler-sdk/cxx/perfetto.hpp - adapted from tests/common/perfetto.hpp - updated json-tool to use <rocprofiler-sdk/cxx/perfetto.hpp> * Update include/rocprofiler-sdk/cxx - add details/delimit.hpp - add details/join.hpp - extend details/mpl.hpp - extend details/operators.hpp * Update lib/rocprofiler-sdk/hsa/async_copy.cpp - update MEMORY_COPY direction names * Preliminary perfetto support * Update lib/rocprofiler-sdk-tool/generatePerfetto.cpp - fix getting roctx msg vs. buffer operation name * Temporary variable restructuring * Perfetto patches after rebasing onto main * Revert lib/rocprofiler-sdk/hsa/async_copy.cpp - revert name * Update lib/rocprofiler-sdk-tool/generatePerfetto.cpp - fix ReadTrace * Update tests/bin/hip-in-libraries - sleep_for * Support PFTRACE output format option in rocprofv3 * Change perfetto logging * Update rocprofv3 tests to generate pftrace output * Minor tweak to json-tool.cpp * Update requirements.txt for perfetto testing * Fix data race on amount_read in generatePerfetto.cpp * Add testing for pftrace output - relatively simple testing which verifies that the pftrace file has the same number of entries as JSON data for HIP/HSA/marker/kernel/memory_copy * Fix import in perfetto_reader.py * Fix data race in generatePerfetto.cpp |
||
|
Jonathan R. Madsen
|
92b7326910 |
Adding JSON support (#860)
* Adding json support minor bugs Fixing tests Fixing formatting issues Fixing test test fix Misc testing fixes Use rocprofiler/cxx/name_info in rocprofiler-sdk-tool fixes to reduce the Json file size Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/tool.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/tool.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/helper.hpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/helper.hpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/generateJSON.hpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/tool.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/tool.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/helper.hpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/tool.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> misc fixes Removing int cast for JSON tests formatting removing a condition test on Navi3 adding debug info Misc fix * CSV updates - fix stats - numerical formatter support for customizing write_csv_entry - misc formatting - get_marker_stats_file * Misc tests/rocprofv3/counter-collection/input2 fixes - rocprofiler_configure_pytest_files in rocprofv3/counter-collection/input2 - removed state code from merge in rocprofv3/counter-collection/input2 * Tool: "Agent-id" -> "Agent_Id" - consistency * Tool update - remove rocprofiler_tool_marker_record_t - add marker_tracing_kind_conversion - fix memory leak in write_json - minor update to get_output_stream - rework handling of marker records * Update tests/pytest-packages/pytest_utils/__init__.py - add collapse_dict_list function for converting a dictionary value that is a list of length one into a directly mapped value * Update tests/rocprofv3/**/conftest.py - use collapse_dict_list when reading in JSONs * Update tests/rocprofv3/counter-collection/input1/validate.py - relax testing requirements gfx1102 (AQLProfile bugs) - in addition to relaxed testing requirements for gfx1101 * Update tests/rocprofv3/tracing/validate.py - fix removal of PID in every marker record * Update tests/rocprofv3/tracing-plus-cc - remove test design that relies on iterating subdirectories * Wrapper around __libc_start_main - Ensures finalization happens before main returns - Update tests/rocprofv3/tracing/validate.py - wrapper around __libc_start_main changed roctx calls * Combine include/rocprofiler-sdk/cxx/serialization.hpp and include/rocprofiler-sdk/external/serialization.hpp - tests/common/serialization.hpp simply includes include/rocprofiler-sdk/cxx/serialization.hpp now * Update lib/rocprofiler-sdk/hip/hip.cpp - tracing function immediately returns when fini_status is non-zero * Update lib/rocprofiler-sdk/hsa/hsa.cpp - remove logging of tracing function when fini_status is non-zero * Update lib/rocprofiler-sdk-tool/CMakeLists.txt - remove rocprofv3_trigger_list_metrics.cpp from TOOL_SOURCES * Update tests/rocprofv3/tracing-plus-cc/CMakeLists.txt - fix depends * Domain statistics * Update tests/rocprofv3/tracing-plus-cc/CMakeLists.txt - do not set ROCP_LOG_LEVEL in env * Remove erroneous <bits/utility.h> include * Restructure tool source + reduce tool table + support multiple formats - buffered_output struct for handling output - support multiple output formats, e.g. --output-format csv,json - rename buffer_type_t -> domain_type - simplified generation of CSV output files - removed rocprofiler_tool_marker_record_t * Update lib/common/container/ring_buffer.hpp - value_type alias in ring_buffer<Tp> * Remove all but one json-execute tests - generate CSV and JSON in same run * Fix include for domain_type.cpp * Update tests/rocprofv3/tracing-plus-cc/input.txt - only specify counters which can be found on gfx8, gfx9, gfx10, gfx11, etc. - use :device= syntax * Update lib/rocprofiler-sdk-tool/config.cpp - support :device=N syntax for counters file - improve stripping comments in PMC files - only read after pmc: * Rework tool library counter collection - fatal error if all requested counters for device are not found - support :device= syntax * Update tests/rocprofv3/tracing-plus-cc/input.txt - removed L2CacheHit (not supported on mi300) * Disable JSON tests in tests/rocprofv3 * Update include/rocprofiler-sdk/cxx/serialization.hpp - support rocprofiler_record_dimension_info_t * Update tool JSON schema - remove domain_type::CODE_OBJECT - rocprofiler_tool_agent_v0_t - rocprofiler_agent_v0_t + counters - rocprofiler_tool_counter_info_t - get_code_object_data() * Update JSON schema for tool * Update lib/rocprofiler-sdk-tool/tool.cpp - fix ROCP_WARNING_IF * rocprofv3 -> rocprofv3.sh - install rocprofv3.sh into sbin - configure_file <source-tree>/rocprofv3.sh -> <binary-tree>/bin/rocprofv3 * Update tool counter collection - rocprofiler_tool_record_counter_t - rocprofiler_tool_counter_collection_record_t * Update tests/rocprofv3/counter-collection/input1/CMakeLists.txt - use rocprofiler_configure_pytest_files for validate.py, conftest.py, and input.txt * Update tests/rocprofv3/counter-collection/input1/validate.py - re-enable test_validate_counter_collection_pmc1_json * Update tests/rocprofv3/counter-collection/input2/validate.py - remove unused code * Update tests/rocprofv3/counter-collection/input2/validate.py - remove unused code * Update tests/rocprofv3/hsa-queue-dependency/validate.py - re-enable JSON tests * Misc tests/rocprofv3 CMake updates * Update tests/rocprofv3/tracing/validate.py - re-enable JSON tests * Update tests/rocprofv3/tracing-hip-in-libraries/validate.py - re-enable JSON tests * Update tests/rocprofv3/tracing/validate.py - remove unused node_exists function * Update tests/rocprofv3/tracing/validate.py - fix test_marker_api_trace_json --------- Co-authored-by: Sriraksha Nagaraj <Sriraksha.Nagaraj@amd.com> |
||
|
Jonathan R. Madsen
|
4d5b71b0e7 |
Update logging (#838)
* Update logging * Remove unused function * Fix lib/rocprofiler-sdk/hsa/pc_sampling.cpp logging compilation * Fix logging FLAGS_vmodule string leak and numerical log level * Update logging * Update glog submodule * Leak fixes * format |
||
|
Jonathan R. Madsen
|
b953774580 |
Improve testing stability (#796)
- Update tests/bin/reproducible-runtime - tweak algorithm for sleeping - Update lib/rocprofiler-sdk/hsa/async_copy - handle egregious skews for async copy times - Update kernel tracing - handle egregious skews for kernel dispatch times - Update lib/rocprofiler-sdk/hsa/code_object - use static object wrappers for code object info - Update lib/rocprofiler-sdk-tool/config.cpp - fix data race in output_keys / get_local_datetime |
||
|
Jonathan R. Madsen
|
87490d0018 |
Update lib/rocprofiler-sdk-tool (#755)
- questionable data race within std::regex in CI - simplify rocprofiler::tool::format - set config::tmp_directory to default to output_path - fs::create_directories for tmp_file - rework get_file_name(...) and compose_tmp_file_name(...) in tool.cpp |
||
|
SrirakshaNag
|
bef14ad1b2 |
rocprofiler-sdk-tool library intermediate binary output (#734)
* Support for binary temporary files * clang formatting * formating ring buffer.hpp * Update source/lib/common/container/ring_buffer.hpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fixing bugs * fix loop range * Fix for v3 test failures * bug fix * fix bug * fix memory leaks * destructing agent_info * Update CMakeLists.txt * clang-tidy fixes * Fix data race on destructor of rocprofiler_agent_t map in rocprofiler-sdk-tool library * Create lib/rocproifler-sdk-tool/tmp_file.* - move tmp_file class into separate header/implementation * Agent Info CSV in rocprofiler-sdk-tool - update tests to use agent_info.csv instead of rocminfo * Update lib/rocprofiler-sdk-tool/tool.cpp - use logical_node_id instead of node_id * Adding stats file * Adding tests for stats * Update scratch memory support - convert scratch memory support to use binary output * Tool Update: scratch memory stats + extended statistics - replace generate_*_csv with generate_csv overloads - added generate_csv for scratch memory - enable stats for scratch memory - replace ROCPROF_*_STATS env variables with ROCPROF_STATS env variable * rocprofv3 update - simple --stats option - add scratch memory trace to --sys-trace * Update tests/rocprofv3/tracing-hip-in-libraries - extend validate.py to test stats data - fix conftest.py for memory_copy_stats_data * Code coverage fixes - invoke __gcov_dump to ensure that code coverage is flushed after finalization --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Ammar ELWazir <ammar.elwazir@amd.com> Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com> |
||
 Benjamin Welton
|
41c0ddd72d |
Convert LOG() -> ROCP_X logging macros. (#695)
* Convert LOG() -> ROCP_X logging macros. This patch converts the LOG() macro to the ROCP_X logging macros. There are the following levels of logs. Logs whos expressions are not evaluated unless the log level is enabled: ROCP_TRACE - VLOG(2) (enabeled by env variable GLOG_v=2) ROCP_INFO - VLOG(1) (enabeled by env variable GLOG_v=1) Logs whos expressions are always evaluated: ROCP_WARNING - LOG(WARNING) ROCP_ERROR - LOG(ERROR) ROCP_FATAL - LOG(FATAL) ROCP_DFATAL - DLOG(FATAL) (only fatal in debug mode) * source formatting (clang-format v11) (#696) Co-authored-by: bwelton <1683479+bwelton@users.noreply.github.com> * Minor fix * Fixes for VLOG before main * fix vmodule * source formatting (clang-format v11) (#718) Co-authored-by: bwelton <1683479+bwelton@users.noreply.github.com> * memory leak fix * Vlog change --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: bwelton <1683479+bwelton@users.noreply.github.com> |
||
 Gopesh Bhardwaj
|
8a25b239bc |
Fixing counter collection in tools and enabling tests (#436)
* Fixing coutner colleciton in tools and enabling tests * fixing tests * improving coverage on test * Adding vector operations app * Fixing tools bug for counter collection * removing roctx linking |
||
|
Jonathan R. Madsen
|
1f4cf1aa39 |
Tools update (#397)
* Srnagara/tool counters collect (#331) * Adding counter collection capability to tools * Adding counter collection feature to tools * Adding counter collection capability to tools * Fixing merge down issues * Small tool fixes for build + prevent profile realloc * Reproducing the counter name query issue in buffered callback * Minor fix for init order + sample that directly uses sdk-tool for debug purposes * Adding a temporary fix to print the counter names * Fixing the output file name and reverting the changes of caching the profile config * Fixing SGPR_Count value * cleaning up debug prints * Adding header to counter collection file * Adding kernel filtering support * Remove threading * Cleaning up the code * Removing redundant prints * Revert "Remove threading" This reverts commit 05c58fb9de826e92cf8d2e3d1c31d5578525dcb4. * Revert "Cleaning up the code" This reverts commit 1d964882bf2396dee8ad020cbb6c83b36e0674e9. * Changing the tools code to align with init-order fix * cmake formatting (cmake-format) (#335) Co-authored-by: SrirakshaNag <SrirakshaNag@users.noreply.github.com> * source formatting (clang-format v11) (#336) Co-authored-by: SrirakshaNag <SrirakshaNag@users.noreply.github.com> * Adding support for async memory copy * source formatting (clang-format v11) (#391) Co-authored-by: SrirakshaNag <SrirakshaNag@users.noreply.github.com> * Fixing header typo * Fixing tool_fini * Replaceing the direction and kind fields values with description * Update lib/rocprofiler-sdk-tool/helper.cpp - Remove use of VLA * Update lib/rocprofiler-sdk-tool/tool.cpp - Formatting * Migrate common/config.* to rocprofiler-sdk-tool * Update lib/rocprofiler-sdk-tool/tool.cpp - fix clang-tidy issues * source formatting (clang-format v11) (#392) Co-authored-by: jrmadsen <jrmadsen@users.noreply.github.com> * Update lib/common/mpl.hpp - is_string_type / is_string_type_impl for deducing if type is a string type * Update include/rocprofiler-sdk/fwd.h - ROCPROFILER_BUFFER_TRACING_MEMORY_COPY_NONE starts at zero * Update lib/rocprofiler-sdk/hsa/async_copy.* - functions for operation ids and names * Update lib/rocprofiler-sdk/buffer_tracing.cpp - support iterating and getting names for ROCPROFILER_BUFFER_TRACING_MEMORY_COPY * Update lib/rocprofiler-sdk-tool/config.* - env ROCPROFILER_ prefix -> ROCPROF_ prefix - add support for memory copy tracing, counter collection, etc. * Update lib/rocprofiler-sdk-tool/helper.* - removed TracerFlushRecord - removed cxa_demangle (use one in common library) - removed GetCounterNames (handled in config) - removed GetKernelNames (handled in config) * Add lib/rocprofiler-sdk-tool/output_file.* - separate out get_output_stream function and output_file struct from tool.cpp * Add lib/rocprofiler-sdk-tool/csv.hpp - write_csv_entry automatically quotes strings - csv_encoder struct enforces correct number of columns * Update lib/rocprofiler-sdk-tool/CMakeLists.txt - add new files * Update lib/rocprofiler-sdk-tool/tool.cpp - update construction of output_file class - add kernel_symbol_data for serializing kernel trace data - use config instead of env lookups - optimize counter collection profile config lookup/creation * Update bin/rocprofv3 - rocprofv3 --help exits with 0 (as it should) - command-line arg for memory copy tracing - command-line arg for mangled kernels - command-line arg for truncated kernels - env ROCPROFILER_ prefix -> env ROCPROF_ prefix * Update tests/async-copy-tracing/validate.py - update test_async_copy_direction to new enum values * Update tests/kernel-tracing/validate.py - update test_async_copy_direction to new enum values * Update tests/tools/json-tool.cpp - add ROCPROFILER_BUFFER_TRACING_MEMORY_COPY to supported buffer_name_info * Update samples/counter_collection/{CMakeLists.txt,main.cpp} - remove counter-collection-sdk-tool * Update .github/workflows/docs.yml - fix paths triggering running the workflow --------- Co-authored-by: Benjamin Welton <bewelton@amd.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: SrirakshaNag <SrirakshaNag@users.noreply.github.com> Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com> Co-authored-by: jrmadsen <jrmadsen@users.noreply.github.com> * adding counter collection support * Adding counter collection test * changing directory structure of counter collection tests * Fixing test path for rocprofv3 * Adding hsa-tracing basic test * cmake formatting (cmake-format) (#362) Co-authored-by: bgopesh <bgopesh@users.noreply.github.com> * counter collection tests drop2 * fixing hsa-trace test for rocprofv3 path * python formatting (black) (#371) Co-authored-by: bgopesh <bgopesh@users.noreply.github.com> * both counter colleciton and tracing should work together * Fixing rocprofv3 path * Attempt to fix Segfault with AddressSanitizer * fixing sanitizer segfault * Update rocprofv3 * Update lib/rocprofiler-sdk-tool/README.md - update env variables * Update lib/rocprofiler-sdk/buffer_tracing.cpp - return ROCPROFILER_STATUS_BUFFER_NOT_FOUND if buffer tracing service is configured with invalid buffer * Update lib/rocprofiler-sdk-tool/tool.cpp - designated hsa API trace buffer * Update tests/hsa-tracing/CMakeLists.txt - Fix environment * Update rocprofv3 - do not override HSA_TOOLS_LIB - support ROCPROF_PRELOAD - LD_PRELOAD librocprofiler-sdk.so * Restructure tests directory - move all rocprofv3 integration tests into subfolder * Update cmake/Templates/rocprofiler-sdk/config.cmake.in - create rocprofiler-sdk::rocprofv3 cmake target * Update tests/rocprofv3/hsa-tracing - improve validate.py - convert input to dict via csv.DictReader * Update tests/apps/CMakeLists.txt - fix build rpath for simple-transpose * Update cmake/rocprofiler_memcheck.cmake - prefer libtsan.so.0 * Update tests/rocprofv3/hsa-tracing - move to tests/rocprofv3/tracing - include kernel tracing and memory copy tracing * Update lib/rocprofiler-sdk-tool/tool.cpp - normalize "_ID" vs. "_Id" in CSV column names (use "_Id") * Update lib/rocprofiler-sdk/buffer.{hpp,cpp} - change signature of buffer::get_buffers() - buffer::get_buffers() uses static_object * Update lib/rocprofiler-sdk/context/context.cpp - update usage of buffer::get_buffers() - now returns pointer * Update lib/rocprofiler-sdk/tests/buffer.cpp - update to change for signature of buffer::get_buffers() * Update tests/rocprofv3/tracing/CMakeLists.txt - use %argt% with -d argument * Update lib/rocprofiler-sdk-tool/tool.cpp - use atexit for finalization * Update tests/rocprofv3/tracing/CMakeLists.txt - tweaked name of tests * Update lib/rocprofiler-sdk/hsa/async_copy.* - async_copy_fini + reference counting signals * Update lib/rocprofiler-sdk/registration.cpp - invoke hsa::async_copy_fini() to prevent data race on signals --------- Co-authored-by: SrirakshaNag <104580803+SrirakshaNag@users.noreply.github.com> Co-authored-by: Benjamin Welton <bewelton@amd.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: SrirakshaNag <SrirakshaNag@users.noreply.github.com> Co-authored-by: gobhardw <gopesh.bhardwaj@amd.com> Co-authored-by: bgopesh <bgopesh@users.noreply.github.com> |