0ba9f26a6a
* Adding json support
minor bugs
Fixing tests
Fixing formatting issues
Fixing test
test fix
Misc testing fixes
Use rocprofiler/cxx/name_info in rocprofiler-sdk-tool
fixes to reduce the Json file size
Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/tool.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/tool.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/helper.hpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/helper.hpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/generateJSON.hpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/tool.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/tool.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/helper.hpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/tool.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
misc fixes
Removing int cast for JSON tests
formatting
removing a condition test on Navi3
adding debug info
Misc fix
* CSV updates
- fix stats
- numerical formatter support for customizing write_csv_entry
- misc formatting
- get_marker_stats_file
* Misc tests/rocprofv3/counter-collection/input2 fixes
- rocprofiler_configure_pytest_files in rocprofv3/counter-collection/input2
- removed state code from merge in rocprofv3/counter-collection/input2
* Tool: "Agent-id" -> "Agent_Id"
- consistency
* Tool update
- remove rocprofiler_tool_marker_record_t
- add marker_tracing_kind_conversion
- fix memory leak in write_json
- minor update to get_output_stream
- rework handling of marker records
* Update tests/pytest-packages/pytest_utils/__init__.py
- add collapse_dict_list function for converting a dictionary value that is a list of length one into a directly mapped value
* Update tests/rocprofv3/**/conftest.py
- use collapse_dict_list when reading in JSONs
* Update tests/rocprofv3/counter-collection/input1/validate.py
- relax testing requirements gfx1102 (AQLProfile bugs)
- in addition to relaxed testing requirements for gfx1101
* Update tests/rocprofv3/tracing/validate.py
- fix removal of PID in every marker record
* Update tests/rocprofv3/tracing-plus-cc
- remove test design that relies on iterating subdirectories
* Wrapper around __libc_start_main
- Ensures finalization happens before main returns
- Update tests/rocprofv3/tracing/validate.py
- wrapper around __libc_start_main changed roctx calls
* Combine include/rocprofiler-sdk/cxx/serialization.hpp and include/rocprofiler-sdk/external/serialization.hpp
- tests/common/serialization.hpp simply includes include/rocprofiler-sdk/cxx/serialization.hpp now
* Update lib/rocprofiler-sdk/hip/hip.cpp
- tracing function immediately returns when fini_status is non-zero
* Update lib/rocprofiler-sdk/hsa/hsa.cpp
- remove logging of tracing function when fini_status is non-zero
* Update lib/rocprofiler-sdk-tool/CMakeLists.txt
- remove rocprofv3_trigger_list_metrics.cpp from TOOL_SOURCES
* Update tests/rocprofv3/tracing-plus-cc/CMakeLists.txt
- fix depends
* Domain statistics
* Update tests/rocprofv3/tracing-plus-cc/CMakeLists.txt
- do not set ROCP_LOG_LEVEL in env
* Remove erroneous <bits/utility.h> include
* Restructure tool source + reduce tool table + support multiple formats
- buffered_output struct for handling output
- support multiple output formats, e.g. --output-format csv,json
- rename buffer_type_t -> domain_type
- simplified generation of CSV output files
- removed rocprofiler_tool_marker_record_t
* Update lib/common/container/ring_buffer.hpp
- value_type alias in ring_buffer<Tp>
* Remove all but one json-execute tests
- generate CSV and JSON in same run
* Fix include for domain_type.cpp
* Update tests/rocprofv3/tracing-plus-cc/input.txt
- only specify counters which can be found on gfx8, gfx9, gfx10, gfx11, etc.
- use :device= syntax
* Update lib/rocprofiler-sdk-tool/config.cpp
- support :device=N syntax for counters file
- improve stripping comments in PMC files
- only read after pmc:
* Rework tool library counter collection
- fatal error if all requested counters for device are not found
- support :device= syntax
* Update tests/rocprofv3/tracing-plus-cc/input.txt
- removed L2CacheHit (not supported on mi300)
* Disable JSON tests in tests/rocprofv3
* Update include/rocprofiler-sdk/cxx/serialization.hpp
- support rocprofiler_record_dimension_info_t
* Update tool JSON schema
- remove domain_type::CODE_OBJECT
- rocprofiler_tool_agent_v0_t
- rocprofiler_agent_v0_t + counters
- rocprofiler_tool_counter_info_t
- get_code_object_data()
* Update JSON schema for tool
* Update lib/rocprofiler-sdk-tool/tool.cpp
- fix ROCP_WARNING_IF
* rocprofv3 -> rocprofv3.sh
- install rocprofv3.sh into sbin
- configure_file <source-tree>/rocprofv3.sh -> <binary-tree>/bin/rocprofv3
* Update tool counter collection
- rocprofiler_tool_record_counter_t
- rocprofiler_tool_counter_collection_record_t
* Update tests/rocprofv3/counter-collection/input1/CMakeLists.txt
- use rocprofiler_configure_pytest_files for validate.py, conftest.py, and input.txt
* Update tests/rocprofv3/counter-collection/input1/validate.py
- re-enable test_validate_counter_collection_pmc1_json
* Update tests/rocprofv3/counter-collection/input2/validate.py
- remove unused code
* Update tests/rocprofv3/counter-collection/input2/validate.py
- remove unused code
* Update tests/rocprofv3/hsa-queue-dependency/validate.py
- re-enable JSON tests
* Misc tests/rocprofv3 CMake updates
* Update tests/rocprofv3/tracing/validate.py
- re-enable JSON tests
* Update tests/rocprofv3/tracing-hip-in-libraries/validate.py
- re-enable JSON tests
* Update tests/rocprofv3/tracing/validate.py
- remove unused node_exists function
* Update tests/rocprofv3/tracing/validate.py
- fix test_marker_api_trace_json
---------
Co-authored-by: Sriraksha Nagaraj <Sriraksha.Nagaraj@amd.com>
[ROCm/rocprofiler-sdk commit: 92b7326910]
160 righe
5.7 KiB
Python
160 righe
5.7 KiB
Python
#!/usr/bin/env python3
|
|
|
|
import sys
|
|
import pytest
|
|
|
|
|
|
def test_hsa_api_trace(hsa_trace_input_data):
|
|
functions = []
|
|
correlation_ids = []
|
|
for row in hsa_trace_input_data:
|
|
assert row["Domain"] in (
|
|
"HSA_CORE_API",
|
|
"HSA_AMD_EXT_API",
|
|
"HSA_IMAGE_EXT_API",
|
|
"HSA_FINALIZE_EXT_API",
|
|
)
|
|
assert int(row["Process_Id"]) > 0
|
|
assert int(row["Thread_Id"]) >= int(row["Process_Id"])
|
|
assert int(row["End_Timestamp"]) >= int(row["Start_Timestamp"])
|
|
functions.append(row["Function"])
|
|
correlation_ids.append(int(row["Correlation_Id"]))
|
|
|
|
correlation_ids = sorted(list(set(correlation_ids)))
|
|
|
|
# deterministic call counts
|
|
num_queue_create_calls = 2
|
|
num_queue_destroy_calls = 2
|
|
num_hsa_mem_free_calls = 2
|
|
|
|
# signal create/destroy calls
|
|
# although the app explicitly only creates 3 signals
|
|
# but hsa_init() internally calls hsa_signal_create
|
|
num_hsa_signal_create_calls = 4
|
|
num_hsa_signal_destroy_calls = 4
|
|
|
|
# all correlation ids are unique
|
|
assert len(correlation_ids) == len(hsa_trace_input_data)
|
|
|
|
functions = list(functions)
|
|
assert "hsa_shut_down" in functions
|
|
assert functions.count("hsa_queue_create") == num_queue_create_calls
|
|
assert functions.count("hsa_queue_destroy") == num_queue_destroy_calls
|
|
assert functions.count("hsa_memory_free") == num_hsa_mem_free_calls
|
|
assert functions.count("hsa_signal_create") == num_hsa_signal_create_calls
|
|
assert functions.count("hsa_signal_destroy") == num_hsa_signal_destroy_calls
|
|
|
|
|
|
def test_kernel_trace(kernel_trace_input_data):
|
|
valid_kernel_names = ("copyA", "copyB", "copyC")
|
|
|
|
assert len(kernel_trace_input_data) == 3
|
|
for row in kernel_trace_input_data:
|
|
assert row["Kind"] == "KERNEL_DISPATCH"
|
|
assert int(row["Agent_Id"]) > 0
|
|
assert int(row["Queue_Id"]) > 0
|
|
assert int(row["Kernel_Id"]) > 0
|
|
assert row["Kernel_Name"] in valid_kernel_names
|
|
assert int(row["Correlation_Id"]) > 0
|
|
assert int(row["Workgroup_Size_X"]) == 64
|
|
assert int(row["Workgroup_Size_Y"]) == 1
|
|
assert int(row["Workgroup_Size_Z"]) == 1
|
|
assert int(row["Grid_Size_X"]) == 64
|
|
assert int(row["Grid_Size_Y"]) == 1
|
|
assert int(row["Grid_Size_Z"]) == 1
|
|
assert int(row["End_Timestamp"]) >= int(row["Start_Timestamp"])
|
|
|
|
|
|
def test_kernel_trace_json(json_data):
|
|
data = json_data["rocprofiler-sdk-tool"]
|
|
valid_kernel_names = ["copyA", "copyB", "copyC"]
|
|
buffer_records = data["buffer_records"]
|
|
buffer_names = data["strings"]["buffer_records"]
|
|
kernel_dispatch_data = buffer_records["kernel_dispatches"]
|
|
|
|
def get_kernel_name(kernel_id):
|
|
return data["kernel_symbols"][kernel_id]["formatted_kernel_name"]
|
|
|
|
assert len(kernel_dispatch_data) == 3
|
|
|
|
for dispatch in kernel_dispatch_data:
|
|
|
|
assert buffer_names[dispatch["kind"]]["kind"] == "KERNEL_DISPATCH"
|
|
dispatch_info = dispatch["dispatch_info"]
|
|
assert dispatch_info["agent_id"]["handle"] > 0
|
|
assert dispatch_info["queue_id"]["handle"] > 0
|
|
assert dispatch_info["kernel_id"] > 0
|
|
|
|
kernel_name = get_kernel_name(dispatch_info["kernel_id"])
|
|
assert kernel_name in valid_kernel_names
|
|
|
|
assert dispatch["correlation_id"]["internal"] > 0
|
|
assert dispatch_info["workgroup_size"]["x"] == 64
|
|
assert dispatch_info["workgroup_size"]["y"] == 1
|
|
assert dispatch_info["workgroup_size"]["z"] == 1
|
|
assert dispatch_info["grid_size"]["x"] == 64
|
|
assert dispatch_info["grid_size"]["y"] == 1
|
|
assert dispatch_info["grid_size"]["z"] == 1
|
|
assert dispatch["end_timestamp"] >= dispatch["start_timestamp"]
|
|
|
|
|
|
def test_hsa_api_trace_json(json_data):
|
|
data = json_data["rocprofiler-sdk-tool"]
|
|
functions = []
|
|
correlation_ids = []
|
|
|
|
def get_operation_name(kind_id, op_id):
|
|
return data["strings"]["buffer_records"][kind_id]["operations"][op_id]
|
|
|
|
def get_kind_name(kind_id):
|
|
return data["strings"]["buffer_records"][kind_id]["kind"]
|
|
|
|
metadata = data["metadata"]
|
|
buffer_records = data["buffer_records"]
|
|
|
|
valid_domain_names = (
|
|
"HSA_CORE_API",
|
|
"HSA_AMD_EXT_API",
|
|
"HSA_IMAGE_EXT_API",
|
|
"HSA_FINALIZE_EXT_API",
|
|
)
|
|
|
|
assert metadata["pid"] > 0
|
|
hsa_api_data = buffer_records["hsa_api"]
|
|
|
|
for itr in hsa_api_data:
|
|
kind = get_kind_name(itr["kind"])
|
|
assert kind in valid_domain_names
|
|
assert itr["end_timestamp"] >= itr["start_timestamp"]
|
|
functions.append(get_operation_name(itr["kind"], itr["operation"]))
|
|
correlation_ids.append(itr["correlation_id"]["internal"])
|
|
|
|
correlation_ids = sorted(list(set(correlation_ids)))
|
|
|
|
# deterministic call counts
|
|
num_queue_create_calls = 2
|
|
num_queue_destroy_calls = 2
|
|
num_hsa_mem_free_calls = 2
|
|
|
|
# signal create/destroy calls
|
|
# although the app explicitly only creates 3 signals
|
|
# but hsa_init() internally calls hsa_signal_create
|
|
num_hsa_signal_create_calls = 4
|
|
num_hsa_signal_destroy_calls = 4
|
|
|
|
# all correlation ids are unique
|
|
assert len(correlation_ids) == len(hsa_api_data)
|
|
|
|
functions = list(functions)
|
|
assert "hsa_shut_down" in functions
|
|
assert functions.count("hsa_queue_create") == num_queue_create_calls
|
|
assert functions.count("hsa_queue_destroy") == num_queue_destroy_calls
|
|
assert functions.count("hsa_memory_free") == num_hsa_mem_free_calls
|
|
assert functions.count("hsa_signal_create") == num_hsa_signal_create_calls
|
|
assert functions.count("hsa_signal_destroy") == num_hsa_signal_destroy_calls
|
|
|
|
|
|
if __name__ == "__main__":
|
|
exit_code = pytest.main(["-x", __file__] + sys.argv[1:])
|
|
sys.exit(exit_code)
|