92b7326910
* Adding json support minor bugs Fixing tests Fixing formatting issues Fixing test test fix Misc testing fixes Use rocprofiler/cxx/name_info in rocprofiler-sdk-tool fixes to reduce the Json file size Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/tool.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/tool.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/helper.hpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/helper.hpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/generateJSON.hpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/tool.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/tool.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/helper.hpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/tool.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> misc fixes Removing int cast for JSON tests formatting removing a condition test on Navi3 adding debug info Misc fix * CSV updates - fix stats - numerical formatter support for customizing write_csv_entry - misc formatting - get_marker_stats_file * Misc tests/rocprofv3/counter-collection/input2 fixes - rocprofiler_configure_pytest_files in rocprofv3/counter-collection/input2 - removed state code from merge in rocprofv3/counter-collection/input2 * Tool: "Agent-id" -> "Agent_Id" - consistency * Tool update - remove rocprofiler_tool_marker_record_t - add marker_tracing_kind_conversion - fix memory leak in write_json - minor update to get_output_stream - rework handling of marker records * Update tests/pytest-packages/pytest_utils/__init__.py - add collapse_dict_list function for converting a dictionary value that is a list of length one into a directly mapped value * Update tests/rocprofv3/**/conftest.py - use collapse_dict_list when reading in JSONs * Update tests/rocprofv3/counter-collection/input1/validate.py - relax testing requirements gfx1102 (AQLProfile bugs) - in addition to relaxed testing requirements for gfx1101 * Update tests/rocprofv3/tracing/validate.py - fix removal of PID in every marker record * Update tests/rocprofv3/tracing-plus-cc - remove test design that relies on iterating subdirectories * Wrapper around __libc_start_main - Ensures finalization happens before main returns - Update tests/rocprofv3/tracing/validate.py - wrapper around __libc_start_main changed roctx calls * Combine include/rocprofiler-sdk/cxx/serialization.hpp and include/rocprofiler-sdk/external/serialization.hpp - tests/common/serialization.hpp simply includes include/rocprofiler-sdk/cxx/serialization.hpp now * Update lib/rocprofiler-sdk/hip/hip.cpp - tracing function immediately returns when fini_status is non-zero * Update lib/rocprofiler-sdk/hsa/hsa.cpp - remove logging of tracing function when fini_status is non-zero * Update lib/rocprofiler-sdk-tool/CMakeLists.txt - remove rocprofv3_trigger_list_metrics.cpp from TOOL_SOURCES * Update tests/rocprofv3/tracing-plus-cc/CMakeLists.txt - fix depends * Domain statistics * Update tests/rocprofv3/tracing-plus-cc/CMakeLists.txt - do not set ROCP_LOG_LEVEL in env * Remove erroneous <bits/utility.h> include * Restructure tool source + reduce tool table + support multiple formats - buffered_output struct for handling output - support multiple output formats, e.g. --output-format csv,json - rename buffer_type_t -> domain_type - simplified generation of CSV output files - removed rocprofiler_tool_marker_record_t * Update lib/common/container/ring_buffer.hpp - value_type alias in ring_buffer<Tp> * Remove all but one json-execute tests - generate CSV and JSON in same run * Fix include for domain_type.cpp * Update tests/rocprofv3/tracing-plus-cc/input.txt - only specify counters which can be found on gfx8, gfx9, gfx10, gfx11, etc. - use :device= syntax * Update lib/rocprofiler-sdk-tool/config.cpp - support :device=N syntax for counters file - improve stripping comments in PMC files - only read after pmc: * Rework tool library counter collection - fatal error if all requested counters for device are not found - support :device= syntax * Update tests/rocprofv3/tracing-plus-cc/input.txt - removed L2CacheHit (not supported on mi300) * Disable JSON tests in tests/rocprofv3 * Update include/rocprofiler-sdk/cxx/serialization.hpp - support rocprofiler_record_dimension_info_t * Update tool JSON schema - remove domain_type::CODE_OBJECT - rocprofiler_tool_agent_v0_t - rocprofiler_agent_v0_t + counters - rocprofiler_tool_counter_info_t - get_code_object_data() * Update JSON schema for tool * Update lib/rocprofiler-sdk-tool/tool.cpp - fix ROCP_WARNING_IF * rocprofv3 -> rocprofv3.sh - install rocprofv3.sh into sbin - configure_file <source-tree>/rocprofv3.sh -> <binary-tree>/bin/rocprofv3 * Update tool counter collection - rocprofiler_tool_record_counter_t - rocprofiler_tool_counter_collection_record_t * Update tests/rocprofv3/counter-collection/input1/CMakeLists.txt - use rocprofiler_configure_pytest_files for validate.py, conftest.py, and input.txt * Update tests/rocprofv3/counter-collection/input1/validate.py - re-enable test_validate_counter_collection_pmc1_json * Update tests/rocprofv3/counter-collection/input2/validate.py - remove unused code * Update tests/rocprofv3/counter-collection/input2/validate.py - remove unused code * Update tests/rocprofv3/hsa-queue-dependency/validate.py - re-enable JSON tests * Misc tests/rocprofv3 CMake updates * Update tests/rocprofv3/tracing/validate.py - re-enable JSON tests * Update tests/rocprofv3/tracing-hip-in-libraries/validate.py - re-enable JSON tests * Update tests/rocprofv3/tracing/validate.py - remove unused node_exists function * Update tests/rocprofv3/tracing/validate.py - fix test_marker_api_trace_json --------- Co-authored-by: Sriraksha Nagaraj <Sriraksha.Nagaraj@amd.com>
140 строки
5.6 KiB
C++
140 строки
5.6 KiB
C++
// MIT License
|
|
//
|
|
// Copyright (c) 2023 Advanced Micro Devices, Inc. All rights reserved.
|
|
//
|
|
// Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
// of this software and associated documentation files (the "Software"), to deal
|
|
// in the Software without restriction, including without limitation the rights
|
|
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
// copies of the Software, and to permit persons to whom the Software is
|
|
// furnished to do so, subject to the following conditions:
|
|
//
|
|
// The above copyright notice and this permission notice shall be included in all
|
|
// copies or substantial portions of the Software.
|
|
//
|
|
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
// SOFTWARE.
|
|
|
|
#include "generateJSON.hpp"
|
|
#include "config.hpp"
|
|
#include "helper.hpp"
|
|
#include "output_file.hpp"
|
|
|
|
#include <rocprofiler-sdk/fwd.h>
|
|
#include <rocprofiler-sdk/marker/api_id.h>
|
|
|
|
#include <utility>
|
|
|
|
namespace rocprofiler
|
|
{
|
|
namespace tool
|
|
{
|
|
void
|
|
write_json(tool_table* tool_functions,
|
|
uint64_t pid,
|
|
std::vector<rocprofiler_agent_v0_t> agent_data,
|
|
std::vector<rocprofiler_tool_counter_info_t> counter_data,
|
|
std::deque<rocprofiler_buffer_tracing_hip_api_record_t>* hip_api_deque,
|
|
std::deque<rocprofiler_buffer_tracing_hsa_api_record_t>* hsa_api_deque,
|
|
std::deque<rocprofiler_buffer_tracing_kernel_dispatch_record_t>* kernel_dispatch_deque,
|
|
std::deque<rocprofiler_buffer_tracing_memory_copy_record_t>* memory_copy_deque,
|
|
std::deque<rocprofiler_tool_counter_collection_record_t>* counter_collection_deque,
|
|
std::deque<rocprofiler_buffer_tracing_marker_api_record_t>* marker_api_deque,
|
|
std::deque<rocprofiler_buffer_tracing_scratch_memory_record_t>* scratch_api_deque)
|
|
|
|
{
|
|
using JSONOutputArchive = cereal::MinimalJSONOutputArchive;
|
|
|
|
constexpr auto json_prec = 32;
|
|
constexpr auto json_indent = JSONOutputArchive::Options::IndentChar::space;
|
|
auto json_opts = JSONOutputArchive::Options{json_prec, json_indent, 1};
|
|
auto filename = std::string_view{"results"};
|
|
auto [output_stream, cleanup] = get_output_stream(filename, ".json");
|
|
|
|
{
|
|
auto json_ar = JSONOutputArchive{*output_stream, json_opts};
|
|
json_ar.setNextName("rocprofiler-sdk-tool");
|
|
json_ar.startNode();
|
|
|
|
json_ar.makeArray();
|
|
json_ar.startNode();
|
|
|
|
// metadata
|
|
{
|
|
json_ar.setNextName("metadata");
|
|
json_ar.startNode();
|
|
auto* timestamps = tool_functions->tool_get_app_timestamps_fn();
|
|
json_ar(cereal::make_nvp("pid", pid));
|
|
json_ar(cereal::make_nvp("init_time", timestamps->app_start_time));
|
|
json_ar(cereal::make_nvp("fini_time", timestamps->app_end_time));
|
|
json_ar.finishNode();
|
|
}
|
|
|
|
json_ar(cereal::make_nvp("agents", agent_data));
|
|
json_ar(cereal::make_nvp("counters", counter_data));
|
|
|
|
{
|
|
auto callback_name_info = get_callback_id_names();
|
|
auto buffer_name_info = get_buffer_id_names();
|
|
auto counter_dims = get_tool_counter_dimension_info();
|
|
auto marker_msg_data = get_callback_roctx_msg();
|
|
|
|
json_ar.setNextName("strings");
|
|
json_ar.startNode();
|
|
json_ar(cereal::make_nvp("callback_records", callback_name_info));
|
|
json_ar(cereal::make_nvp("buffer_records", buffer_name_info));
|
|
json_ar(cereal::make_nvp("marker_api", marker_msg_data));
|
|
|
|
{
|
|
json_ar.setNextName("counters");
|
|
json_ar.startNode();
|
|
json_ar(cereal::make_nvp("dimension_ids", counter_dims));
|
|
json_ar.finishNode();
|
|
}
|
|
|
|
json_ar.finishNode();
|
|
}
|
|
|
|
{
|
|
auto kern_sym_data = get_kernel_symbol_data();
|
|
auto code_obj_data = get_code_object_data();
|
|
|
|
json_ar(cereal::make_nvp("code_objects", code_obj_data));
|
|
json_ar(cereal::make_nvp("kernel_symbols", kern_sym_data));
|
|
}
|
|
|
|
{
|
|
json_ar.setNextName("callback_records");
|
|
json_ar.startNode();
|
|
json_ar(cereal::make_nvp("counter_collection", *counter_collection_deque));
|
|
json_ar.finishNode();
|
|
}
|
|
|
|
{
|
|
json_ar.setNextName("buffer_records");
|
|
json_ar.startNode();
|
|
json_ar(cereal::make_nvp("kernel_dispatches", *kernel_dispatch_deque));
|
|
json_ar(cereal::make_nvp("hip_api", *hip_api_deque));
|
|
json_ar(cereal::make_nvp("hsa_api", *hsa_api_deque));
|
|
json_ar(cereal::make_nvp("memory_copy", *memory_copy_deque));
|
|
json_ar(cereal::make_nvp("scratch_api", *scratch_api_deque));
|
|
json_ar(cereal::make_nvp("marker_api", *marker_api_deque));
|
|
json_ar.finishNode();
|
|
}
|
|
|
|
json_ar.finishNode(); // end array
|
|
json_ar.finishNode();
|
|
}
|
|
|
|
*output_stream << std::flush;
|
|
if(cleanup) cleanup(output_stream);
|
|
}
|
|
|
|
} // namespace tool
|
|
} // namespace rocprofiler
|