0ba9f26a6a
* Adding json support
minor bugs
Fixing tests
Fixing formatting issues
Fixing test
test fix
Misc testing fixes
Use rocprofiler/cxx/name_info in rocprofiler-sdk-tool
fixes to reduce the Json file size
Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/tool.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/tool.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/helper.hpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/helper.hpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/generateJSON.hpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/tool.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/tool.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/helper.hpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/tool.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Update source/lib/rocprofiler-sdk-tool/generateJSON.cpp
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
misc fixes
Removing int cast for JSON tests
formatting
removing a condition test on Navi3
adding debug info
Misc fix
* CSV updates
- fix stats
- numerical formatter support for customizing write_csv_entry
- misc formatting
- get_marker_stats_file
* Misc tests/rocprofv3/counter-collection/input2 fixes
- rocprofiler_configure_pytest_files in rocprofv3/counter-collection/input2
- removed state code from merge in rocprofv3/counter-collection/input2
* Tool: "Agent-id" -> "Agent_Id"
- consistency
* Tool update
- remove rocprofiler_tool_marker_record_t
- add marker_tracing_kind_conversion
- fix memory leak in write_json
- minor update to get_output_stream
- rework handling of marker records
* Update tests/pytest-packages/pytest_utils/__init__.py
- add collapse_dict_list function for converting a dictionary value that is a list of length one into a directly mapped value
* Update tests/rocprofv3/**/conftest.py
- use collapse_dict_list when reading in JSONs
* Update tests/rocprofv3/counter-collection/input1/validate.py
- relax testing requirements gfx1102 (AQLProfile bugs)
- in addition to relaxed testing requirements for gfx1101
* Update tests/rocprofv3/tracing/validate.py
- fix removal of PID in every marker record
* Update tests/rocprofv3/tracing-plus-cc
- remove test design that relies on iterating subdirectories
* Wrapper around __libc_start_main
- Ensures finalization happens before main returns
- Update tests/rocprofv3/tracing/validate.py
- wrapper around __libc_start_main changed roctx calls
* Combine include/rocprofiler-sdk/cxx/serialization.hpp and include/rocprofiler-sdk/external/serialization.hpp
- tests/common/serialization.hpp simply includes include/rocprofiler-sdk/cxx/serialization.hpp now
* Update lib/rocprofiler-sdk/hip/hip.cpp
- tracing function immediately returns when fini_status is non-zero
* Update lib/rocprofiler-sdk/hsa/hsa.cpp
- remove logging of tracing function when fini_status is non-zero
* Update lib/rocprofiler-sdk-tool/CMakeLists.txt
- remove rocprofv3_trigger_list_metrics.cpp from TOOL_SOURCES
* Update tests/rocprofv3/tracing-plus-cc/CMakeLists.txt
- fix depends
* Domain statistics
* Update tests/rocprofv3/tracing-plus-cc/CMakeLists.txt
- do not set ROCP_LOG_LEVEL in env
* Remove erroneous <bits/utility.h> include
* Restructure tool source + reduce tool table + support multiple formats
- buffered_output struct for handling output
- support multiple output formats, e.g. --output-format csv,json
- rename buffer_type_t -> domain_type
- simplified generation of CSV output files
- removed rocprofiler_tool_marker_record_t
* Update lib/common/container/ring_buffer.hpp
- value_type alias in ring_buffer<Tp>
* Remove all but one json-execute tests
- generate CSV and JSON in same run
* Fix include for domain_type.cpp
* Update tests/rocprofv3/tracing-plus-cc/input.txt
- only specify counters which can be found on gfx8, gfx9, gfx10, gfx11, etc.
- use :device= syntax
* Update lib/rocprofiler-sdk-tool/config.cpp
- support :device=N syntax for counters file
- improve stripping comments in PMC files
- only read after pmc:
* Rework tool library counter collection
- fatal error if all requested counters for device are not found
- support :device= syntax
* Update tests/rocprofv3/tracing-plus-cc/input.txt
- removed L2CacheHit (not supported on mi300)
* Disable JSON tests in tests/rocprofv3
* Update include/rocprofiler-sdk/cxx/serialization.hpp
- support rocprofiler_record_dimension_info_t
* Update tool JSON schema
- remove domain_type::CODE_OBJECT
- rocprofiler_tool_agent_v0_t
- rocprofiler_agent_v0_t + counters
- rocprofiler_tool_counter_info_t
- get_code_object_data()
* Update JSON schema for tool
* Update lib/rocprofiler-sdk-tool/tool.cpp
- fix ROCP_WARNING_IF
* rocprofv3 -> rocprofv3.sh
- install rocprofv3.sh into sbin
- configure_file <source-tree>/rocprofv3.sh -> <binary-tree>/bin/rocprofv3
* Update tool counter collection
- rocprofiler_tool_record_counter_t
- rocprofiler_tool_counter_collection_record_t
* Update tests/rocprofv3/counter-collection/input1/CMakeLists.txt
- use rocprofiler_configure_pytest_files for validate.py, conftest.py, and input.txt
* Update tests/rocprofv3/counter-collection/input1/validate.py
- re-enable test_validate_counter_collection_pmc1_json
* Update tests/rocprofv3/counter-collection/input2/validate.py
- remove unused code
* Update tests/rocprofv3/counter-collection/input2/validate.py
- remove unused code
* Update tests/rocprofv3/hsa-queue-dependency/validate.py
- re-enable JSON tests
* Misc tests/rocprofv3 CMake updates
* Update tests/rocprofv3/tracing/validate.py
- re-enable JSON tests
* Update tests/rocprofv3/tracing-hip-in-libraries/validate.py
- re-enable JSON tests
* Update tests/rocprofv3/tracing/validate.py
- remove unused node_exists function
* Update tests/rocprofv3/tracing/validate.py
- fix test_marker_api_trace_json
---------
Co-authored-by: Sriraksha Nagaraj <Sriraksha.Nagaraj@amd.com>
[ROCm/rocprofiler-sdk commit: 92b7326910]
114 строки
3.6 KiB
C++
114 строки
3.6 KiB
C++
// MIT License
|
|
//
|
|
// Copyright (c) 2023 Advanced Micro Devices, Inc. All rights reserved.
|
|
//
|
|
// Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
// of this software and associated documentation files (the "Software"), to deal
|
|
// in the Software without restriction, including without limitation the rights
|
|
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
// copies of the Software, and to permit persons to whom the Software is
|
|
// furnished to do so, subject to the following conditions:
|
|
//
|
|
// The above copyright notice and this permission notice shall be included in all
|
|
// copies or substantial portions of the Software.
|
|
//
|
|
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
// SOFTWARE.
|
|
|
|
#pragma once
|
|
|
|
#include "helper.hpp"
|
|
#include "tmp_file.hpp"
|
|
|
|
#include "lib/common/container/ring_buffer.hpp"
|
|
#include "lib/common/logging.hpp"
|
|
#include "lib/common/units.hpp"
|
|
|
|
#include <fmt/format.h>
|
|
|
|
#include <deque>
|
|
#include <mutex>
|
|
#include <string>
|
|
#include <tuple>
|
|
#include <utility>
|
|
|
|
template <typename Tp>
|
|
using ring_buffer_t = rocprofiler::common::container::ring_buffer<Tp>;
|
|
|
|
std::string
|
|
compose_tmp_file_name(domain_type buffer_type);
|
|
|
|
template <typename Tp>
|
|
std::tuple<Tp*, tmp_file*>
|
|
get_tmp_file_buffer(domain_type type)
|
|
{
|
|
static Tp* _buffer = new Tp(rocprofiler::common::units::get_page_size());
|
|
static tmp_file* _tmp_file = new tmp_file(compose_tmp_file_name(type));
|
|
return std::tuple(_buffer, _tmp_file);
|
|
}
|
|
|
|
template <typename Tp>
|
|
void
|
|
offload_buffer(domain_type type)
|
|
{
|
|
auto [_tmp_buf, _tmp_file] = get_tmp_file_buffer<Tp>(type);
|
|
auto _lk = std::lock_guard<std::mutex>(_tmp_file->file_mutex);
|
|
[[maybe_unused]] static auto _success = _tmp_file->open();
|
|
auto& _fs = _tmp_file->stream;
|
|
_tmp_file->file_pos.emplace(_fs.tellg());
|
|
_tmp_buf->save(_fs);
|
|
_tmp_buf->clear();
|
|
CHECK(_tmp_buf->is_empty() == true);
|
|
}
|
|
|
|
template <typename Tp>
|
|
void
|
|
write_ring_buffer(Tp _v, domain_type type)
|
|
{
|
|
auto [_tmp_buf, _tmp_file] = get_tmp_file_buffer<ring_buffer_t<Tp>>(type);
|
|
auto* ptr = _tmp_buf->request(false);
|
|
if(ptr == nullptr)
|
|
{
|
|
offload_buffer<ring_buffer_t<Tp>>(type);
|
|
ptr = _tmp_buf->request(false);
|
|
CHECK(ptr != nullptr);
|
|
}
|
|
*ptr = std::move(_v);
|
|
}
|
|
|
|
template <typename Tp>
|
|
void
|
|
flush_tmp_buffer(domain_type type)
|
|
{
|
|
auto [_tmp_buf, _tmp_file] = get_tmp_file_buffer<Tp>(type);
|
|
if(!_tmp_buf->is_empty()) offload_buffer<Tp>(type);
|
|
}
|
|
|
|
template <typename Tp>
|
|
std::deque<Tp>
|
|
read_tmp_file(domain_type type)
|
|
{
|
|
auto _data = std::deque<Tp>{};
|
|
|
|
auto [_tmp_buf, _tmp_file] = get_tmp_file_buffer<Tp>(type);
|
|
auto _lk = std::lock_guard<std::mutex>{_tmp_file->file_mutex};
|
|
auto& _fs = _tmp_file->stream;
|
|
if(_fs.is_open()) _fs.close();
|
|
_tmp_file->open(std::ios::binary | std::ios::in);
|
|
for(auto itr : _tmp_file->file_pos)
|
|
{
|
|
_fs.seekg(itr); // set to the absolute position
|
|
if(_fs.eof()) break;
|
|
Tp _buffer;
|
|
_buffer.load(_fs);
|
|
_data.emplace_back(std::move(_buffer));
|
|
}
|
|
|
|
return _data;
|
|
}
|