AI/rocm-systems
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

rocprofv3: summary reports + more JSON metadata (#1029)

Browse Source 
* Move include/rocprofiler-sdk/cxx/details/delimit.hpp to tokenize.hpp

* Update docs/how-to/using-rocprofv3.rst

- fix code block indents
- reorder rocprofv3 options, limit them to important options
- add docs for `--runtime-trace`

* Update rocprofv3.py

- parser argument groups
- new `--runtime-trace` option
- new `--summary` option
- new `--summary-per-domain` option
- new `--summary-groups` option
- new `--summary-output-file` option
- new `--summary-units` option

* Update lib/rocprofiler-sdk/hsa/async_copy.cpp

- fix async copy operation names: add "MEMORY_COPY_" prefix

* lib/rocprofiler-sdk-tool: update statistics.{hpp,cpp}

- statistics<>::get_percent function
- stats_entry_t struct
- stats_formatter struct
- percentage struct
- std::to_string(::rocprofiler::tool::percentage)

* lib/rocprofiler-sdk-tool: update domain_type.{hpp,cpp}

- reorder domain_type enum values

* lib/rocprofiler-sdk-tool: update generateCSV.{hpp,cpp}

- separate writing CSV from accumulating statistics
- a lot of functionality was moved to statistics.{hpp,cpp}

* lib/rocprofiler-sdk-tool: update output_file.{hpp,cpp}

- output_stream_t struct
- get_output_stream(...) returns output_stream_t instance

* lib/rocprofiler-sdk-tool: update generateJSON.cpp

- update get_output_stream usage to output_stream_t

* lib/rocprofiler-sdk-tool: update generateOTF2.cpp

- header include order tweak

* lib/rocprofiler-sdk-tool: update buffered_output.hpp

- stats_data_t was renamed to stats_entry_t

* lib/rocprofiler-sdk-tool: update generatePerfetto.cpp

- header include tweak

* lib/rocprofiler-sdk-tool: update tmp_file_buffer.hpp

- emit warning message if write_ring_buffer fails after offloading instead of aborting
- prefer placement new instead of assignment in write_ring_buffer

* lib/rocprofiler-sdk-tool: add generateStats.{hpp,cpp}

- functions for accumulating statistics

* Update tests/rocprofv3/tracing-hip-in-libraries/CMakeLists.txt

- accommodate tweak to CSV output file name for HIP and HSA traces

* lib/rocprofiler-sdk-tool: update config.{hpp,cpp}

- new config variables
  - stats_summary
  - stats_summary_per_domain
  - summary_output
  - stats_summary_unit_value
  - stats_summary_unit
  - stats_summary_file
  - stats_summary_groups
- support output keys for hostname: %hostname% / %h

* lib/rocprofiler-sdk-tool: update tool.cpp

- support summary output

* Documentation fixes

* Test for summary output

* Update tests/bin/transpose to use more ROCTx

- also support building with the roctracer ROCTx

* Remove roctxMark from OTF2 + fix kernel-rename tests

- following more ROCTx calls in transpose, kernel-rename validation had to be updated

* JSON metadata + JSON summary

- add serialization support for config
- add serialization support for statistics
- additions to json spec
  - rocprofiler-sdk-tool/metadata/config
  - rocprofiler-sdk-tool/metadata/command
  - rocprofiler-sdk-tool/summary
- config output_keys support for NVIDIA %q{<ENV-VAR>} syntax
- config output_keys support keys within keys

* rocprofv3 --summary-groups warning if no domain matches

- emit warning if a regex in for summary groups did not match any domain names

* Compile fix for lib/rocprofiler-sdk-tool/tool.cpp

- get_config().scratch_memory_trace
- pass contributions to write_json

* Update rocprofv3.py to preload rocprofiler-sdk-roctx

- appended to LD_PRELOAD when args.marker_trace is enabled

* Fix ReST link errors about subtitle underline being too short

* Patch tokenization of config::stats_summary_groups

- guard against array values of empty strings

* Tweak rocprofv3 summary test

- input-summary.yaml (used by rocprofv3-test-summary-inp-yaml-execute) only provides one summary group regex

* Disable LD_PRELOAD of librocprofiler-sdk-roctx.so

- this causes problems in the sanitizers, will be addressed in another PR
This commit is contained in:
Jonathan R. Madsen
2024-09-09 11:20:55 -05:00
committed by GitHub
parent 1354452cf0
commit 395f01b689
40 changed files with 2293 additions and 580 deletions
Download Patch File Download Diff File Expand all files Collapse all files
source/lib/rocprofiler-sdk-tool/generateCSV.cpp
+77 -154
View File
@@ -21,9 +21,10 @@
// SOFTWARE.
#include "generateCSV.hpp"
#include "config.hpp"
#include "csv.hpp"
#include "generateStats.hpp"
#include "helper.hpp"
#include "lib/rocprofiler-sdk-tool/config.hpp"
#include "statistics.hpp"
#include <rocprofiler-sdk/fwd.h>
@@ -41,47 +42,6 @@ namespace tool
{
namespace
{
using stats_data_t = statistics<uint64_t, float_type>;
using stats_map_t = std::map<std::string_view, stats_data_t>;
struct percentage
{
float_type value = {};
friend std::ostream& operator<<(std::ostream& os, percentage val) { return (os << val.value); }
};
struct stats_formatter
{
template <typename Tp>
std::ostream& operator()(std::ostream& ofs, const Tp& _val) const
{
using value_type = common::mpl::unqualified_type_t<Tp>;
if constexpr(std::is_floating_point<value_type>::value)
{
constexpr value_type one_hundredth = 1.0e-2;
if(_val > one_hundredth)
ofs << std::setprecision(6) << std::fixed;
else
ofs << std::setprecision(8) << std::scientific;
}
else if constexpr(std::is_same<Tp, percentage>::value)
{
constexpr float_type one = 1.0;
constexpr float_type one_hundredth = 1.0e-2;
if(_val.value >= one)
ofs << std::setprecision(2) << std::fixed;
else if(_val.value > one_hundredth)
ofs << std::setprecision(4) << std::fixed;
else
ofs << std::setprecision(3) << std::scientific;
}
return ofs;
}
};
tool::output_file
get_stats_output_file(std::string name)
{
@@ -99,10 +59,10 @@ get_stats_output_file(std::string name)
}};
}
stats_data_t
write_stats(output_file&& ofs, const stats_map_t& data_v)
void
write_stats(output_file&& ofs, const stats_entry_vec_t& data_v)
{
auto data = std::vector<std::pair<std::string_view, stats_data_t>>{};
auto data = stats_entry_vec_t{};
auto _duration = stats_data_t{};
for(const auto& [id, value] : data_v)
{
@@ -136,8 +96,6 @@ write_stats(output_file&& ofs, const stats_map_t& data_v)
value.get_stddev());
ofs << _row.str() << std::flush;
}
return _duration;
}
} // namespace
@@ -275,14 +233,17 @@ generate_csv(tool_table* /*tool_functions*/, std::vector<rocprofiler_agent_v0_t>
}
}
stats_data_t
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_kernel_dispatch_record_t>& data)
const std::deque<rocprofiler_buffer_tracing_kernel_dispatch_record_t>& data,
const stats_entry_t& stats)
{
if(data.empty()) return stats_data_t{};
if(data.empty()) return;
auto kernel_stats = stats_map_t{};
auto ofs = tool::output_file{"kernel_trace",
if(tool::get_config().stats && stats)
write_stats(get_stats_output_file("kernel_stats"), stats.entries);
auto ofs = tool::output_file{"kernel_trace",
tool::csv::kernel_trace_csv_encoder{},
{"Kind",
"Agent_Id",
@@ -329,27 +290,21 @@ generate_csv(tool_table*
record.dispatch_info.grid_size.y,
record.dispatch_info.grid_size.z);
if(tool::get_config().stats)
kernel_stats[kernel_name] += (record.end_timestamp - record.start_timestamp);
ofs << row_ss.str();
}
auto _duration = stats_data_t{};
if(tool::get_config().stats)
_duration = write_stats(get_stats_output_file("kernel_stats"), kernel_stats);
return _duration;
}
stats_data_t
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_hip_api_record_t>& data)
const std::deque<rocprofiler_buffer_tracing_hip_api_record_t>& data,
const stats_entry_t& stats)
{
if(data.empty()) return stats_data_t{};
if(data.empty()) return;
auto hip_stats = stats_map_t{};
auto ofs = tool::output_file{"hip_api_trace",
if(tool::get_config().stats && stats)
write_stats(get_stats_output_file("hip_api_stats"), stats.entries);
auto ofs = tool::output_file{"hip_api_trace",
tool::csv::api_csv_encoder{},
{"Domain",
"Function",
@@ -372,28 +327,21 @@ generate_csv(tool_table* tool
record.start_timestamp,
record.end_timestamp);
if(tool::get_config().stats)
hip_stats[api_name] += (record.end_timestamp - record.start_timestamp);
ofs << row_ss.str();
}
auto _duration = stats_data_t{};
if(tool::get_config().stats)
{
_duration = write_stats(get_stats_output_file("hip_stats"), hip_stats);
}
return _duration;
}
stats_data_t
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_hsa_api_record_t>& data)
const std::deque<rocprofiler_buffer_tracing_hsa_api_record_t>& data,
const stats_entry_t& stats)
{
if(data.empty()) return stats_data_t{};
if(data.empty()) return;
auto hsa_stats = stats_map_t{};
auto ofs = tool::output_file{"hsa_api_trace",
if(tool::get_config().stats && stats)
write_stats(get_stats_output_file("hsa_api_stats"), stats.entries);
auto ofs = tool::output_file{"hsa_api_trace",
tool::csv::api_csv_encoder{},
{"Domain",
"Function",
@@ -417,28 +365,21 @@ generate_csv(tool_table* tool
record.start_timestamp,
record.end_timestamp);
if(tool::get_config().stats)
hsa_stats[api_name] += (record.end_timestamp - record.start_timestamp);
ofs << row_ss.str();
}
auto _duration = stats_data_t{};
if(tool::get_config().stats)
{
_duration = write_stats(get_stats_output_file("hsa_stats"), hsa_stats);
}
return _duration;
}
stats_data_t
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_memory_copy_record_t>& data)
const std::deque<rocprofiler_buffer_tracing_memory_copy_record_t>& data,
const stats_entry_t& stats)
{
if(data.empty()) return stats_data_t{};
if(data.empty()) return;
auto memory_copy_stats = stats_map_t{};
auto ofs = tool::output_file{"memory_copy_trace",
if(tool::get_config().stats && stats)
write_stats(get_stats_output_file("memory_copy_stats"), stats.entries);
auto ofs = tool::output_file{"memory_copy_trace",
tool::csv::memory_copy_csv_encoder{},
{"Kind",
"Direction",
@@ -461,28 +402,21 @@ generate_csv(tool_table*
record.start_timestamp,
record.end_timestamp);
if(tool::get_config().stats)
memory_copy_stats[api_name] += (record.end_timestamp - record.start_timestamp);
ofs << row_ss.str();
}
auto _duration = stats_data_t{};
if(tool::get_config().stats)
{
_duration = write_stats(get_stats_output_file("memory_copy_stats"), memory_copy_stats);
}
return _duration;
}
stats_data_t
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_marker_api_record_t>& data)
const std::deque<rocprofiler_buffer_tracing_marker_api_record_t>& data,
const stats_entry_t& stats)
{
if(data.empty()) return stats_data_t{};
if(data.empty()) return;
auto marker_stats = stats_map_t{};
auto ofs = tool::output_file{"marker_api_trace",
if(tool::get_config().stats && stats)
write_stats(get_stats_output_file("marker_api_stats"), stats.entries);
auto ofs = tool::output_file{"marker_api_trace",
tool::csv::marker_csv_encoder{},
{"Domain",
"Function",
@@ -518,25 +452,19 @@ generate_csv(tool_table* t
record.start_timestamp,
record.end_timestamp);
if(tool::get_config().stats)
marker_stats[_name] += (record.end_timestamp - record.start_timestamp);
ofs << row_ss.str();
}
auto _duration = stats_data_t{};
if(tool::get_config().stats)
{
_duration = write_stats(get_stats_output_file("marker_stats"), marker_stats);
}
return _duration;
}
stats_data_t
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_tool_counter_collection_record_t>& data)
const std::deque<rocprofiler_tool_counter_collection_record_t>& data,
const stats_entry_t& stats)
{
if(data.empty()) return stats_data_t{};
if(data.empty()) return;
if(tool::get_config().stats && stats)
write_stats(get_stats_output_file("counter_collection_stats"), stats.entries);
auto ofs = tool::output_file{"counter_collection",
tool::csv::counter_collection_csv_encoder{},
@@ -601,14 +529,17 @@ generate_csv(tool_table* too
}
ofs << row_ss.str();
}
return stats_data_t{};
}
stats_data_t
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_scratch_memory_record_t>& data)
const std::deque<rocprofiler_buffer_tracing_scratch_memory_record_t>& data,
const stats_entry_t& stats)
{
if(data.empty()) return stats_data_t{};
if(data.empty()) return;
if(tool::get_config().stats && stats)
write_stats(get_stats_output_file("scratch_memory_stats"), stats.entries);
auto ofs = tool::output_file{"scratch_memory_trace",
tool::csv::scratch_memory_encoder{},
@@ -623,7 +554,6 @@ generate_csv(tool_table*
"End_Timestamp",
}};
auto scratch_memory_stats = stats_map_t{};
for(const auto& record : data)
{
auto row_ss = std::stringstream{};
@@ -641,45 +571,38 @@ generate_csv(tool_table*
record.start_timestamp,
record.end_timestamp);
if(tool::get_config().stats)
scratch_memory_stats[op_name] += (record.end_timestamp - record.start_timestamp);
ofs << row_ss.str();
}
auto _duration = stats_data_t{};
if(tool::get_config().stats)
{
_duration =
write_stats(get_stats_output_file("scratch_memory_stats"), scratch_memory_stats);
}
return _duration;
}
void
generate_csv(tool_table* /*tool_functions*/, std::unordered_map<domain_type, stats_data_t>& data)
generate_csv(tool_table* /*tool_functions*/, const domain_stats_vec_t& data_v)
{
using csv_encoder_t = rocprofiler::tool::csv::stats_csv_encoder;
if(!tool::get_config().stats) return;
auto _data = data_v;
auto _total_stats = stats_data_t{};
for(const auto& itr : data)
_total_stats += itr.second;
for(const auto& itr : _data)
_total_stats += itr.second.total;
if(_total_stats.get_count() == 0) return;
std::sort(_data.begin(), _data.end(), [](const auto& lhs, const auto& rhs) {
return (lhs.second.total.get_sum() > rhs.second.total.get_sum());
});
auto ofs = get_stats_output_file("domain_stats");
constexpr float_type one_hundred = 100.0;
const float_type _total_duration = _total_stats.get_sum();
for(const auto& [type, value] : data)
const float_type _total_duration = _total_stats.get_sum();
for(const auto& [type, value] : _data)
{
auto name = get_domain_column_name(type);
auto duration_ns = value.get_sum();
auto calls = value.get_count();
float_type avg_ns = value.get_mean();
float_type percent_v = (duration_ns / _total_duration) * one_hundred;
auto name = get_domain_column_name(type);
auto duration_ns = value.total.get_sum();
auto calls = value.total.get_count();
auto avg_ns = value.total.get_mean();
auto percent_v = value.total.get_percent(_total_duration);
auto _row = std::stringstream{};
csv_encoder_t::write_row<stats_formatter>(_row,
@@ -688,9 +611,9 @@ generate_csv(tool_table* /*tool_functions*/, std::unordered_map<domain_type, sta
duration_ns,
avg_ns,
percentage{percent_v},
value.get_min(),
value.get_max(),
value.get_stddev());
value.total.get_min(),
value.total.get_max(),
value.total.get_stddev());
ofs << _row.str() << std::flush;
}
}