AI/rocm-systems
2
0
Rozštěpit 0
Zdrojový kód Úkoly Pull requesty Akce Balíčky Projekty Vydání Wiki Aktivita

rocprofv3: summary reports + more JSON metadata (#1029)

Procházet zdrojové kódy 
* Move include/rocprofiler-sdk/cxx/details/delimit.hpp to tokenize.hpp

* Update docs/how-to/using-rocprofv3.rst

- fix code block indents
- reorder rocprofv3 options, limit them to important options
- add docs for `--runtime-trace`

* Update rocprofv3.py

- parser argument groups
- new `--runtime-trace` option
- new `--summary` option
- new `--summary-per-domain` option
- new `--summary-groups` option
- new `--summary-output-file` option
- new `--summary-units` option

* Update lib/rocprofiler-sdk/hsa/async_copy.cpp

- fix async copy operation names: add "MEMORY_COPY_" prefix

* lib/rocprofiler-sdk-tool: update statistics.{hpp,cpp}

- statistics<>::get_percent function
- stats_entry_t struct
- stats_formatter struct
- percentage struct
- std::to_string(::rocprofiler::tool::percentage)

* lib/rocprofiler-sdk-tool: update domain_type.{hpp,cpp}

- reorder domain_type enum values

* lib/rocprofiler-sdk-tool: update generateCSV.{hpp,cpp}

- separate writing CSV from accumulating statistics
- a lot of functionality was moved to statistics.{hpp,cpp}

* lib/rocprofiler-sdk-tool: update output_file.{hpp,cpp}

- output_stream_t struct
- get_output_stream(...) returns output_stream_t instance

* lib/rocprofiler-sdk-tool: update generateJSON.cpp

- update get_output_stream usage to output_stream_t

* lib/rocprofiler-sdk-tool: update generateOTF2.cpp

- header include order tweak

* lib/rocprofiler-sdk-tool: update buffered_output.hpp

- stats_data_t was renamed to stats_entry_t

* lib/rocprofiler-sdk-tool: update generatePerfetto.cpp

- header include tweak

* lib/rocprofiler-sdk-tool: update tmp_file_buffer.hpp

- emit warning message if write_ring_buffer fails after offloading instead of aborting
- prefer placement new instead of assignment in write_ring_buffer

* lib/rocprofiler-sdk-tool: add generateStats.{hpp,cpp}

- functions for accumulating statistics

* Update tests/rocprofv3/tracing-hip-in-libraries/CMakeLists.txt

- accommodate tweak to CSV output file name for HIP and HSA traces

* lib/rocprofiler-sdk-tool: update config.{hpp,cpp}

- new config variables
  - stats_summary
  - stats_summary_per_domain
  - summary_output
  - stats_summary_unit_value
  - stats_summary_unit
  - stats_summary_file
  - stats_summary_groups
- support output keys for hostname: %hostname% / %h

* lib/rocprofiler-sdk-tool: update tool.cpp

- support summary output

* Documentation fixes

* Test for summary output

* Update tests/bin/transpose to use more ROCTx

- also support building with the roctracer ROCTx

* Remove roctxMark from OTF2 + fix kernel-rename tests

- following more ROCTx calls in transpose, kernel-rename validation had to be updated

* JSON metadata + JSON summary

- add serialization support for config
- add serialization support for statistics
- additions to json spec
  - rocprofiler-sdk-tool/metadata/config
  - rocprofiler-sdk-tool/metadata/command
  - rocprofiler-sdk-tool/summary
- config output_keys support for NVIDIA %q{<ENV-VAR>} syntax
- config output_keys support keys within keys

* rocprofv3 --summary-groups warning if no domain matches

- emit warning if a regex in for summary groups did not match any domain names

* Compile fix for lib/rocprofiler-sdk-tool/tool.cpp

- get_config().scratch_memory_trace
- pass contributions to write_json

* Update rocprofv3.py to preload rocprofiler-sdk-roctx

- appended to LD_PRELOAD when args.marker_trace is enabled

* Fix ReST link errors about subtitle underline being too short

* Patch tokenization of config::stats_summary_groups

- guard against array values of empty strings

* Tweak rocprofv3 summary test

- input-summary.yaml (used by rocprofv3-test-summary-inp-yaml-execute) only provides one summary group regex

* Disable LD_PRELOAD of librocprofiler-sdk-roctx.so

- this causes problems in the sanitizers, will be addressed in another PR
Tento commit je obsažen v:
Jonathan R. Madsen
2024-09-09 11:20:55 -05:00
odevzdal GitHub
rodič 1354452cf0
revize 395f01b689
40 změnil soubory, kde provedl 2293 přidání a 580 odebrání
Stáhněte soubor záplaty Stáhněte rozdílový soubor Rozbalit všechny soubory Sbalit všechny soubory
source/lib/rocprofiler-sdk-tool/CMakeLists.txt
+3
Zobrazit soubor
@@ -13,6 +13,7 @@ set(TOOL_HEADERS
generateJSON.hpp
generateOTF2.hpp
generatePerfetto.hpp
generateStats.hpp
helper.hpp
output_file.hpp
statistics.hpp
@@ -26,9 +27,11 @@ set(TOOL_SOURCES
generateJSON.cpp
generateOTF2.cpp
generatePerfetto.cpp
generateStats.cpp
helper.cpp
main.c
output_file.cpp
statistics.cpp
tmp_file_buffer.cpp
tmp_file.cpp
tool.cpp)
source/lib/rocprofiler-sdk-tool/buffered_output.hpp
+1 -1
Zobrazit soubor
@@ -59,7 +59,7 @@ struct buffered_output
operator bool() const { return enabled; }
std::deque<Tp> element_data = {};
stats_data_t stats = {};
stats_entry_t stats = {};
private:
bool enabled = false;
source/lib/rocprofiler-sdk-tool/config.cpp
+100 -7
Zobrazit soubor
@@ -28,12 +28,14 @@
#include "lib/common/environment.hpp"
#include "lib/common/filesystem.hpp"
#include "lib/common/logging.hpp"
#include "lib/common/units.hpp"
#include "lib/common/utility.hpp"
#include <rocprofiler-sdk/cxx/details/delimit.hpp>
#include <rocprofiler-sdk/cxx/details/tokenize.hpp>
#include <fmt/core.h>
#include <linux/limits.h>
#include <unistd.h>
#include <algorithm>
#include <chrono>
@@ -44,6 +46,7 @@
#include <set>
#include <sstream>
#include <string>
#include <string_view>
#include <vector>
namespace rocprofiler
@@ -67,8 +70,14 @@ const auto* launch_clock = as_pointer(std::chrono::system_clock::now());
const auto* launch_datetime =
get_local_datetime(get_env("ROCP_TIME_FORMAT", "%F_%H.%M"), launch_time);
const auto env_regexes =
new std::array<std::regex, 2>{std::regex{"(.*)%(env|ENV)\\{([A-Z0-9_]+)\\}%(.*)"},
std::regex{"(.*)\\$(env|ENV)\\{([A-Z0-9_]+)\\}(.*)"}};
new std::array<std::regex, 3>{std::regex{"(.*)%(env|ENV)\\{([A-Z0-9_]+)\\}%(.*)"},
std::regex{"(.*)\\$(env|ENV)\\{([A-Z0-9_]+)\\}(.*)"},
std::regex{"(.*)%q\\{([A-Z0-9_]+)\\}(.*)"}};
// env regex examples:
// - %env{USER}% Consistent with other output key formats (start+end with %)
// - $ENV{USER} Similar to CMake
// - %q{USER} Compatibility with NVIDIA
//
std::string*
get_local_datetime(const std::string& dt_format, std::time_t*& _dt_curr)
@@ -86,6 +95,22 @@ get_local_datetime(const std::string& dt_format, std::time_t*& _dt_curr)
return nullptr;
}
std::string
get_hostname()
{
auto _hostname_buff = std::array<char, PATH_MAX>{};
_hostname_buff.fill('\0');
if(gethostname(_hostname_buff.data(), _hostname_buff.size() - 1) != 0)
{
auto _err = errno;
ROCP_WARNING << "Hostname unknown. gethostname failed with error code " << _err << ": "
<< strerror(_err);
return std::string{"UNKNOWN_HOSTNAME"};
}
return std::string{_hostname_buff.data()};
}
inline bool
not_is_space(int ch)
{
@@ -279,6 +304,34 @@ config::config()
const auto supported_perfetto_backends = std::set<std::string_view>{"inprocess", "system"};
LOG_IF(FATAL, supported_perfetto_backends.count(perfetto_backend) == 0)
<< "Unsupported perfetto backend type: " << perfetto_backend;
if(stats_summary_unit == "sec")
stats_summary_unit_value = common::units::sec;
else if(stats_summary_unit == "msec")
stats_summary_unit_value = common::units::msec;
else if(stats_summary_unit == "usec")
stats_summary_unit_value = common::units::usec;
else if(stats_summary_unit == "nsec")
stats_summary_unit_value = common::units::nsec;
else
{
ROCP_FATAL << "Unsupported summary units value: " << stats_summary_unit;
}
if(auto _summary_grps = get_env("ROCPROF_STATS_SUMMARY_GROUPS", ""); !_summary_grps.empty())
{
stats_summary_groups =
sdk::parse::tokenize(_summary_grps, std::vector<std::string_view>{"##@@##"});
// remove any empty strings (just in case these slipped through)
stats_summary_groups.erase(std::remove_if(stats_summary_groups.begin(),
stats_summary_groups.end(),
[](const auto& itr) { return itr.empty(); }),
stats_summary_groups.end());
}
// enable summary output if any of these are enabled
summary_output = (stats_summary || stats_summary_per_domain || !stats_summary_groups.empty());
}
std::vector<output_key>
@@ -374,8 +427,10 @@ output_keys(std::string _tag)
}
auto _launch_time = (launch_datetime) ? *launch_datetime : std::string{".UNKNOWN_LAUNCH_TIME."};
auto _hostname = get_hostname();
for(auto&& itr : std::initializer_list<output_key>{
{"%hostname%", _hostname, "Network hostname"},
{"%pid%", _proc_id, "Process identifier"},
{"%ppid%", _parent_id, "Parent process identifier"},
{"%pgid%", _pgroup_id, "Process group identifier"},
@@ -392,6 +447,7 @@ output_keys(std::string _tag)
}
for(auto&& itr : std::initializer_list<output_key>{
{"%h", _hostname, "Shorthand for %hostname%"},
{"%p", _proc_id, "Shorthand for %pid%"},
{"%j", _slurm_job_id, "Shorthand for %job%"},
{"%r", _slurm_proc_id, "Shorthand for %rank%"},
@@ -404,8 +460,10 @@ output_keys(std::string _tag)
return _options;
}
namespace
{
std::string
format(std::string _fpath, const std::string& _tag)
format_impl(std::string _fpath, const std::vector<output_key>& _keys)
{
if(_fpath.find('%') == std::string::npos && _fpath.find('$') == std::string::npos)
return _fpath;
@@ -416,20 +474,36 @@ format(std::string _fpath, const std::string& _tag)
_v.replace(pos, pitr.key.length(), pitr.value);
};
for(auto&& itr : output_keys(_tag))
for(auto&& itr : _keys)
_replace(_fpath, itr);
// environment and configuration variables
try
{
auto strip_leading_and_replace =
[](std::string_view inp_v, std::initializer_list<char> keys, const char* val) {
auto inp = std::string{inp_v};
for(auto key : keys)
{
auto pos = std::string::npos;
while((pos = inp.find(key)) == 0)
inp = inp.substr(pos + 1);
while((pos = inp.find(key)) != std::string::npos)
inp = inp.replace(pos, 1, val);
}
return inp;
};
for(const auto& _re : *env_regexes)
{
while(std::regex_search(_fpath, _re))
{
auto _var = std::regex_replace(_fpath, _re, "$3");
std::string _val = get_env<std::string>(_var, "");
auto _beg = std::regex_replace(_fpath, _re, "$1");
auto _end = std::regex_replace(_fpath, _re, "$4");
_val = strip_leading_and_replace(_val, {'\t', ' ', '/'}, "_");
auto _beg = std::regex_replace(_fpath, _re, "$1");
auto _end = std::regex_replace(_fpath, _re, "$4");
_fpath = fmt::format("{}{}{}", _beg, _val, _end);
}
}
@@ -454,6 +528,25 @@ format(std::string _fpath, const std::string& _tag)
return _fpath;
}
std::string
format(std::string _fpath, const std::vector<output_key>& _keys)
{
if(_fpath.find('%') == std::string::npos && _fpath.find('$') == std::string::npos)
return _fpath;
auto _ref = _fpath;
_fpath = format_impl(std::move(_fpath), _keys);
return (_fpath == _ref) ? _fpath : format(std::move(_fpath), _keys);
}
} // namespace
std::string
format(std::string _fpath, const std::string& _tag)
{
return format(std::move(_fpath), output_keys(_tag));
}
std::string
format_name(std::string_view _name, const config& _cfg)
{
source/lib/rocprofiler-sdk-tool/config.hpp
+104 -16
Zobrazit soubor
@@ -25,8 +25,14 @@
#include "lib/common/environment.hpp"
#include "lib/common/filesystem.hpp"
#include "lib/common/mpl.hpp"
#include <rocprofiler-sdk/cxx/serialization.hpp>
#include <fmt/format.h>
#include <set>
#include <sstream>
#include <string>
#include <unordered_set>
#include <vector>
@@ -38,6 +44,8 @@ namespace tool
namespace fs = common::filesystem;
using common::get_env;
struct config;
enum class config_context
{
global = 0,
@@ -48,6 +56,19 @@ enum class config_context
perfetto_plugin,
};
void
initialize();
template <config_context ContextT = config_context::global>
config&
get_config();
std::string
format_name(std::string_view _name, const config& = get_config<>());
std::string
format(std::string _fpath, const std::string& _tag = {});
int
get_mpi_size();
@@ -67,25 +88,32 @@ struct config
bool hsa_finalizer_ext_api_trace = get_env("ROCPROF_HSA_FINALIZER_EXT_API_TRACE", false);
bool marker_api_trace = get_env("ROCPROF_MARKER_API_TRACE", false);
bool memory_copy_trace = get_env("ROCPROF_MEMORY_COPY_TRACE", false);
bool scratch_memory = get_env("ROCPROF_SCRATCH_MEMORY_TRACE", false);
bool scratch_memory_trace = get_env("ROCPROF_SCRATCH_MEMORY_TRACE", false);
bool counter_collection = get_env("ROCPROF_COUNTER_COLLECTION", false);
bool hip_runtime_api_trace = get_env("ROCPROF_HIP_RUNTIME_API_TRACE", false);
bool hip_compiler_api_trace = get_env("ROCPROF_HIP_COMPILER_API_TRACE", false);
bool list_metrics = get_env("ROCPROF_LIST_METRICS", false);
bool list_metrics_output_file = get_env("ROCPROF_OUTPUT_LIST_METRICS_FILE", false);
bool stats = get_env("ROCPROF_STATS", false);
bool stats_summary = get_env("ROCPROF_STATS_SUMMARY", false);
bool stats_summary_per_domain = get_env("ROCPROF_STATS_SUMMARY_PER_DOMAIN", false);
bool csv_output = false;
bool json_output = false;
bool pftrace_output = false;
bool otf2_output = false;
bool summary_output = false;
bool kernel_rename = get_env("ROCPROF_KERNEL_RENAME", false);
int mpi_size = get_mpi_size();
int mpi_rank = get_mpi_rank();
size_t perfetto_shmem_size_hint = get_env("ROCPROF_PERFETTO_SHMEM_SIZE_HINT_KB", 64);
size_t perfetto_buffer_size = get_env("ROCPROF_PERFETTO_BUFFER_SIZE_KB", 1024000);
std::string output_path = get_env("ROCPROF_OUTPUT_PATH", fs::current_path().string());
std::string output_file = get_env("ROCPROF_OUTPUT_FILE_NAME", std::to_string(getpid()));
std::string tmp_directory = get_env("ROCPROF_TMPDIR", output_path);
uint64_t stats_summary_unit_value = 1;
std::string stats_summary_unit = get_env("ROCPROF_STATS_SUMMARY_UNITS", "nsec");
std::string output_path = get_env("ROCPROF_OUTPUT_PATH", fs::current_path().string());
std::string output_file =
get_env("ROCPROF_OUTPUT_FILE_NAME", fmt::format("%hostname%/{}", getpid()));
std::string tmp_directory = get_env("ROCPROF_TMPDIR", output_path);
std::string stats_summary_file = get_env("ROCPROF_STATS_SUMMARY_OUTPUT", "stderr");
std::string kernel_filter_include =
get_env("ROCPROF_KERNEL_FILTER_INCLUDE_REGEX", std::string{".*"});
@@ -94,11 +122,68 @@ struct config
std::string perfetto_buffer_fill_policy =
get_env("ROCPROF_PERFETTO_BUFFER_FILL_POLICY", std::string{"discard"});
std::string perfetto_backend = get_env("ROCPROF_PERFETTO_BACKEND", std::string{"inprocess"});
std::unordered_set<uint32_t> kernel_filter_range = {};
std::set<std::string> counters = {};
std::unordered_set<uint32_t> kernel_filter_range = {};
std::set<std::string> counters = {};
std::vector<std::string> stats_summary_groups = {};
template <typename ArchiveT>
void save(ArchiveT&) const;
template <typename ArchiveT>
void load(ArchiveT&)
{}
};
template <config_context ContextT = config_context::global>
template <typename ArchiveT>
void
config::save(ArchiveT& ar) const
{
#define CFG_SERIALIZE_MEMBER(VAR) ar(cereal::make_nvp(#VAR, VAR))
#define CFG_SERIALIZE_NAMED_MEMBER(NAME, VAR) ar(cereal::make_nvp(NAME, VAR))
CFG_SERIALIZE_MEMBER(demangle);
CFG_SERIALIZE_MEMBER(truncate);
CFG_SERIALIZE_MEMBER(kernel_trace);
CFG_SERIALIZE_MEMBER(hsa_core_api_trace);
CFG_SERIALIZE_MEMBER(hsa_amd_ext_api_trace);
CFG_SERIALIZE_MEMBER(hsa_image_ext_api_trace);
CFG_SERIALIZE_MEMBER(hsa_finalizer_ext_api_trace);
CFG_SERIALIZE_MEMBER(marker_api_trace);
CFG_SERIALIZE_MEMBER(memory_copy_trace);
CFG_SERIALIZE_MEMBER(scratch_memory_trace);
CFG_SERIALIZE_MEMBER(counter_collection);
CFG_SERIALIZE_MEMBER(hip_runtime_api_trace);
CFG_SERIALIZE_MEMBER(hip_compiler_api_trace);
CFG_SERIALIZE_MEMBER(kernel_rename);
CFG_SERIALIZE_NAMED_MEMBER("summary", stats_summary);
CFG_SERIALIZE_NAMED_MEMBER("summary_per_domain", stats_summary_per_domain);
CFG_SERIALIZE_NAMED_MEMBER("summary_groups", stats_summary_groups);
CFG_SERIALIZE_NAMED_MEMBER("summary_unit", stats_summary_unit);
CFG_SERIALIZE_NAMED_MEMBER("summary_file", stats_summary_file);
CFG_SERIALIZE_MEMBER(perfetto_shmem_size_hint);
CFG_SERIALIZE_MEMBER(perfetto_buffer_size);
CFG_SERIALIZE_MEMBER(perfetto_buffer_fill_policy);
CFG_SERIALIZE_MEMBER(perfetto_backend);
CFG_SERIALIZE_NAMED_MEMBER("raw_tmp_directory", tmp_directory);
CFG_SERIALIZE_NAMED_MEMBER("raw_output_path", output_path);
CFG_SERIALIZE_NAMED_MEMBER("raw_output_file", output_file);
CFG_SERIALIZE_NAMED_MEMBER("tmp_directory", format(tmp_directory));
CFG_SERIALIZE_NAMED_MEMBER("output_path", format(output_path));
CFG_SERIALIZE_NAMED_MEMBER("output_file", format(output_file));
CFG_SERIALIZE_MEMBER(counters);
CFG_SERIALIZE_MEMBER(kernel_filter_include);
CFG_SERIALIZE_MEMBER(kernel_filter_exclude);
CFG_SERIALIZE_MEMBER(kernel_filter_range);
#undef CFG_SERIALIZE_MEMBER
#undef CFG_SERIALIZE_NAMED_MEMBER
}
template <config_context ContextT>
config&
get_config()
{
@@ -119,6 +204,11 @@ struct output_key
{
output_key(std::string _key, std::string _val, std::string _desc = {});
template <typename Tp,
typename Up = Tp,
std::enable_if_t<!common::mpl::is_string_type<Up>::value, int> = 0>
output_key(std::string _key, Tp&& _val, std::string _desc = {});
operator std::pair<std::string, std::string>() const;
std::string key = {};
@@ -126,16 +216,14 @@ struct output_key
std::string description = {};
};
template <typename Tp, typename Up, std::enable_if_t<!common::mpl::is_string_type<Up>::value, int>>
output_key::output_key(std::string _key, Tp&& _val, std::string _desc)
: key{std::move(_key)}
, value{fmt::format("{}", std::forward<Tp>(_val))}
, description{std::move(_desc)}
{}
std::vector<output_key>
output_keys(std::string _tag = {});
std::string
format(std::string _fpath, const std::string& _tag = {});
std::string
format_name(std::string_view _name, const config& = get_config<>());
void
initialize();
} // namespace tool
} // namespace rocprofiler
source/lib/rocprofiler-sdk-tool/domain_type.cpp
+5 -5
Zobrazit soubor
@@ -37,13 +37,13 @@ struct domain_type_name;
static constexpr auto filename = FILENAME; \
};
DEFINE_BUFFER_TYPE_NAME(HSA, "HSA_API", "hsa_trace")
DEFINE_BUFFER_TYPE_NAME(HIP, "HIP_API", "hip_trace")
DEFINE_BUFFER_TYPE_NAME(MEMORY_COPY, "MEMORY_COPY", "memory_copy")
DEFINE_BUFFER_TYPE_NAME(COUNTER_COLLECTION, "COUNTER_COLLECTION", "counter_collection")
DEFINE_BUFFER_TYPE_NAME(HSA, "HSA_API", "hsa_api")
DEFINE_BUFFER_TYPE_NAME(HIP, "HIP_API", "hip_api")
DEFINE_BUFFER_TYPE_NAME(MARKER, "MARKER_API", "marker_api")
DEFINE_BUFFER_TYPE_NAME(KERNEL_DISPATCH, "KERNEL_DISPATCH", "kernel_dispatch")
DEFINE_BUFFER_TYPE_NAME(MARKER, "MARKER", "marker_trace")
DEFINE_BUFFER_TYPE_NAME(MEMORY_COPY, "MEMORY_COPY", "memory_copy")
DEFINE_BUFFER_TYPE_NAME(SCRATCH_MEMORY, "SCRATCH_MEMORY", "scratch_memory")
DEFINE_BUFFER_TYPE_NAME(COUNTER_COLLECTION, "COUNTER_COLLECTION", "counter_collection")
#undef DEFINE_BUFFER_TYPE_NAME
source/lib/rocprofiler-sdk-tool/domain_type.hpp
+3 -3
Zobrazit soubor
@@ -28,11 +28,11 @@ enum class domain_type
{
HSA = 0,
HIP,
MEMORY_COPY,
COUNTER_COLLECTION,
KERNEL_DISPATCH,
MARKER,
KERNEL_DISPATCH,
MEMORY_COPY,
SCRATCH_MEMORY,
COUNTER_COLLECTION,
LAST,
};
source/lib/rocprofiler-sdk-tool/generateCSV.cpp
+77 -154
Zobrazit soubor
@@ -21,9 +21,10 @@
// SOFTWARE.
#include "generateCSV.hpp"
#include "config.hpp"
#include "csv.hpp"
#include "generateStats.hpp"
#include "helper.hpp"
#include "lib/rocprofiler-sdk-tool/config.hpp"
#include "statistics.hpp"
#include <rocprofiler-sdk/fwd.h>
@@ -41,47 +42,6 @@ namespace tool
{
namespace
{
using stats_data_t = statistics<uint64_t, float_type>;
using stats_map_t = std::map<std::string_view, stats_data_t>;
struct percentage
{
float_type value = {};
friend std::ostream& operator<<(std::ostream& os, percentage val) { return (os << val.value); }
};
struct stats_formatter
{
template <typename Tp>
std::ostream& operator()(std::ostream& ofs, const Tp& _val) const
{
using value_type = common::mpl::unqualified_type_t<Tp>;
if constexpr(std::is_floating_point<value_type>::value)
{
constexpr value_type one_hundredth = 1.0e-2;
if(_val > one_hundredth)
ofs << std::setprecision(6) << std::fixed;
else
ofs << std::setprecision(8) << std::scientific;
}
else if constexpr(std::is_same<Tp, percentage>::value)
{
constexpr float_type one = 1.0;
constexpr float_type one_hundredth = 1.0e-2;
if(_val.value >= one)
ofs << std::setprecision(2) << std::fixed;
else if(_val.value > one_hundredth)
ofs << std::setprecision(4) << std::fixed;
else
ofs << std::setprecision(3) << std::scientific;
}
return ofs;
}
};
tool::output_file
get_stats_output_file(std::string name)
{
@@ -99,10 +59,10 @@ get_stats_output_file(std::string name)
}};
}
stats_data_t
write_stats(output_file&& ofs, const stats_map_t& data_v)
void
write_stats(output_file&& ofs, const stats_entry_vec_t& data_v)
{
auto data = std::vector<std::pair<std::string_view, stats_data_t>>{};
auto data = stats_entry_vec_t{};
auto _duration = stats_data_t{};
for(const auto& [id, value] : data_v)
{
@@ -136,8 +96,6 @@ write_stats(output_file&& ofs, const stats_map_t& data_v)
value.get_stddev());
ofs << _row.str() << std::flush;
}
return _duration;
}
} // namespace
@@ -275,14 +233,17 @@ generate_csv(tool_table* /*tool_functions*/, std::vector<rocprofiler_agent_v0_t>
}
}
stats_data_t
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_kernel_dispatch_record_t>& data)
const std::deque<rocprofiler_buffer_tracing_kernel_dispatch_record_t>& data,
const stats_entry_t& stats)
{
if(data.empty()) return stats_data_t{};
if(data.empty()) return;
auto kernel_stats = stats_map_t{};
auto ofs = tool::output_file{"kernel_trace",
if(tool::get_config().stats && stats)
write_stats(get_stats_output_file("kernel_stats"), stats.entries);
auto ofs = tool::output_file{"kernel_trace",
tool::csv::kernel_trace_csv_encoder{},
{"Kind",
"Agent_Id",
@@ -329,27 +290,21 @@ generate_csv(tool_table*
record.dispatch_info.grid_size.y,
record.dispatch_info.grid_size.z);
if(tool::get_config().stats)
kernel_stats[kernel_name] += (record.end_timestamp - record.start_timestamp);
ofs << row_ss.str();
}
auto _duration = stats_data_t{};
if(tool::get_config().stats)
_duration = write_stats(get_stats_output_file("kernel_stats"), kernel_stats);
return _duration;
}
stats_data_t
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_hip_api_record_t>& data)
const std::deque<rocprofiler_buffer_tracing_hip_api_record_t>& data,
const stats_entry_t& stats)
{
if(data.empty()) return stats_data_t{};
if(data.empty()) return;
auto hip_stats = stats_map_t{};
auto ofs = tool::output_file{"hip_api_trace",
if(tool::get_config().stats && stats)
write_stats(get_stats_output_file("hip_api_stats"), stats.entries);
auto ofs = tool::output_file{"hip_api_trace",
tool::csv::api_csv_encoder{},
{"Domain",
"Function",
@@ -372,28 +327,21 @@ generate_csv(tool_table* tool
record.start_timestamp,
record.end_timestamp);
if(tool::get_config().stats)
hip_stats[api_name] += (record.end_timestamp - record.start_timestamp);
ofs << row_ss.str();
}
auto _duration = stats_data_t{};
if(tool::get_config().stats)
{
_duration = write_stats(get_stats_output_file("hip_stats"), hip_stats);
}
return _duration;
}
stats_data_t
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_hsa_api_record_t>& data)
const std::deque<rocprofiler_buffer_tracing_hsa_api_record_t>& data,
const stats_entry_t& stats)
{
if(data.empty()) return stats_data_t{};
if(data.empty()) return;
auto hsa_stats = stats_map_t{};
auto ofs = tool::output_file{"hsa_api_trace",
if(tool::get_config().stats && stats)
write_stats(get_stats_output_file("hsa_api_stats"), stats.entries);
auto ofs = tool::output_file{"hsa_api_trace",
tool::csv::api_csv_encoder{},
{"Domain",
"Function",
@@ -417,28 +365,21 @@ generate_csv(tool_table* tool
record.start_timestamp,
record.end_timestamp);
if(tool::get_config().stats)
hsa_stats[api_name] += (record.end_timestamp - record.start_timestamp);
ofs << row_ss.str();
}
auto _duration = stats_data_t{};
if(tool::get_config().stats)
{
_duration = write_stats(get_stats_output_file("hsa_stats"), hsa_stats);
}
return _duration;
}
stats_data_t
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_memory_copy_record_t>& data)
const std::deque<rocprofiler_buffer_tracing_memory_copy_record_t>& data,
const stats_entry_t& stats)
{
if(data.empty()) return stats_data_t{};
if(data.empty()) return;
auto memory_copy_stats = stats_map_t{};
auto ofs = tool::output_file{"memory_copy_trace",
if(tool::get_config().stats && stats)
write_stats(get_stats_output_file("memory_copy_stats"), stats.entries);
auto ofs = tool::output_file{"memory_copy_trace",
tool::csv::memory_copy_csv_encoder{},
{"Kind",
"Direction",
@@ -461,28 +402,21 @@ generate_csv(tool_table*
record.start_timestamp,
record.end_timestamp);
if(tool::get_config().stats)
memory_copy_stats[api_name] += (record.end_timestamp - record.start_timestamp);
ofs << row_ss.str();
}
auto _duration = stats_data_t{};
if(tool::get_config().stats)
{
_duration = write_stats(get_stats_output_file("memory_copy_stats"), memory_copy_stats);
}
return _duration;
}
stats_data_t
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_marker_api_record_t>& data)
const std::deque<rocprofiler_buffer_tracing_marker_api_record_t>& data,
const stats_entry_t& stats)
{
if(data.empty()) return stats_data_t{};
if(data.empty()) return;
auto marker_stats = stats_map_t{};
auto ofs = tool::output_file{"marker_api_trace",
if(tool::get_config().stats && stats)
write_stats(get_stats_output_file("marker_api_stats"), stats.entries);
auto ofs = tool::output_file{"marker_api_trace",
tool::csv::marker_csv_encoder{},
{"Domain",
"Function",
@@ -518,25 +452,19 @@ generate_csv(tool_table* t
record.start_timestamp,
record.end_timestamp);
if(tool::get_config().stats)
marker_stats[_name] += (record.end_timestamp - record.start_timestamp);
ofs << row_ss.str();
}
auto _duration = stats_data_t{};
if(tool::get_config().stats)
{
_duration = write_stats(get_stats_output_file("marker_stats"), marker_stats);
}
return _duration;
}
stats_data_t
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_tool_counter_collection_record_t>& data)
const std::deque<rocprofiler_tool_counter_collection_record_t>& data,
const stats_entry_t& stats)
{
if(data.empty()) return stats_data_t{};
if(data.empty()) return;
if(tool::get_config().stats && stats)
write_stats(get_stats_output_file("counter_collection_stats"), stats.entries);
auto ofs = tool::output_file{"counter_collection",
tool::csv::counter_collection_csv_encoder{},
@@ -601,14 +529,17 @@ generate_csv(tool_table* too
}
ofs << row_ss.str();
}
return stats_data_t{};
}
stats_data_t
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_scratch_memory_record_t>& data)
const std::deque<rocprofiler_buffer_tracing_scratch_memory_record_t>& data,
const stats_entry_t& stats)
{
if(data.empty()) return stats_data_t{};
if(data.empty()) return;
if(tool::get_config().stats && stats)
write_stats(get_stats_output_file("scratch_memory_stats"), stats.entries);
auto ofs = tool::output_file{"scratch_memory_trace",
tool::csv::scratch_memory_encoder{},
@@ -623,7 +554,6 @@ generate_csv(tool_table*
"End_Timestamp",
}};
auto scratch_memory_stats = stats_map_t{};
for(const auto& record : data)
{
auto row_ss = std::stringstream{};
@@ -641,45 +571,38 @@ generate_csv(tool_table*
record.start_timestamp,
record.end_timestamp);
if(tool::get_config().stats)
scratch_memory_stats[op_name] += (record.end_timestamp - record.start_timestamp);
ofs << row_ss.str();
}
auto _duration = stats_data_t{};
if(tool::get_config().stats)
{
_duration =
write_stats(get_stats_output_file("scratch_memory_stats"), scratch_memory_stats);
}
return _duration;
}
void
generate_csv(tool_table* /*tool_functions*/, std::unordered_map<domain_type, stats_data_t>& data)
generate_csv(tool_table* /*tool_functions*/, const domain_stats_vec_t& data_v)
{
using csv_encoder_t = rocprofiler::tool::csv::stats_csv_encoder;
if(!tool::get_config().stats) return;
auto _data = data_v;
auto _total_stats = stats_data_t{};
for(const auto& itr : data)
_total_stats += itr.second;
for(const auto& itr : _data)
_total_stats += itr.second.total;
if(_total_stats.get_count() == 0) return;
std::sort(_data.begin(), _data.end(), [](const auto& lhs, const auto& rhs) {
return (lhs.second.total.get_sum() > rhs.second.total.get_sum());
});
auto ofs = get_stats_output_file("domain_stats");
constexpr float_type one_hundred = 100.0;
const float_type _total_duration = _total_stats.get_sum();
for(const auto& [type, value] : data)
const float_type _total_duration = _total_stats.get_sum();
for(const auto& [type, value] : _data)
{
auto name = get_domain_column_name(type);
auto duration_ns = value.get_sum();
auto calls = value.get_count();
float_type avg_ns = value.get_mean();
float_type percent_v = (duration_ns / _total_duration) * one_hundred;
auto name = get_domain_column_name(type);
auto duration_ns = value.total.get_sum();
auto calls = value.total.get_count();
auto avg_ns = value.total.get_mean();
auto percent_v = value.total.get_percent(_total_duration);
auto _row = std::stringstream{};
csv_encoder_t::write_row<stats_formatter>(_row,
@@ -688,9 +611,9 @@ generate_csv(tool_table* /*tool_functions*/, std::unordered_map<domain_type, sta
duration_ns,
avg_ns,
percentage{percent_v},
value.get_min(),
value.get_max(),
value.get_stddev());
value.total.get_min(),
value.total.get_max(),
value.total.get_stddev());
ofs << _row.str() << std::flush;
}
}
source/lib/rocprofiler-sdk-tool/generateCSV.hpp
+35 -30
Zobrazit soubor
@@ -22,6 +22,7 @@
#pragma once
#include "generateStats.hpp"
#include "helper.hpp"
#include "statistics.hpp"
@@ -31,41 +32,45 @@ namespace rocprofiler
{
namespace tool
{
using float_type = double;
using stats_data_t = statistics<uint64_t, float_type>;
void
generate_csv(tool_table* tool_functions, std::vector<rocprofiler_agent_v0_t>& data);
stats_data_t
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_kernel_dispatch_record_t>& data);
stats_data_t
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_hip_api_record_t>& data);
stats_data_t
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_hsa_api_record_t>& data);
stats_data_t
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_memory_copy_record_t>& data);
stats_data_t
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_marker_api_record_t>& data);
stats_data_t
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_tool_counter_collection_record_t>& data);
stats_data_t
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_scratch_memory_record_t>& data);
const std::deque<rocprofiler_buffer_tracing_kernel_dispatch_record_t>& data,
const stats_entry_t& stats);
void
generate_csv(tool_table* tool_functions, std::unordered_map<domain_type, stats_data_t>& data);
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_hip_api_record_t>& data,
const stats_entry_t& stats);
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_hsa_api_record_t>& data,
const stats_entry_t& stats);
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_memory_copy_record_t>& data,
const stats_entry_t& stats);
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_marker_api_record_t>& data,
const stats_entry_t& stats);
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_tool_counter_collection_record_t>& data,
const stats_entry_t& stats);
void
generate_csv(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_scratch_memory_record_t>& data,
const stats_entry_t& stats);
void
generate_csv(tool_table* tool_functions, const domain_stats_vec_t& data);
} // namespace tool
} // namespace rocprofiler
source/lib/rocprofiler-sdk-tool/generateJSON.cpp
+33 -8
Zobrazit soubor
@@ -24,8 +24,10 @@
#include "config.hpp"
#include "helper.hpp"
#include "output_file.hpp"
#include "statistics.hpp"
#include "lib/common/string_entry.hpp"
#include "lib/common/utility.hpp"
#include <rocprofiler-sdk/fwd.h>
#include <rocprofiler-sdk/marker/api_id.h>
@@ -39,6 +41,7 @@ namespace tool
void
write_json(tool_table* tool_functions,
uint64_t pid,
const domain_stats_vec_t& domain_stats,
std::vector<rocprofiler_agent_v0_t> agent_data,
std::vector<rocprofiler_tool_counter_info_t> counter_data,
std::deque<rocprofiler_buffer_tracing_hip_api_record_t>* hip_api_deque,
@@ -52,14 +55,14 @@ write_json(tool_table* tool
{
using JSONOutputArchive = cereal::MinimalJSONOutputArchive;
constexpr auto json_prec = 32;
constexpr auto json_indent = JSONOutputArchive::Options::IndentChar::space;
auto json_opts = JSONOutputArchive::Options{json_prec, json_indent, 1};
auto filename = std::string_view{"results"};
auto [output_stream, cleanup] = get_output_stream(filename, ".json");
constexpr auto json_prec = 32;
constexpr auto json_indent = JSONOutputArchive::Options::IndentChar::space;
auto json_opts = JSONOutputArchive::Options{json_prec, json_indent, 1};
auto filename = std::string_view{"results"};
auto ofs = get_output_stream(filename, ".json");
{
auto json_ar = JSONOutputArchive{*output_stream, json_opts};
auto json_ar = JSONOutputArchive{*ofs.stream, json_opts};
json_ar.setNextName("rocprofiler-sdk-tool");
json_ar.startNode();
@@ -74,6 +77,29 @@ write_json(tool_table* tool
json_ar(cereal::make_nvp("pid", pid));
json_ar(cereal::make_nvp("init_time", timestamps->app_start_time));
json_ar(cereal::make_nvp("fini_time", timestamps->app_end_time));
json_ar(cereal::make_nvp("config", get_config()));
json_ar(cereal::make_nvp("command", common::read_command_line(getpid())));
json_ar.finishNode();
}
// summary
{
json_ar.setNextName("summary");
json_ar.startNode();
json_ar.makeArray();
for(const auto& itr : domain_stats)
{
auto _name = get_domain_column_name(itr.first);
json_ar.startNode();
json_ar(cereal::make_nvp("domain", std::string{_name}));
json_ar(cereal::make_nvp("stats", itr.second));
// itr.second.serialize(json_ar, 0);
json_ar.finishNode();
}
json_ar.finishNode();
}
@@ -154,8 +180,7 @@ write_json(tool_table* tool
json_ar.finishNode();
}
*output_stream << std::flush;
if(cleanup) cleanup(output_stream);
ofs.close();
}
} // namespace tool
source/lib/rocprofiler-sdk-tool/generateJSON.hpp
+2
Zobrazit soubor
@@ -23,6 +23,7 @@
#pragma once
#include "helper.hpp"
#include "statistics.hpp"
namespace rocprofiler
{
@@ -31,6 +32,7 @@ namespace tool
void
write_json(tool_table* tool_functions,
uint64_t pid,
const domain_stats_vec_t& domain_stats,
std::vector<rocprofiler_agent_v0_t> agent_data,
std::vector<rocprofiler_tool_counter_info_t> counter_data,
std::deque<rocprofiler_buffer_tracing_hip_api_record_t>* hip_api_deque,
source/lib/rocprofiler-sdk-tool/generateOTF2.cpp
+7 -2
Zobrazit soubor
@@ -21,13 +21,14 @@
// SOFTWARE.
#include "generateOTF2.hpp"
#include "config.hpp"
#include "helper.hpp"
#include "output_file.hpp"
#include "lib/common/filesystem.hpp"
#include "lib/common/mpl.hpp"
#include "lib/common/units.hpp"
#include "lib/common/utility.hpp"
#include "lib/rocprofiler-sdk-tool/config.hpp"
#include "output_file.hpp"
#include <rocprofiler-sdk/fwd.h>
#include <rocprofiler-sdk/marker/api_id.h>
@@ -510,6 +511,10 @@ write_otf2(tool_table* tool
if(!_inp) return;
for(auto itr : *_inp)
{
if(itr.kind == ROCPROFILER_BUFFER_TRACING_MARKER_CORE_API &&
itr.operation == ROCPROFILER_MARKER_CORE_API_ID_roctxMarkA)
continue;
using value_type = common::mpl::unqualified_type_t<decltype(itr)>;
auto name = buffer_names.at(itr.kind, itr.operation);
auto paradigm = OTF2_PARADIGM_HIP;
source/lib/rocprofiler-sdk-tool/generatePerfetto.cpp
+7 -10
Zobrazit soubor
@@ -21,11 +21,11 @@
// SOFTWARE.
#include "generatePerfetto.hpp"
#include "config.hpp"
#include "helper.hpp"
#include "output_file.hpp"
#include "lib/common/utility.hpp"
#include "lib/rocprofiler-sdk-tool/config.hpp"
#include <rocprofiler-sdk/fwd.h>
#include <rocprofiler-sdk/marker/api_id.h>
@@ -503,23 +503,20 @@ write_perfetto(
tracing_session->FlushBlocking();
tracing_session->StopBlocking();
auto filename = std::string{"results"};
auto cleanup = std::function<void(std::ostream*&)>{};
std::ostream* ofs = nullptr;
std::tie(ofs, cleanup) = get_output_stream(filename, ".pftrace");
auto filename = std::string{"results"};
auto ofs = get_output_stream(filename, ".pftrace");
auto amount_read = std::atomic<size_t>{0};
auto is_done = std::promise<void>{};
auto _mtx = std::mutex{};
auto _reader = [ofs, &_mtx, &is_done, &amount_read](
auto _reader = [&ofs, &_mtx, &is_done, &amount_read](
::perfetto::TracingSession::ReadTraceCallbackArgs _args) {
auto _lk = std::unique_lock<std::mutex>{_mtx};
if(_args.data && _args.size > 0)
{
ROCP_TRACE << "Writing " << _args.size << " B to trace...";
// Write the trace data into file
ofs->write(_args.data, _args.size);
ofs.stream->write(_args.data, _args.size);
amount_read += _args.size;
}
ROCP_INFO_IF(!_args.has_more && amount_read > 0)
@@ -540,10 +537,10 @@ write_perfetto(
tracing_session.reset();
ROCP_TRACE << "Flushing trace output stream...";
(*ofs) << std::flush;
(*ofs.stream) << std::flush;
ROCP_TRACE << "Destroying trace output stream...";
if(cleanup) cleanup(ofs);
ofs.close();
}
} // namespace tool
source/lib/rocprofiler-sdk-tool/generateStats.cpp
+350
Zobrazit soubor
@@ -0,0 +1,350 @@
// MIT License
//
// Copyright (c) 2023 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#include "generateStats.hpp"
#include "config.hpp"
#include "helper.hpp"
#include "lib/common/logging.hpp"
#include "lib/rocprofiler-sdk-tool/domain_type.hpp"
#include "output_file.hpp"
#include "statistics.hpp"
#include <fmt/format.h>
#include <rocprofiler-sdk/fwd.h>
#include <rocprofiler-sdk/marker/api_id.h>
#include <unistd.h>
#include <cstdint>
#include <iomanip>
#include <regex>
#include <sstream>
#include <string_view>
#include <utility>
namespace rocprofiler
{
namespace tool
{
namespace
{
stats_entry_t
get_stats(const stats_map_t& data_v)
{
auto _stats = stats_entry_t{};
for(const auto& [id, value] : data_v)
{
_stats.entries.emplace_back(id, value);
_stats.total += value;
}
return _stats.sort();
}
} // namespace
stats_entry_t
generate_stats(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_kernel_dispatch_record_t>& data)
{
auto kernel_stats = stats_map_t{};
for(const auto& record : data)
{
auto kernel_name = tool_functions->tool_get_kernel_name_fn(
record.dispatch_info.kernel_id, record.correlation_id.external.value);
kernel_stats[kernel_name] += (record.end_timestamp - record.start_timestamp);
}
return get_stats(kernel_stats);
}
stats_entry_t
generate_stats(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_hip_api_record_t>& data)
{
auto hip_stats = stats_map_t{};
for(const auto& record : data)
{
auto api_name = tool_functions->tool_get_operation_name_fn(record.kind, record.operation);
hip_stats[api_name] += (record.end_timestamp - record.start_timestamp);
}
return get_stats(hip_stats);
}
stats_entry_t
generate_stats(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_hsa_api_record_t>& data)
{
auto hsa_stats = stats_map_t{};
for(const auto& record : data)
{
auto api_name = tool_functions->tool_get_operation_name_fn(record.kind, record.operation);
hsa_stats[api_name] += (record.end_timestamp - record.start_timestamp);
}
return get_stats(hsa_stats);
}
stats_entry_t
generate_stats(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_memory_copy_record_t>& data)
{
auto memory_copy_stats = stats_map_t{};
for(const auto& record : data)
{
auto api_name = tool_functions->tool_get_operation_name_fn(record.kind, record.operation);
memory_copy_stats[api_name] += (record.end_timestamp - record.start_timestamp);
}
return get_stats(memory_copy_stats);
}
stats_entry_t
generate_stats(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_marker_api_record_t>& data)
{
auto marker_stats = stats_map_t{};
for(const auto& record : data)
{
auto _name = std::string_view{};
if(record.kind == ROCPROFILER_BUFFER_TRACING_MARKER_CORE_API &&
(record.operation == ROCPROFILER_MARKER_CORE_API_ID_roctxMarkA ||
record.operation == ROCPROFILER_MARKER_CORE_API_ID_roctxRangePushA ||
record.operation == ROCPROFILER_MARKER_CORE_API_ID_roctxRangeStartA))
{
_name = tool_functions->tool_get_roctx_msg_fn(record.correlation_id.internal);
}
else
{
_name = tool_functions->tool_get_operation_name_fn(record.kind, record.operation);
}
marker_stats[_name] += (record.end_timestamp - record.start_timestamp);
}
return get_stats(marker_stats);
}
stats_entry_t
generate_stats(tool_table* /*tool_functions*/,
const std::deque<rocprofiler_tool_counter_collection_record_t>& /*data*/)
{
return stats_entry_t{};
}
stats_entry_t
generate_stats(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_scratch_memory_record_t>& data)
{
auto scratch_memory_stats = stats_map_t{};
for(const auto& record : data)
{
auto op_name = tool_functions->tool_get_operation_name_fn(record.kind, record.operation);
scratch_memory_stats[op_name] += (record.end_timestamp - record.start_timestamp);
}
return get_stats(scratch_memory_stats);
}
namespace
{
void
generate_stats(output_stream_t& os,
std::string_view label,
const domain_stats_vec_t& data_v,
std::string_view indent_v)
{
auto _data = stats_entry_t{};
auto _cols = std::unordered_map<std::string_view, domain_type>{};
auto _get_entry = [&_data, &_cols](domain_type _domain,
std::string_view _key) -> stats_data_t* {
for(auto& itr : _data.entries)
{
if(itr.first == _key) return &itr.second;
}
_cols.emplace(_key, _domain);
auto& itr = _data.entries.emplace_back(_key, stats_data_t{});
return &itr.second;
};
uint64_t name_width = 40;
uint64_t domain_width = 12;
for(const auto& itr : data_v)
{
for(const auto& eitr : itr.second.entries)
{
_data.total += eitr.second;
auto* _entry = _get_entry(itr.first, eitr.first);
*CHECK_NOTNULL(_entry) += eitr.second;
name_width = std::max(name_width, eitr.first.length());
}
domain_width = std::max(domain_width, get_domain_column_name(itr.first).length());
}
if(!_data) return;
std::sort(_data.entries.begin(), _data.entries.end(), [](const auto& lhs, const auto& rhs) {
return (lhs.second.get_sum() > rhs.second.get_sum());
});
const float_type _total_duration = _data.total.get_sum();
os << fmt::format("\n{}ROCPROFV3 {}:\n\n", indent_v, label) << std::flush;
{
auto _header = fmt::format(
"| {:^{}} | {:^{}} | {:^15} | {:^15} | {:^15} | {:^13} | {:^15} | {:^15} | {:^15} |",
"NAME",
name_width,
"DOMAIN",
domain_width,
"CALLS",
fmt::format("DURATION ({})", tool::get_config().stats_summary_unit),
fmt::format("AVERAGE ({})", tool::get_config().stats_summary_unit),
"PERCENT (INC)",
fmt::format("MIN ({})", tool::get_config().stats_summary_unit),
fmt::format("MAX ({})", tool::get_config().stats_summary_unit),
"STDDEV");
(*os.stream) << indent_v << _header << "\n" << std::flush;
auto _div =
fmt::format("|-{0:-^{1}}-|-{0:-^{2}}-|-{0:-^15}-|-{0:-^15}-|-{0:-^15}-|-{0:-^13}"
"-|-{0:-^15}-|-{0:-^15}-|-{0:-^15}-|",
"",
name_width,
domain_width);
(*os.stream) << indent_v << _div << "\n" << std::flush;
}
for(const auto& [type, value] : _data.entries)
{
auto name = type;
auto duration_ns = value.get_sum();
auto calls = value.get_count();
auto avg_ns = value.get_mean();
auto percent_v = value.get_percent(_total_duration);
auto percent = std::to_string(percent_v);
auto _row = std::string{};
if(tool::get_config().stats_summary_unit_value > 1)
{
auto _unit_div = static_cast<double>(tool::get_config().stats_summary_unit_value);
_row = fmt::format("{}| {:<{}} | {:<{}} | {:15} | {:15} | {:15.3e} | {:>13} | {:15} | "
"{:15} | {:15.3e} |",
indent_v,
name,
name_width,
get_domain_column_name(_cols.at(name)),
domain_width,
calls,
duration_ns / _unit_div,
avg_ns / _unit_div,
percent,
value.get_min() / _unit_div,
value.get_max() / _unit_div,
value.get_stddev() / _unit_div);
}
else
{
_row = fmt::format("{}| {:<{}} | {:<{}} | {:15} | {:15} | {:15.3e} | {:>13} | {:15} | "
"{:15} | {:15.3e} |",
indent_v,
name,
name_width,
get_domain_column_name(_cols.at(name)),
domain_width,
calls,
duration_ns,
avg_ns,
percent,
value.get_min(),
value.get_max(),
value.get_stddev());
}
(*os.stream) << _row << "\n" << std::flush;
}
(*os.stream) << "\n" << std::flush;
}
} // namespace
void
generate_stats(tool_table* /*tool_functions*/, const domain_stats_vec_t& inp_data)
{
auto data_v = inp_data;
std::sort(data_v.begin(), data_v.end(), [](const auto& lhs, const auto& rhs) {
return lhs.first < rhs.first;
});
output_stream_t _os = get_output_stream(tool::get_config().stats_summary_file, ".txt");
auto _indent = (_os.writes_to_file()) ? std::string_view{} : std::string_view{" "};
if(tool::get_config().stats_summary_per_domain)
{
for(const auto& itr : data_v)
{
if(!itr.second) continue;
auto _name = fmt::format("{} SUMMARY", get_domain_column_name(itr.first));
auto _tmp = domain_stats_vec_t{};
_tmp.emplace_back(itr.first, itr.second);
generate_stats(_os, _name, _tmp, _indent);
}
}
if(!tool::get_config().stats_summary_groups.empty())
{
auto domain_groups = std::vector<domain_stats_vec_t>{};
for(const auto& itr : tool::get_config().stats_summary_groups)
{
auto _names = std::vector<std::string>{};
auto _tmp = domain_stats_vec_t{};
for(const auto& ditr : data_v)
{
auto _col_name = get_domain_column_name(ditr.first);
if(std::regex_match(_col_name.data(), std::regex{itr}))
{
if(!ditr.second) continue;
_names.emplace_back(_col_name);
_tmp.emplace_back(ditr.first, ditr.second);
}
}
ROCP_CI_LOG_IF(WARNING, _names.empty())
<< "summary group regex '" << itr << "' matched with zero domain groups";
auto _name = fmt::format("{} SUMMARY", fmt::join(_names.begin(), _names.end(), " + "));
generate_stats(_os, _name, _tmp, _indent);
}
}
if(tool::get_config().stats_summary) generate_stats(_os, "SUMMARY", data_v, _indent);
}
} // namespace tool
} // namespace rocprofiler
source/lib/rocprofiler-sdk-tool/generateStats.hpp
+63
Zobrazit soubor
@@ -0,0 +1,63 @@
// MIT License
//
// Copyright (c) 2023 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#pragma once
#include "helper.hpp"
#include "statistics.hpp"
namespace rocprofiler
{
namespace tool
{
stats_entry_t
generate_stats(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_kernel_dispatch_record_t>& data);
stats_entry_t
generate_stats(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_hip_api_record_t>& data);
stats_entry_t
generate_stats(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_hsa_api_record_t>& data);
stats_entry_t
generate_stats(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_memory_copy_record_t>& data);
stats_entry_t
generate_stats(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_marker_api_record_t>& data);
stats_entry_t
generate_stats(tool_table* tool_functions,
const std::deque<rocprofiler_tool_counter_collection_record_t>& data);
stats_entry_t
generate_stats(tool_table* tool_functions,
const std::deque<rocprofiler_buffer_tracing_scratch_memory_record_t>& data);
void
generate_stats(tool_table* tool_functions, const domain_stats_vec_t& data);
} // namespace tool
} // namespace rocprofiler
source/lib/rocprofiler-sdk-tool/output_file.cpp
+13 -6
Zobrazit soubor
@@ -33,6 +33,12 @@ namespace tool
{
namespace fs = common::filesystem;
namespace
{
const auto stdout_names = std::unordered_set<std::string_view>{"stdout", "STDOUT"};
const auto stderr_names = std::unordered_set<std::string_view>{"stderr", "STDERR"};
} // namespace
std::string
get_output_filename(std::string_view fname, std::string_view ext)
{
@@ -55,16 +61,17 @@ get_output_filename(std::string_view fname, std::string_view ext)
return tool::format(output_path / fmt::format("{}_{}{}", output_prefix, fname, _ext));
}
std::pair<std::ostream*, output_stream_dtor_t>
output_stream_t
get_output_stream(std::string_view fname, std::string_view ext)
{
auto cfg_output_path = tool::format(tool::get_config().output_path);
if(cfg_output_path == "stdout" || cfg_output_path == "STDOUT")
if(stdout_names.count(cfg_output_path) > 0 || stdout_names.count(fname) > 0)
return {&std::cout, [](auto*&) {}};
else if(cfg_output_path == "stderr" || cfg_output_path == "STDERR")
else if(stderr_names.count(cfg_output_path) > 0 || stderr_names.count(fname) > 0)
return {&std::cout, [](auto*&) {}};
else if(cfg_output_path.empty())
else if(cfg_output_path.empty() || fname.empty())
return {&std::clog, [](auto*&) {}};
auto output_file = get_output_filename(fname, ext);
@@ -82,12 +89,12 @@ get_output_stream(std::string_view fname, std::string_view ext)
output_file::~output_file()
{
if(m_stream)
if(m_os.stream)
ROCP_INFO << "Closing result file: " << m_name;
else
ROCP_WARNING << "output_file::~output_file does not have a output stream instance!";
m_dtor(m_stream);
m_os.close();
}
} // namespace tool
} // namespace rocprofiler
source/lib/rocprofiler-sdk-tool/output_file.hpp
+45 -12
Zobrazit soubor
@@ -26,7 +26,6 @@
#include "csv.hpp"
#include "lib/common/filesystem.hpp"
#include "lib/rocprofiler-sdk-tool/csv.hpp"
#include <array>
#include <iostream>
@@ -41,12 +40,48 @@ namespace rocprofiler
{
namespace tool
{
using output_stream_dtor_t = void (*)(std::ostream*&);
using ostream_dtor_t = void (*)(std::ostream*&);
using output_stream_pair_t = std::pair<std::ostream*, ostream_dtor_t>;
struct output_stream_t
{
output_stream_t() = default;
output_stream_t(std::ostream* _os, ostream_dtor_t _dtor)
: stream{_os}
, dtor{_dtor}
{}
~output_stream_t() { close(); }
output_stream_t(const output_stream_t&) = delete;
output_stream_t(output_stream_t&&) noexcept = default;
output_stream_t& operator=(const output_stream_t&) = delete;
output_stream_t& operator=(output_stream_t&&) noexcept = default;
explicit operator bool() const { return stream != nullptr; }
template <typename Tp>
std::ostream& operator<<(Tp&& value)
{
return ((stream) ? *stream : std::cerr) << std::forward<Tp>(value) << std::flush;
}
void close()
{
if(stream) (*stream) << std::flush;
if(dtor) dtor(stream);
}
bool writes_to_file() const { return (dynamic_cast<std::ofstream*>(stream) != nullptr); }
std::ostream* stream = nullptr;
ostream_dtor_t dtor = nullptr;
};
std::string
get_output_filename(std::string_view fname, std::string_view ext);
std::pair<std::ostream*, output_stream_dtor_t>
output_stream_t
get_output_stream(std::string_view fname, std::string_view ext);
struct output_file
@@ -65,16 +100,15 @@ struct output_file
std::ostream& operator<<(T&& value)
{
auto _lk = std::unique_lock<std::mutex>{m_mutex};
return ((m_stream) ? *m_stream : std::cerr) << std::forward<T>(value) << std::flush;
return ((m_os.stream) ? *m_os.stream : std::cerr) << std::forward<T>(value) << std::flush;
}
operator bool() const { return m_stream != nullptr; }
operator bool() const { return m_os.stream != nullptr; }
private:
const std::string m_name = {};
std::mutex m_mutex = {};
std::ostream* m_stream = nullptr;
output_stream_dtor_t m_dtor = [](std::ostream*&) {};
const std::string m_name = {};
std::mutex m_mutex = {};
output_stream_t m_os = {};
};
template <size_t N>
@@ -82,9 +116,8 @@ output_file::output_file(std::string name,
csv::csv_encoder<N> encoder,
std::array<std::string_view, N>&& header)
: m_name{std::move(name)}
, m_os{get_output_stream(m_name, ".csv")}
{
std::tie(m_stream, m_dtor) = get_output_stream(m_name, ".csv");
for(auto& itr : header)
{
ROCP_FATAL_IF(itr.empty())
@@ -92,7 +125,7 @@ output_file::output_file(std::string name,
}
// write the csv header
if(m_stream) encoder.write_row(*m_stream, header);
if(m_os.stream) encoder.write_row(*m_os.stream, header);
}
} // namespace tool
} // namespace rocprofiler
source/lib/rocprofiler-sdk-tool/statistics.cpp
+75
Zobrazit soubor
@@ -0,0 +1,75 @@
// MIT License
//
// Copyright (c) 2023 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#include "statistics.hpp"
namespace rocprofiler
{
namespace tool
{
template <typename Tp>
std::ostream&
stats_formatter::operator()(std::ostream& ofs, const Tp& _val) const
{
using value_type = common::mpl::unqualified_type_t<Tp>;
if constexpr(std::is_floating_point<value_type>::value)
{
constexpr value_type one_hundredth = 1.0e-2;
if(_val > one_hundredth)
ofs << std::setprecision(6) << std::fixed;
else
ofs << std::setprecision(8) << std::scientific;
}
else if constexpr(std::is_same<Tp, percentage>::value)
{
constexpr float_type one = 1.0;
constexpr float_type one_hundredth = 1.0e-2;
if(_val.value >= one)
ofs << std::setprecision(2) << std::fixed;
else if(_val.value > one_hundredth)
ofs << std::setprecision(4) << std::fixed;
else
ofs << std::setprecision(3) << std::scientific;
}
return ofs;
}
#define STATS_FORMATTER_INSTANTIATE_TEMPLATE(TYPE) \
template std::ostream& stats_formatter::operator()(std::ostream&, const TYPE&) const;
STATS_FORMATTER_INSTANTIATE_TEMPLATE(std::string)
STATS_FORMATTER_INSTANTIATE_TEMPLATE(std::string_view)
STATS_FORMATTER_INSTANTIATE_TEMPLATE(percentage)
STATS_FORMATTER_INSTANTIATE_TEMPLATE(float)
STATS_FORMATTER_INSTANTIATE_TEMPLATE(double)
STATS_FORMATTER_INSTANTIATE_TEMPLATE(uint8_t)
STATS_FORMATTER_INSTANTIATE_TEMPLATE(uint16_t)
STATS_FORMATTER_INSTANTIATE_TEMPLATE(uint32_t)
STATS_FORMATTER_INSTANTIATE_TEMPLATE(uint64_t)
STATS_FORMATTER_INSTANTIATE_TEMPLATE(int8_t)
STATS_FORMATTER_INSTANTIATE_TEMPLATE(int16_t)
STATS_FORMATTER_INSTANTIATE_TEMPLATE(int32_t)
STATS_FORMATTER_INSTANTIATE_TEMPLATE(int64_t)
} // namespace tool
} // namespace rocprofiler
source/lib/rocprofiler-sdk-tool/statistics.hpp
+120
Zobrazit soubor
@@ -22,6 +22,13 @@
#pragma once
#include "domain_type.hpp"
#include "lib/common/logging.hpp"
#include "lib/common/mpl.hpp"
#include <rocprofiler-sdk/cxx/serialization.hpp>
#include <cmath>
#include <cstdint>
#include <fstream>
@@ -29,7 +36,9 @@
#include <iomanip>
#include <iostream>
#include <limits>
#include <map>
#include <type_traits>
#include <vector>
namespace rocprofiler
{
@@ -93,6 +102,8 @@ public:
}
float_type get_stddev() const { return ::std::sqrt(::std::abs(get_variance())); }
float_type get_percent(float_type _total) const;
float_type get_percent(const this_type&) const;
// Modifications
void reset()
@@ -209,6 +220,107 @@ public:
{
return statistics(lhs) -= rhs;
}
template <typename ArchiveT>
void serialize(ArchiveT& ar, const unsigned int) const
{
ar(cereal::make_nvp("count", m_cnt));
ar(cereal::make_nvp("sum", m_sum));
ar(cereal::make_nvp("sqr", m_sqr));
ar(cereal::make_nvp("min", m_min));
ar(cereal::make_nvp("max", m_max));
ar(cereal::make_nvp("mean", get_mean()));
ar(cereal::make_nvp("stddev", get_stddev()));
ar(cereal::make_nvp("variance", get_variance()));
}
};
template <typename Tp, typename Fp>
typename statistics<Tp, Fp>::float_type
statistics<Tp, Fp>::get_percent(float_type _total) const
{
constexpr float_type one_hundred = 100.0;
const float_type _sum = get_sum();
ROCP_WARNING_IF(static_cast<int64_t>(_sum) > static_cast<int64_t>(_total))
<< "percentage calculation > 100%. sum=" << _sum << " > total=" << _total;
return (_sum / _total) * one_hundred;
}
template <typename Tp, typename Fp>
typename statistics<Tp, Fp>::float_type
statistics<Tp, Fp>::get_percent(const statistics<Tp, Fp>& _rhs) const
{
return get_percent(_rhs.get_sum());
}
using float_type = double;
using stats_data_t = statistics<uint64_t, float_type>;
using stats_map_t = std::map<std::string_view, stats_data_t>;
using stats_pair_t = std::pair<std::string_view, stats_data_t>;
using stats_entry_vec_t = std::vector<stats_pair_t>;
inline bool
default_stats_sorter(const stats_pair_t& lhs, const stats_pair_t& rhs)
{
return (lhs.second.get_sum() > rhs.second.get_sum());
}
struct stats_entry_t
{
using sort_predicate_t = bool (*)(const stats_pair_t&, const stats_pair_t&);
stats_entry_t() = default;
~stats_entry_t() = default;
stats_entry_t(const stats_entry_t&) = default;
stats_entry_t(stats_entry_t&&) noexcept = default;
stats_entry_t& operator=(const stats_entry_t&) = default;
stats_entry_t& operator=(stats_entry_t&&) noexcept = default;
template <typename FuncT = sort_predicate_t>
stats_entry_t& sort(FuncT&& _predicate = default_stats_sorter);
explicit operator bool() const { return (total.get_count() > 0 && !entries.empty()); }
stats_data_t total = {};
stats_entry_vec_t entries = {};
template <typename ArchiveT>
void serialize(ArchiveT& ar, const unsigned int) const
{
total.serialize(ar, 0);
auto _entries_map = std::map<std::string, stats_data_t>{};
for(const auto& itr : entries)
_entries_map.emplace(std::string{itr.first}, itr.second);
ar(cereal::make_nvp("operations", _entries_map));
}
};
template <typename FuncT>
stats_entry_t&
stats_entry_t::sort(FuncT&& _predicate)
{
std::sort(entries.begin(), entries.end(), std::forward<FuncT>(_predicate));
return *this;
}
using domain_stats_t = std::pair<domain_type, stats_entry_t>;
using domain_stats_vec_t = std::vector<domain_stats_t>;
struct stats_formatter
{
template <typename Tp>
std::ostream& operator()(std::ostream& ofs, const Tp& _val) const;
};
struct percentage
{
float_type value = {};
friend std::ostream& operator<<(std::ostream& os, percentage val)
{
return (stats_formatter{}(os, val) << val.value);
}
};
} // namespace tool
} // namespace rocprofiler
@@ -228,4 +340,12 @@ min(::rocprofiler::tool::statistics<Tp> lhs, const Tp& rhs)
{
return lhs.get_min(rhs);
}
inline std::string
to_string(::rocprofiler::tool::percentage val)
{
auto _ss = std::stringstream{};
_ss << val;
return _ss.str();
}
} // namespace std
source/lib/rocprofiler-sdk-tool/tmp_file_buffer.hpp
+50 -3
Zobrazit soubor
@@ -22,6 +22,7 @@
#pragma once
#include "domain_type.hpp"
#include "helper.hpp"
#include "tmp_file.hpp"
@@ -35,6 +36,7 @@
#include <mutex>
#include <string>
#include <tuple>
#include <type_traits>
#include <utility>
template <typename Tp>
@@ -71,15 +73,60 @@ void
write_ring_buffer(Tp _v, domain_type type)
{
auto [_tmp_buf, _tmp_file] = get_tmp_file_buffer<ring_buffer_t<Tp>>(type);
if(_tmp_buf->capacity() == 0) return;
if(_tmp_buf->capacity() == 0)
{
ROCP_INFO << "rocprofv3 is dropping record from domain " << get_domain_column_name(type)
<< ". Buffer has a capacity of zero.";
return;
}
auto* ptr = _tmp_buf->request(false);
if(ptr == nullptr)
{
offload_buffer<ring_buffer_t<Tp>>(type);
ptr = _tmp_buf->request(false);
CHECK(ptr != nullptr);
// if failed, try again
if(!ptr) ptr = _tmp_buf->request(false);
// after second failure, emit warning message
ROCP_CI_LOG_IF(WARNING, !ptr)
<< "rocprofv3 is dropping record from domain " << get_domain_column_name(type)
<< ". No space in buffer: "
<< fmt::format(
"capacity={}, record_size={}, used_count={}, free_count={} | raw_info=[{}]",
_tmp_buf->capacity(),
_tmp_buf->data_size(),
_tmp_buf->count(),
_tmp_buf->free(),
_tmp_buf->as_string());
}
if(ptr)
{
if constexpr(std::is_move_constructible<Tp>::value)
{
new(ptr) Tp{std::move(_v)};
}
else if constexpr(std::is_move_assignable<Tp>::value)
{
*ptr = std::move(_v);
}
else if constexpr(std::is_copy_constructible<Tp>::value)
{
new(ptr) Tp{_v};
}
else if constexpr(std::is_copy_assignable<Tp>::value)
{
*ptr = _v;
}
else
{
static_assert(std::is_void<Tp>::value,
"data type is neither move/copy constructible nor move/copy assignable");
}
}
*ptr = std::move(_v);
}
template <typename Tp>
source/lib/rocprofiler-sdk-tool/tool.cpp
+26 -7
Zobrazit soubor
@@ -28,8 +28,10 @@
#include "generateJSON.hpp"
#include "generateOTF2.hpp"
#include "generatePerfetto.hpp"
#include "generateStats.hpp"
#include "helper.hpp"
#include "output_file.hpp"
#include "statistics.hpp"
#include "tmp_file.hpp"
#include "lib/common/environment.hpp"
@@ -1335,7 +1337,7 @@ tool_init(rocprofiler_client_finalize_t fini_func, void* tool_data)
"buffer tracing service for memory copy configure");
}
if(tool::get_config().scratch_memory)
if(tool::get_config().scratch_memory_trace)
{
ROCPROFILER_CALL(rocprofiler_create_buffer(get_client_ctx(),
buffer_size,
@@ -1500,21 +1502,32 @@ api_registration_callback(rocprofiler_intercept_table_t,
"Iterate rocporfiler agents")
}
using stats_data_t = ::rocprofiler::tool::stats_data_t;
using stats_data_t = ::rocprofiler::tool::stats_data_t;
using stats_entry_t = ::rocprofiler::tool::stats_entry_t;
using domain_stats_vec_t = ::rocprofiler::tool::domain_stats_vec_t;
template <typename Tp, domain_type DomainT>
void
generate_output(rocprofiler::tool::buffered_output<Tp, DomainT>& output_v,
std::unordered_map<domain_type, stats_data_t>& contributions_v)
domain_stats_vec_t& contributions_v)
{
if(!output_v) return;
output_v.read();
if(tool::get_config().stats || tool::get_config().summary_output)
{
output_v.stats = rocprofiler::tool::generate_stats(tool_functions, output_v.element_data);
}
if(output_v.stats)
{
contributions_v.emplace_back(output_v.buffer_type_v, output_v.stats);
}
if(tool::get_config().csv_output)
{
output_v.stats = rocprofiler::tool::generate_csv(tool_functions, output_v.element_data);
contributions_v.emplace(output_v.buffer_type_v, output_v.stats);
rocprofiler::tool::generate_csv(tool_functions, output_v.element_data, output_v.stats);
}
}
@@ -1543,7 +1556,7 @@ tool_fini(void* /*tool_data*/)
auto counters_output =
counter_collection_buffered_output_t{tool::get_config().counter_collection};
auto scratch_memory_output =
scratch_memory_buffered_output_t{tool::get_config().scratch_memory};
scratch_memory_buffered_output_t{tool::get_config().scratch_memory_trace};
auto node_id_sort = [](const auto& lhs, const auto& rhs) { return lhs.node_id < rhs.node_id; };
@@ -1561,7 +1574,7 @@ tool_fini(void* /*tool_data*/)
rocprofiler::tool::generate_csv(tool_functions, _agents);
}
auto contributions = std::unordered_map<domain_type, stats_data_t>{};
auto contributions = domain_stats_vec_t{};
generate_output(kernel_dispatch_output, contributions);
generate_output(hsa_output, contributions);
@@ -1580,6 +1593,7 @@ tool_fini(void* /*tool_data*/)
{
rocprofiler::tool::write_json(tool_functions,
getpid(),
contributions,
_agents,
_counters,
&hip_output.element_data,
@@ -1617,6 +1631,11 @@ tool_fini(void* /*tool_data*/)
&scratch_memory_output.element_data);
}
if(tool::get_config().summary_output)
{
rocprofiler::tool::generate_stats(tool_functions, contributions);
}
auto destroy_output = [](auto& _buffered_output_v) { _buffered_output_v.destroy(); };
destroy_output(kernel_dispatch_output);
source/lib/rocprofiler-sdk/hsa/async_copy.cpp
+1 -1
Zobrazit soubor
@@ -72,7 +72,7 @@ struct async_copy_info;
struct async_copy_info<ROCPROFILER_MEMORY_COPY_##DIRECTION> \
{ \
static constexpr auto operation_idx = ROCPROFILER_MEMORY_COPY_##DIRECTION; \
static constexpr auto name = #DIRECTION; \
static constexpr auto name = "MEMORY_COPY_" #DIRECTION; \
};
SPECIALIZE_ASYNC_COPY_INFO(NONE)