Agent, Counters, and AQL (#55)
* Migrate XML counter defs and reader from v1/v2 * Current Working Set * Modified parser * Evaluate AST Start * Update lib/common/xml - move definitions out of class declaration * Update lib/rocprofiler/counters/parser - update build of bison and flex build - reproducible generation - add ROCPROFILER_REGENERATE_COUNTERS_PARSER option - fix namespacing * Update lib/rocprofiler/counters/xml - change location of XML files and install them * Update lib/rocprofiler/counter/tests - normalize the test names - improve test failures (more clear about where failure is) * Update lib/rocprofiler/counters - fix namespace - update to new XML metrics directory * Update lib/rocprofiler/CMakeLists.txt - link to object library * Update lib/rocprofiler/hsa/types.hpp - reorganize includes * Add metric loading class/printers * Agent Implementation * Queue Implementation (#79) * Queue Implementation * API Implementation For Counters (part 1) (#80) * API Implementation For Counters * Bewelton/counter collection 3 (#84) * Added counter sample * More changes * More changes * Update samples/counter_collection - mostly formatting * Update include/rocprofiler/counters.h - formatting * Add lib.common/synchronized.hpp - Synchronized struct * Update lib/rocprofiler/counters/xml/basic_counters.xml - whitespace * Update scripts/patch-parser.cmake - tweaks for consistency * Update lib/rocprofiler/counters/parser/tests/parser_tests.cpp - formatting * Update lib/rocprofiler/counters/parser - improve consistency in rocprofiler-expr-parser-patch - update parser.{h,cpp} and scanner.cpp - formatting + regenerated * Update lib/rocprofiler/aql - formatting - clang-tidy fixes - guard against memory pool access errors * Update lib/rocprofiler/aql/tests - formatting - update use of get_val - normalize test names * Update lib/rocprofiler/counters/tests - formatting - patch basic_counters and derived_counters - normalize test names * Update lib/rocprofiler/aql/tests - set_tests_properties * Update test labels - fix minor issue with gtest labels * Update lib/rocprofiler/counters - formatting - clang-tidy fixes * Update lib/rocprofiler/hsa - fix includes - formatting - clang-tidy fixes - tweak to queue_controller_init interface * Update lib/rocprofiler - include fixes - namespace fixes - clang-tidy fixes - formatting * Update scripts/run-ci.py - exclude counters/parser from code coverage (generated files) * Update include/rocprofiler/counters.h - fix doxygen comment * Update lib/rocprofiler/aql/packet_construct.cpp - guard against HSA_AMD_MEMORY_POOL_ACCESS_DISALLOWED_BY_DEFAULT and HSA_AMD_MEMORY_POOL_ACCESS_NEVER_ALLOWED * Update lib/rocprofiler/counters/parser/raw_ast.hpp - clang-tidy fixes * Update lib/rocprofiler/counters/evaluate_ast.hpp - clang-tidy fixes * Update lib/rocprofiler/aql/tests - disable packet_generation_single and packet_generation_multi tests - the entire implementation rocprofiler::get_ext_table() is incorrect * Minor fixes before cleanup * More changes * More fixes * More fixes * source formatting (clang-format v11) (#99) Co-authored-by: bwelton <bwelton@users.noreply.github.com> * Revert PTL submodule * Update scripts/run-ci.py - exclude counters/parser from code coverage (generated files) * Migrating counters state to context * Linting * source formatting (clang-format v11) (#101) Co-authored-by: bwelton <bwelton@users.noreply.github.com> * revert run-ci * Testing fixes * More test changes * Fix minor typo * Small queue change * Small queue change * source formatting (clang-format v11) (#102) Co-authored-by: bwelton <bwelton@users.noreply.github.com> * source formatting (clang-format v11) (#105) Co-authored-by: bwelton <bwelton@users.noreply.github.com> * Documentation Change * More documentation fixes * source formatting (clang-format v11) (#106) Co-authored-by: bwelton <bwelton@users.noreply.github.com> * Threading fixes * Threading fixes * source formatting (clang-format v11) (#107) Co-authored-by: bwelton <bwelton@users.noreply.github.com> * Threading fixes * More test fixes * More agent fixes * More build fixes * source formatting (clang-format v11) (#109) Co-authored-by: bwelton <bwelton@users.noreply.github.com> * changed test timeouts * Build fix * Build fix * Updates to agent * source formatting (clang-format v11) (#114) Co-authored-by: bwelton <bwelton@users.noreply.github.com> * cmake formatting (cmake-format) (#113) Co-authored-by: bwelton <bwelton@users.noreply.github.com> * remove git worktree folder * Doc update * testing fix * Another test fix * More test changes * Rebase * source formatting (clang-format v11) (#116) Co-authored-by: bwelton <bwelton@users.noreply.github.com> * Documentation * source formatting (clang-format v11) (#119) Co-authored-by: bwelton <bwelton@users.noreply.github.com> * PTL Changes * Minor agent fix for empty labels * source formatting (clang-format v11) (#120) Co-authored-by: bwelton <bwelton@users.noreply.github.com> * Minor agent fix for empty labels * Refactor read_map * source formatting (clang-format v11) (#121) Co-authored-by: bwelton <bwelton@users.noreply.github.com> * Refactor read_map * Cache fixes * source formatting (clang-format v11) (#122) Co-authored-by: bwelton <bwelton@users.noreply.github.com> --------- Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: bwelton <bwelton@users.noreply.github.com>
This commit is contained in:
Benjamin Welton
GitHub
@@ -22,15 +22,22 @@ THE SOFTWARE.
|
||||
|
||||
#include "metrics.hpp"
|
||||
|
||||
#include <rocprofiler/rocprofiler.h>
|
||||
|
||||
#include "lib/common/synchronized.hpp"
|
||||
#include "lib/common/utility.hpp"
|
||||
#include "lib/common/xml.hpp"
|
||||
|
||||
#include "glog/logging.h"
|
||||
|
||||
#include <dlfcn.h> // for dladdr
|
||||
#include <atomic>
|
||||
#include <cstdlib>
|
||||
#include <filesystem>
|
||||
#include <optional>
|
||||
|
||||
#include "glog/logging.h"
|
||||
#include "lib/common/xml.hpp"
|
||||
#include "rocprofiler/rocprofiler.h"
|
||||
|
||||
namespace rocprofiler
|
||||
{
|
||||
namespace counters
|
||||
{
|
||||
namespace
|
||||
@@ -38,10 +45,11 @@ namespace
|
||||
MetricMap
|
||||
loadXml(const std::string& filename)
|
||||
{
|
||||
MetricMap ret;
|
||||
static std::atomic<uint64_t> id = 0;
|
||||
MetricMap ret;
|
||||
DLOG(INFO) << "Loading Counter Config: " << filename;
|
||||
// todo: return unique_ptr....
|
||||
auto xml = xml::Xml::Create(filename);
|
||||
auto xml = common::Xml::Create(filename);
|
||||
LOG_IF(FATAL, !xml)
|
||||
<< "Could not open XML Counter Config File (set env ROCPROFILER_METRICS_PATH)";
|
||||
|
||||
@@ -68,23 +76,25 @@ loadXml(const std::string& filename)
|
||||
node->opts["block"],
|
||||
node->opts["event"],
|
||||
node->opts["descr"],
|
||||
node->opts["expr"]);
|
||||
node->opts["expr"],
|
||||
node->opts["special"],
|
||||
id);
|
||||
id++;
|
||||
}
|
||||
}
|
||||
|
||||
DLOG(INFO) << fmt::format("{}", ret);
|
||||
return ret;
|
||||
}
|
||||
|
||||
std::string
|
||||
findViaInstallPath(const std::string& filename)
|
||||
{
|
||||
Dl_info dl_info;
|
||||
Dl_info dl_info = {};
|
||||
DLOG(INFO) << filename << " is being looked up via install path";
|
||||
if(dladdr(reinterpret_cast<const void*>(rocprofiler_query_available_agents), &dl_info) != 0)
|
||||
{
|
||||
return std::filesystem::path{dl_info.dli_fname}.remove_filename() /
|
||||
fmt::format("../lib/{}", filename);
|
||||
return std::filesystem::path{dl_info.dli_fname}.parent_path().parent_path() /
|
||||
fmt::format("share/rocprofiler/{}", filename);
|
||||
}
|
||||
return filename;
|
||||
}
|
||||
@@ -92,10 +102,10 @@ findViaInstallPath(const std::string& filename)
|
||||
std::string
|
||||
findViaEnvironment(const std::string& filename)
|
||||
{
|
||||
if(getenv("ROCPROFILER_METRICS_PATH"))
|
||||
if(const char* metrics_path = nullptr; (metrics_path = getenv("ROCPROFILER_METRICS_PATH")))
|
||||
{
|
||||
DLOG(INFO) << filename << " is being looked up via env variable ROCPROFILER_METRICS_PATH";
|
||||
return std::filesystem::path{std::string(getenv("ROCPROFILER_METRICS_PATH"))} / filename;
|
||||
return std::filesystem::path{std::string{metrics_path}} / filename;
|
||||
}
|
||||
// No environment variable, lookup via install path
|
||||
return findViaInstallPath(filename);
|
||||
@@ -115,4 +125,58 @@ getBaseHardwareMetrics()
|
||||
return loadXml(findViaEnvironment("basic_counters.xml"));
|
||||
}
|
||||
|
||||
}; // namespace counters
|
||||
const MetricIdMap&
|
||||
getMetricIdMap()
|
||||
{
|
||||
static MetricIdMap id_map = []() {
|
||||
MetricIdMap map;
|
||||
for(const auto& [_, val] : getMetricMap())
|
||||
{
|
||||
for(const auto& metric : val)
|
||||
{
|
||||
map.emplace(metric.id(), metric);
|
||||
}
|
||||
}
|
||||
return map;
|
||||
}();
|
||||
return id_map;
|
||||
}
|
||||
|
||||
const MetricMap&
|
||||
getMetricMap()
|
||||
{
|
||||
static MetricMap map = []() {
|
||||
MetricMap ret = getBaseHardwareMetrics();
|
||||
for(auto& [key, val] : getDerivedHardwareMetrics())
|
||||
{
|
||||
auto [iter, inserted] = ret.emplace(key, val);
|
||||
if(!inserted)
|
||||
{
|
||||
iter->second.insert(iter->second.end(), val.begin(), val.end());
|
||||
}
|
||||
}
|
||||
return ret;
|
||||
}();
|
||||
return map;
|
||||
}
|
||||
|
||||
const std::vector<Metric>&
|
||||
getMetricsForAgent(const std::string& agent)
|
||||
{
|
||||
static const std::vector<Metric> empty;
|
||||
const auto& map = getMetricMap();
|
||||
if(const auto* metric_ptr = rocprofiler::common::get_val(map, agent))
|
||||
{
|
||||
return *metric_ptr;
|
||||
}
|
||||
|
||||
return empty;
|
||||
}
|
||||
|
||||
bool
|
||||
operator<(Metric const& lhs, Metric const& rhs)
|
||||
{
|
||||
return lhs.id() < rhs.id();
|
||||
}
|
||||
} // namespace counters
|
||||
} // namespace rocprofiler
|
||||
|
||||
Reference in New Issue
Block a user