[SDK] Standardize rocprofiler-sdk counter definition YAML schema (#370)

* Convert YAML Format

Convert YAML format and reader to properly read the YAML.

Comparison between output's from the YAML show only changes in ordering
of architectures (and ids).

* Test fixes

* Add script for converting the YAML schema to source/scripts

* Update documentation

* Change the extra counter code block to YAML

* Add missing new line at EOF

* remove name issues

---------

Co-authored-by: Benjamin Welton <bewelton@amd.com>
Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>

source/docs/how-to/using-rocprofv3.rst

@@ -85,7 +85,7 @@ The following table lists the commonly used ``rocprofv3`` command-line options c
        | ``--hsa-trace`` [BOOL] |br| |br| |br| |br| |br| |br| |br| |br|
        | ``--rccl-trace`` [BOOL] |br| |br| |br| |br|
        | ``--kokkos-trace`` [BOOL] |br| |br| |br| |br|
-       | ``--rocdecode-trace`` [BOOL] 
+       | ``--rocdecode-trace`` [BOOL]
      - | Combination of ``--hip-runtime-trace`` and ``--hip-compiler-trace``. This option only enables the HIP API tracing. Unlike previous iterations of ``rocprof``, this option doesn't enable kernel tracing, memory copy tracing, and so on. |br| |br|
        | Collects marker (ROCTx) traces. Similar to ``--roctx-trace`` option in earlier ``rocprof`` versions, but with improved ``ROCTx`` library with more features. |br| |br|
        | Collects kernel dispatch traces. |br| |br|
@@ -170,7 +170,7 @@ The following table lists the commonly used ``rocprofv3`` command-line options c
    * - Other
      - | ``--preload`` PRELOAD  |br| |br|
        | ``--minimum-output-data`` |br| |br|
-       | ``--disable-signal-handlers`` 
+       | ``--disable-signal-handlers``
      - | Specifies libraries to prepend to ``LD_PRELOAD``. It is useful for sanitizer libraries. |br| |br|
        | Output files are generated only if output data size is greater than minimum output data size. It can be used for controlling the generation of output files so that user don't recieve empty files. The input is in KB units. |br| |br|
        | Disables the signal handlers in the rocprofv3 tool. It disables the prioritizing of rocprofv3 signal handler over application installed signal handler. When --disable-signal-handlers is set to true, and application has its signal handler on SIGSEGV or similar installed, then its signal handler will be used not the rocprofv3 signal handler. Note: glog still installs signal handlers which provide backtraces.
@@ -894,17 +894,47 @@ While the basic counters and derived metrics are available for collection by def
 
 You can define the extra counters in a YAML file as shown:
 
-.. code-block:: shell
+.. code-block:: yaml
 
-   $ cat extra_counters.yaml
+    rocprofiler-sdk:
+      counters-schema-version: 1
+      counters:
+        - name: GRBM_GUI_ACTIVE_SUM
+          description: "Unit: cycles"
+          properties: []
+          definitions:
+            - architectures:
+                - gfx10
+                - gfx1010
+                - gfx1030
+                - gfx1031
+                - gfx1032
+                - gfx11
+                - gfx1100
+                - gfx1101
+                - gfx1102
+                - gfx9
+                - gfx906
+                - gfx908
+                - gfx90a
+                - gfx942
+              expression: reduce(GRBM_GUI_ACTIVE,max)*CU_NUM
+        - name: CPC_CPC_STAT_BUSY
+          description: CPC Busy.
+          properties: []
+          definitions:
+            - architectures:
+                - gfx940
+                - gfx941
+              block: CPC
+              event: 25
 
-   GRBM_GUI_ACTIVE_SUM:
-      architectures:
-         gfx942/gfx10/gfx1010/gfx1030/gfx1031/gfx11/gfx1032/gfx1102/gfx906/gfx1100/gfx1101/gfx908/gfx90a/gfx9:
-      expression: reduce(GRBM_GUI_ACTIVE,max)*CU_NUM
-      description: 'Unit: cycles'
+Please note, the above sample uses the ``CPC_CPC_STAT_BUSY`` counter definition for the ``gfx940``
+and ``gfx941`` architectures to demonstrate the YAML schema when counters have different
+architecture-specific definitions.
 
-To collect the extra counters defined in the `extra_counters.yaml` file , use:
+If this YAML is placed in a ``extra_counters.yaml`` file, to collect the extra counters defined
+in the ``extra_counters.yaml`` file, use the ``-E`` / ``--extra-counters`` option:
 
 .. code-block:: shell
 
@@ -1021,7 +1051,7 @@ The generated Perfetto trace file can be opened in the `Perfetto UI <https://ui.
 You can also combine this with the system trace option to get a more comprehensive view of the system's performance. For example, you can use the following command to collect both system trace and performance counter data:
 
 .. code-block:: bash
-   
+
   rocprofv3 --pmc SQ_WAVES GRBM_COUNT --sys-trace --output-format pftrace -- <application_path>
 
 .. image:: /data/perfetto_counters.png

source/lib/rocprofiler-sdk/counters/metrics.cpp

@@ -107,6 +107,7 @@ loadYAML(const std::string& filename, std::optional<ArchMetric> add_metric)
 {
     // Stores metrics that are added via the API
     static MetricMap added_metrics;
+    YAML::Node       append_yaml;
 
     MetricMap ret;
     auto      override = getCustomCounterDefinition().wlock([&](auto& data) {
@@ -121,62 +122,53 @@ loadYAML(const std::string& filename, std::optional<ArchMetric> add_metric)
         std::ifstream file(filename);
         counter_data << file.rdbuf();
     }
-
-    if(!override.data.empty())
+    else
     {
         ROCP_INFO << "Adding Override Config Data: " << override.data;
         counter_data << override.data;
     }
 
     auto     yaml       = YAML::Load(counter_data.str());
+    auto     header     = yaml["rocprofiler-sdk"]["counters"];
     uint64_t current_id = 0;
-
-    for(auto it = yaml.begin(); it != yaml.end(); ++it)
+    if(!override.data.empty() && override.append)
     {
-        auto counter_name = it->first.as<std::string>();
-        if(counter_name == "schema-version") continue;
-        auto counter_def  = it->second;
-        auto def_iterator = counter_def["architectures"];
-
-        for(auto def_it = def_iterator.begin(); def_it != def_iterator.end(); ++def_it)
+        append_yaml = YAML::Load(override.data);
+        if(append_yaml["rocprofiler-sdk"] && append_yaml["rocprofiler-sdk"]["counters"])
         {
-            auto archs = def_it->first.as<std::string>();
-            auto def   = def_it->second;
-            // To save space in the YAML file, we combine architectures with the same
-            // definition into a single entry. Split these out into separate entries.
-            // architectures:
-            //     gfx10/gfx1010/gfx1030/gfx1031/.....9:
-            //     expression: 400*SQ_WAIT_INST_LDS/SQ_WAVES/GRBM_GUI_ACTIVE
-            std::vector<std::string> result;
-            std::stringstream        ss(archs);
-            std::string              arch_name;
-
-            while(std::getline(ss, arch_name, '/'))
+            for(const auto& counter : append_yaml["rocprofiler-sdk"]["counters"])
             {
-                auto& metricVec = ret.emplace(arch_name, std::vector<Metric>()).first->second;
+                header.push_back(counter);
+            }
+        }
+    }
+
+    for(const auto& counter : header)
+    {
+        auto counter_name = counter["name"].as<std::string>();
+        auto description  = counter["description"].as<std::string>();
+        for(const auto& definition : counter["definitions"])
+        {
+            for(const auto& arch : definition["architectures"])
+            {
+                auto& metricVec =
+                    ret.emplace(arch.as<std::string>(), std::vector<Metric>()).first->second;
                 if(metricVec.empty())
                 {
                     const auto constants = get_constants(current_id);
                     metricVec.insert(metricVec.end(), constants.begin(), constants.end());
                     current_id += constants.size();
                 }
-
-                std::string description;
-                if(def["description"])
-                    description = def["description"].as<std::string>();
-                else if(counter_def["description"])
-                    description = counter_def["description"].as<std::string>();
                 metricVec.emplace_back(
-                    arch_name,
+                    arch.as<std::string>(),
                     counter_name,
-                    (def["block"] ? def["block"].as<std::string>() : ""),
-                    (def["event"] ? def["event"].as<std::string>() : ""),
+                    (definition["block"] ? definition["block"].as<std::string>() : ""),
+                    (definition["event"] ? definition["event"].as<std::string>() : ""),
                     description,
-                    (def["expression"] ? def["expression"].as<std::string>() : ""),
+                    (definition["expression"] ? definition["expression"].as<std::string>() : ""),
                     "",
                     current_id);
                 current_id++;
-                ROCP_TRACE << fmt::format("Inserted info {}: {}", arch_name, metricVec.back());
             }
         }
     }

source/lib/rocprofiler-sdk/counters/tests/core.cpp

@@ -755,11 +755,33 @@ TEST(core, public_api_iterate_agents)
 TEST(core, check_load_counter_def_append)
 {
     const std::string test_yaml = R"(
-TEST_YAML_LOAD:
-  architectures:
-    gfx950/gfx942/gfx10/gfx1010/gfx1030/gfx1031/gfx11/gfx1032/gfx1102/gfx906/gfx1100/gfx1101/gfx908/gfx90a/gfx9/gfx12/gfx1200/gfx1201:
-      expression: reduce(GRBM_GUI_ACTIVE,max)*CU_NUM
-  description: 'Unit: cycles'
+rocprofiler-sdk:
+  counters-schema-version: 1
+  counters:
+    - name: TEST_YAML_LOAD
+      description: cycles
+      properties: []
+      definitions:
+        - architectures:  
+          - gfx950
+          - gfx942
+          - gfx10
+          - gfx1010
+          - gfx1030
+          - gfx1031
+          - gfx11
+          - gfx1032
+          - gfx1102
+          - gfx906
+          - gfx1100
+          - gfx1101
+          - gfx908
+          - gfx90a
+          - gfx9
+          - gfx12
+          - gfx1200
+          - gfx1201
+          expression: reduce(GRBM_GUI_ACTIVE,max)*CU_NUM
     )";
     ASSERT_EQ(hsa_init(), HSA_STATUS_SUCCESS);
     test_init();
@@ -782,17 +804,61 @@ TEST_YAML_LOAD:
 TEST(core, check_load_counter_def)
 {
     const std::string test_yaml = R"(
-GRBM_GUI_ACTIVE:
-  architectures:
-    gfx950/gfx942/gfx941/gfx10/gfx1010/gfx1030/gfx1031/gfx11/gfx1032/gfx1102/gfx906/gfx1100/gfx1101/gfx940/gfx908/gfx900/gfx90a/gfx9/gfx12/gfx1200/gfx1201:
-      block: GRBM
-      event: 2
-  description: The GUI is Active
-TEST_YAML_LOAD:
-  architectures:
-    gfx950/gfx942/gfx10/gfx1010/gfx1030/gfx1031/gfx11/gfx1032/gfx1102/gfx906/gfx1100/gfx1101/gfx908/gfx90a/gfx9/gfx12/gfx1200/gfx1201:
-      expression: reduce(GRBM_GUI_ACTIVE,max)
-  description: cycles
+rocprofiler-sdk:
+  counters-schema-version: 1
+  counters:  
+    - name: GRBM_GUI_ACTIVE
+      description: The GUI is Active
+      properties: []
+      definitions:
+        - architectures:  
+          - gfx950
+          - gfx942
+          - gfx941
+          - gfx10
+          - gfx1010
+          - gfx1030
+          - gfx1031
+          - gfx11
+          - gfx1032
+          - gfx1102
+          - gfx906
+          - gfx1100
+          - gfx1101
+          - gfx940
+          - gfx908
+          - gfx900
+          - gfx90a
+          - gfx9
+          - gfx12
+          - gfx1200
+          - gfx1201
+          block: GRBM
+          event: 2
+    - name: TEST_YAML_LOAD
+      description: cycles
+      properties: []
+      definitions:
+        - architectures:  
+          - gfx950
+          - gfx942
+          - gfx10
+          - gfx1010
+          - gfx1030
+          - gfx1031
+          - gfx11
+          - gfx1032
+          - gfx1102
+          - gfx906
+          - gfx1100
+          - gfx1101
+          - gfx908
+          - gfx90a
+          - gfx9
+          - gfx12
+          - gfx1200
+          - gfx1201
+          expression: reduce(GRBM_GUI_ACTIVE,max)
     )";
     ASSERT_EQ(hsa_init(), HSA_STATUS_SUCCESS);
     test_init();

source/scripts/convert-counter-collection-yaml-schema.py

@@ -0,0 +1,91 @@
+#!/usr/bin/env python3
+
+import sys
+import yaml
+import argparse
+
+
+def fatal_error(msg, exit_code=1):
+    sys.stderr.write(f"Fatal error: {msg}\n")
+    sys.stderr.flush()
+    sys.exit(exit_code)
+
+
+def parse_yaml(yaml_file):
+    try:
+        with open(yaml_file, "r") as file:
+            data = yaml.safe_load(file)
+        return data
+
+    except yaml.YAMLError as exc:
+        fatal_error(f"{exc}")
+
+
+def dump_yaml(data, ostream):
+    yaml.dump(
+        data,
+        ostream,
+        default_flow_style=False,
+        canonical=False,
+        indent=1,
+        width=120,
+        sort_keys=False,
+    )
+
+
+def convert_yaml_schema(inp):
+    data = {"rocprofiler-sdk": {}}
+    data["rocprofiler-sdk"]["counters-schema-version"] = inp["schema-version"]
+    data["rocprofiler-sdk"]["counters"] = []
+
+    for key, itr in inp.items():
+        if key == "schema-version":
+            pass
+        else:
+            entry = {
+                "name": key,
+                "description": itr["description"],
+                "properties": [],
+                "definitions": [],
+            }
+            for arch, aitr in itr["architectures"].items():
+                definition = {"architectures": sorted(arch.split("/"))}
+
+                # expression or block + event
+                for dkey, ditr in aitr.items():
+                    definition[dkey] = ditr
+
+                # add the arch-specific counter definition
+                entry["definitions"].append(definition)
+
+            # add the counter entry
+            data["rocprofiler-sdk"]["counters"].append(entry)
+
+    return data
+
+
+if __name__ == "__main__":
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument("-i", "--input", help="Input file", type=str, required=True)
+    parser.add_argument(
+        "-o",
+        "--output",
+        help="Output file (stdout if not provided)",
+        type=str,
+        default=None,
+        required=False,
+    )
+    args = parser.parse_args(sys.argv[1:])
+
+    inp = parse_yaml(args.input)
+    if "rocprofiler-sdk" not in inp.keys():
+        data = convert_yaml_schema(inp)
+    else:
+        data = inp
+
+    if args.output is None:
+        dump_yaml(data, sys.stdout)
+    else:
+        with open(args.output, "w") as outfile:
+            dump_yaml(data, outfile)

tests/rocprofv3/counter-collection/extra_counters/extra_counters.yaml

@@ -1,5 +1,27 @@
-TEST_YAML_LOAD:
-  architectures:
-    gfx950/gfx942/gfx10/gfx1010/gfx1030/gfx1031/gfx11/gfx1032/gfx1102/gfx906/gfx1100/gfx1101/gfx908/gfx90a/gfx9/gfx12/gfx1200/gfx1201:
-      expression: reduce(GRBM_GUI_ACTIVE,max)*CU_NUM
-  description: 'Unit: cycles'
+rocprofiler-sdk:
+  counters-schema-version: 1
+  counters:
+    - name: TEST_YAML_LOAD
+      description: cycles
+      properties: []
+      definitions:
+        - architectures:
+          - gfx950
+          - gfx942
+          - gfx10
+          - gfx1010
+          - gfx1030
+          - gfx1031
+          - gfx11
+          - gfx1032
+          - gfx1102
+          - gfx906
+          - gfx1100
+          - gfx1101
+          - gfx908
+          - gfx90a
+          - gfx9
+          - gfx12
+          - gfx1200
+          - gfx1201
+          expression: reduce(GRBM_GUI_ACTIVE,max)*CU_NUM