AI/rocm-systems
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Evaluation portion for metrics (#123)

Browse Source 
* EvaluateAST and validation of RawAST

* Adding MetricDimension class and concepts

* set_dimensions() and improved ValidateRawAST()

* source formatting (clang-format v11) (#124)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Addressing 1st round of review comments

* Modified the parser production rules to support the right syntax for REDUCE and SELECT derived metric expressions

* changes to raw_ast.hpp and fmt::format()

* Parser tests updated to support corrected REDUCE and SELECT syntax

* changes to EvaluateAST::set_dimensions() and other dimension related code changes

* Added a test for EvaluateAST::evaluate() to test basic arithmetic on EvaluateAST

* Format source code (via clang-format v11) on sauverma/evaluate-ast (#146)

* source formatting (clang-format v11)

* Add dimension information to counter record

Restructures counter records to have the following design:

rocprofiler_record_id_t which is an int64_t that encodes
both the counter id and dimension information for the
record. The first 16 bits are reserved for the counter id while
the last 48 are split among the dimensions specified in
rocprofiler_dimension_t (currently 8 bits per dimension).
Each of the 8 bits for the dimension stores the dimension
value for that dimension for this record (i.e. a value of 8
on dimension XCC would denote XCC[8] for the counter). The
split among the dimensions will automatically adjust as
dimensions are added or removed.

The record also contains a union of {int64_t hw_counter, double
derived_counter} to specify the value of the record at
rocprofiler_record_id_t. int64_t denotes a physical hardware
counter that has integer types while the double is used for derived
counters (which type this counters values are needs to be queried
separately).

* Integration of new id type + other fixes

---------

Co-authored-by: sauverma93 <sauverma93@users.noreply.github.com>
Co-authored-by: Benjamin Welton <bewelton@amd.com>

* Fixed sissues with reduce() implementation and added a test for reduce()

* Updated parser syntax for reduce() and updated the parser test. Disabled the test for select()

* Build warning fixes

* Modifications to support fetching xcc/etc info from agent

* Initial plumbing working for single counters, cleanup+tests still needed

* Remove string comparison from reduce ops

* source formatting (clang-format v11) (#163)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* cmake formatting (cmake-format) (#164)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* source formatting (clang-format v11) (#171)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Merged with master

* source formatting (clang-format v11) (#172)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* source formatting (clang-format v11) (#173)

Co-authored-by: bwelton <bwelton@users.noreply.github.com>

* Test fix

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: bwelton <bwelton@users.noreply.github.com>
Co-authored-by: sauverma93 <sauverma93@users.noreply.github.com>
Co-authored-by: Benjamin Welton <bewelton@amd.com>

[ROCm/rocprofiler-sdk commit: 63775f241a]
This commit is contained in:
Saurabh Verma
2023-11-03 23:10:40 -05:00
committed by GitHub
parent 370a86830b
commit e8fa4d3873
30 changed files with 1960 additions and 973 deletions
Download Patch File Download Diff File Expand all files Collapse all files
projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/parser.cpp
+319 -579
View File
File diff suppressed because it is too large Load Diff
projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/parser.h
+39 -32
View File
@@ -1,8 +1,8 @@
/* A Bison parser, made by GNU Bison 3.5.1. */
/* A Bison parser, made by GNU Bison 3.8.2. */
/* Bison interface for Yacc-like parsers in C
Copyright (C) 1984, 1989-1990, 2000-2015, 2018-2020 Free Software Foundation,
Copyright (C) 1984, 1989-1990, 2000-2015, 2018-2021 Free Software Foundation,
Inc.
This program is free software: you can redistribute it and/or modify
@@ -16,7 +16,7 @@
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>. */
along with this program. If not, see <https://www.gnu.org/licenses/>. */
/* As a special exception, you may create a larger work that contains
part or all of the Bison parser skeleton and distribute that work
@@ -31,8 +31,9 @@
This special exception was added by the Free Software Foundation in
version 2.2 of Bison. */
/* Undocumented macros, especially those whose name start with YY_,
are private implementation details. Do not rely on them. */
/* DO NOT RELY ON FEATURES THAT ARE NOT DOCUMENTED in the manual,
especially those whose name start with YY_ or yy_. They are
private implementation details that can be changed or removed. */
#ifndef YY_YY_ROCPROFILER_SOURCE_LIB_ROCPROFILER_COUNTERS_PARSER_PARSER_H_INCLUDED
#define YY_YY_ROCPROFILER_SOURCE_LIB_ROCPROFILER_COUNTERS_PARSER_PARSER_H_INCLUDED
@@ -50,35 +51,40 @@ extern int yydebug;
using namespace rocprofiler::counters;
#define YYDEBUG 1
#line 54 "parser.h"
#line 55 "parser.h"
/* Token type. */
/* Token kinds. */
#ifndef YYTOKENTYPE
# define YYTOKENTYPE
enum yytokentype
{
ADD = 258,
SUB = 259,
MUL = 260,
DIV = 261,
ABS = 262,
EQUALS = 263,
OP = 264,
CP = 265,
O_SQ = 266,
C_SQ = 267,
COLON = 268,
EOL = 269,
UMINUS = 270,
CM = 271,
NUMBER = 272,
RANGE = 273,
NAME = 274,
REDUCE = 275,
SELECT = 276,
LOWER_THAN_ELSE = 277,
ELSE = 278
YYEMPTY = -2,
YYEOF = 0, /* "end of file" */
YYerror = 256, /* error */
YYUNDEF = 257, /* "invalid token" */
ADD = 258, /* ADD */
SUB = 259, /* SUB */
MUL = 260, /* MUL */
DIV = 261, /* DIV */
ABS = 262, /* ABS */
EQUALS = 263, /* EQUALS */
OP = 264, /* OP */
CP = 265, /* CP */
O_SQ = 266, /* O_SQ */
C_SQ = 267, /* C_SQ */
COLON = 268, /* COLON */
EOL = 269, /* EOL */
UMINUS = 270, /* UMINUS */
CM = 271, /* CM */
NUMBER = 272, /* NUMBER */
RANGE = 273, /* RANGE */
NAME = 274, /* NAME */
REDUCE = 275, /* REDUCE */
SELECT = 276, /* SELECT */
LOWER_THAN_ELSE = 277, /* LOWER_THAN_ELSE */
ELSE = 278 /* ELSE */
};
typedef enum yytokentype yytoken_kind_t;
#endif
/* Value type. */
@@ -87,11 +93,12 @@ union YYSTYPE
{
# line 34 "parser.y"
RawAST* a; /* For ast node */
int64_t d;
char* s;
RawAST* a; /* For ast node */
LinkedList* ll; /* For linked list node */
int64_t d;
char* s;
# line 95 "parser.h"
# line 102 "parser.h"
};
typedef union YYSTYPE YYSTYPE;
# define YYSTYPE_IS_TRIVIAL 1
projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/parser.y
+28 -28
View File
@@ -33,6 +33,7 @@ void yyerror(rocprofiler::counters::RawAST**, const char *s) { LOG(ERROR) << s;
/*declare data types*/
%union {
RawAST* a; /* For ast node */
LinkedList* ll; /* For linked list node */
int64_t d;
char* s;
}
@@ -43,6 +44,7 @@ void yyerror(rocprofiler::counters::RawAST**, const char *s) { LOG(ERROR) << s;
%type <a> exp /* set data type for expressions */
%type <s> NAME
%type <d> NUMBER
%type <ll> reduce_dim_args select_dim_args
%nonassoc LOWER_THAN_ELSE
%nonassoc ELSE
@@ -54,14 +56,6 @@ void yyerror(rocprofiler::counters::RawAST**, const char *s) { LOG(ERROR) << s;
top:
exp { *result = $1;};
// line: /* nothing */
// | line exp EOL {
// // TODO
// //printf("= %g\n", eval($2)); //evaluate and print the AST
// //printf("> ");
// }
// | line EOL { printf("> "); } /* blank line or a comment */
// ;
exp: NUMBER { $$ = new RawAST(NUMBER_NODE, $1); }
| exp ADD exp { $$ = new RawAST(ADDITION_NODE, {$1, $3}); }
@@ -69,35 +63,41 @@ exp: NUMBER { $$ = new RawAST(NUMBER_NODE, $1); }
| exp MUL exp { $$ = new RawAST(MULTIPLY_NODE, {$1, $3}); }
| exp DIV exp { $$ = new RawAST(DIVIDE_NODE, {$1, $3}); }
| OP exp CP { $$ = $2; }
| O_SQ exp COLON exp C_SQ { $$ = new RawAST(RANGE_NODE, {$2, $4}); }
| NAME { $$ = new RawAST(REFERENCE_NODE, $1);
free($1);
}
| NAME EQUALS exp { $$ = new RawAST(REFERENCE_SET, $1, $3);
free($1);
}
| NAME EQUALS exp CM exp { $$ = new RawAST(REFERENCE_SET, $1, $3, $5);
free($1);
}
| REDUCE OP exp CM NAME CP { $$ = new RawAST(REDUCE_NODE, $3, $5);
| REDUCE OP exp CM NAME CP {
$$ = new RawAST(REDUCE_NODE, $3, $5, NULL);
free($5);
}
| REDUCE OP exp CM NAME CM exp CP { $$ = new RawAST(REDUCE_NODE, $3, $5, $7);
| REDUCE OP exp CM NAME CM O_SQ reduce_dim_args C_SQ CP {
$$ = new RawAST(REDUCE_NODE, $3, $5, $8);
free($5);
}
| SELECT OP exp CM NAME CP { $$ = new RawAST(SELECT_NODE, $3, $5);
free($5);
| SELECT OP exp CM O_SQ select_dim_args C_SQ CP {
$$ = new RawAST(SELECT_NODE, $3, $6);
}
| SELECT OP exp CM NAME CM exp CP { $$ = new RawAST(SELECT_NODE, $3, $5, $7);
free($5);
}
// | NAME O_SQ POS_INTEGER C_SQ { $$ = create_index_access_node($1, $3); }
;
%%
// void yyerror(char const *s)
// {
// fprintf(stderr, "check error saurabh: %s\n", s);
// }
reduce_dim_args: NAME { $$ = new LinkedList($1, NULL);
free($1);
}
| NAME CM reduce_dim_args { $$ = new LinkedList($1, $3);
free($1);
}
;
select_dim_args: NAME EQUALS NUMBER { $$ = new LinkedList($1, $3, NULL);
free($1);
}
| NAME EQUALS NUMBER CM select_dim_args { $$ = new LinkedList($1, $3, $5);
free($1);
}
;
%%
projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/raw_ast.hpp
+125 -51
View File
@@ -3,6 +3,8 @@
#include <map>
#include <optional>
#include <string>
#include <unordered_map>
#include <unordered_set>
#include <utility>
#include <variant>
#include <vector>
@@ -11,6 +13,9 @@
#include <fmt/ranges.h>
#include <glog/logging.h>
#include "lib/common/utility.hpp"
#include "lib/rocprofiler/counters/id_decode.hpp"
namespace rocprofiler
{
namespace counters
@@ -25,16 +30,31 @@ enum NodeType
RANGE_NODE,
REDUCE_NODE,
REFERENCE_NODE,
REFERENCE_SET,
SELECT_NODE,
SUBTRACTION_NODE,
};
struct LinkedList
{
std::string name;
int data{-1};
LinkedList* next{nullptr};
LinkedList(const char* v, LinkedList* next_node)
: name(std::string{CHECK_NOTNULL(v)})
, next(next_node)
{}
LinkedList(const char* v, int d, LinkedList* next_node)
: name(std::string{CHECK_NOTNULL(v)})
, data(d)
, next(next_node)
{}
};
struct RawAST
{
// Node type
NodeType type{NONE}; // Operation to perform on the counter set
NodeType operation{NONE};
NodeType type{NONE}; // Operation to perform on the counter set
std::string reduce_op;
// Stores either the name or digit dependening on whether this
// is a name or number
@@ -44,10 +64,12 @@ struct RawAST
// Operation is applied to all counters in this set.
std::vector<RawAST*> counter_set;
// Reference set to remove dimensions (such as shader)
// from the result. This is a future looking change and
// will be unsupported in 6.0.
std::vector<RawAST*> reference_set;
// Dimension set to remove dimensions (such as shader engine)
// from the result.
std::unordered_set<rocprofiler_profile_counter_instance_types> reduce_dimension_set;
// Dimension set to select certain dimensions from the result
std::unordered_map<rocprofiler_profile_counter_instance_types, int> select_dimension_set;
// Range restriction on this node
RawAST* range{nullptr};
@@ -61,7 +83,6 @@ struct RawAST
}
};
deleteVec(reference_set);
deleteVec(counter_set);
delete range;
}
@@ -77,47 +98,86 @@ struct RawAST
, value(v)
{}
// Reduce/Select operation constructor. Counter is the counter AST
// to use for the reduce/select op, op is how to reduce (i.e. SUM,AVG,etc),
// refs is the reference set AST. This reference set is copied to flatten
// the AST.
RawAST(NodeType t, RawAST* counter, const char* op, RawAST* refs = nullptr)
static const auto& get_dim_map()
{
static const auto dim_map = []() {
std::map<std::string, rocprofiler_profile_counter_instance_types> out;
const auto& dims = dimension_map();
for(const auto& [id, name] : dims)
{
out.emplace(name, id);
}
return out;
}();
return dim_map;
}
// Reduce operation constructor. Counter is the counter AST
// to use for the reduce op, op is how to reduce (i.e. SUM,AVG,etc),
// dimensions contains the set of dimensions which we want to keep
// in the result. Dimensions not specified are all reduced according to op
RawAST(NodeType t, RawAST* counter, const char* op, LinkedList* dimensions)
: type(t)
, value(std::string{CHECK_NOTNULL(op)})
, reduce_op(CHECK_NOTNULL(op))
, counter_set({counter})
{
copy_reference_set(refs);
if(dimensions)
{
while(dimensions)
{
const rocprofiler_profile_counter_instance_types* dim =
rocprofiler::common::get_val(get_dim_map(), std::string{dimensions->name});
if(!dim)
{
throw std::runtime_error(
fmt::format("Unknown Dimension - {}", dimensions->name));
}
reduce_dimension_set.insert(*dim);
LinkedList* current = dimensions;
dimensions = dimensions->next;
delete current;
}
}
}
// Select operation constructor. Counter is the counter AST
// to use for the reduce op, refs is the reference set AST.
// dimensions contains the mapping for selecting dimensions
// (XCC=1,SE=2,...)
RawAST(NodeType t, RawAST* counter, LinkedList* dimensions)
: type(t)
, counter_set({counter})
{
if(dimensions)
{
LinkedList* ptr = dimensions;
while(ptr)
{
const rocprofiler_profile_counter_instance_types* dim =
rocprofiler::common::get_val(get_dim_map(), dimensions->name);
if(!dim)
{
throw std::runtime_error(
fmt::format("Unknown Dimension - {}", dimensions->name));
}
select_dimension_set.insert({*dim, ptr->data});
LinkedList* current = ptr;
ptr = ptr->next;
delete current;
}
}
else
{
LOG(ERROR) << "select_dimension_set creation failed.";
}
}
RawAST(NodeType t, std::vector<RawAST*> c)
: type(t)
, counter_set(std::move(c))
{}
// Following two calls are for future reference set settings
// for select/reduce ops.
// Referene set constructor, refs is a pointer to an existing
// reference set when multiple references are given (i.e.
// shader=X,anotherRef=Y,....).
RawAST(NodeType t, const char* v, RawAST* r, RawAST* refs = nullptr)
: type(t)
, value(std::string{CHECK_NOTNULL(v)})
, range(r)
{
LOG(ERROR) << "BUilding bad ast";
copy_reference_set(refs);
}
// Flattens reference set tree into this node.
void copy_reference_set(RawAST* ast)
{
if(!ast) return;
reference_set.push_back(ast);
reference_set.insert(
reference_set.end(), ast->reference_set.begin(), ast->reference_set.end());
ast->reference_set.clear();
}
};
} // namespace counters
} // namespace rocprofiler
@@ -146,15 +206,14 @@ struct formatter<rocprofiler::counters::RawAST>
{rocprofiler::counters::RANGE_NODE, "RANGE_NODE"},
{rocprofiler::counters::REDUCE_NODE, "REDUCE_NODE"},
{rocprofiler::counters::REFERENCE_NODE, "REFERENCE_NODE"},
{rocprofiler::counters::REFERENCE_SET, "REFERENCE_SET"},
{rocprofiler::counters::SELECT_NODE, "SELECT_NODE"},
{rocprofiler::counters::SUBTRACTION_NODE, "SUBTRACTION_NODE"},
};
auto out = fmt::format_to(ctx.out(),
"{{\"Type\":\"{}\", \"Operation\":\"{}\",",
"{{\"Type\":\"{}\", \"REDUCE_OP\":\"{}\",",
NodeTypeToString.at(ast.type),
NodeTypeToString.at(ast.operation));
ast.reduce_op);
if(const auto* string_val = std::get_if<std::string>(&ast.value))
{
@@ -170,19 +229,34 @@ struct formatter<rocprofiler::counters::RawAST>
out = fmt::format_to(out, " \"Range\":{},", *ast.range);
}
out = fmt::format_to(out, "\"ReferenceSet\":[");
for(const auto& ref : ast.reference_set)
{
out = fmt::format_to(
out, "{}{}", *CHECK_NOTNULL(ref), ref == ast.reference_set.back() ? "" : ",");
}
out = fmt::format_to(out, "], \"CounterSet\":[");
out = fmt::format_to(out, " \"Counter_Set\":[");
for(const auto& ref : ast.counter_set)
{
out = fmt::format_to(
out, "{}{}", *CHECK_NOTNULL(ref), ref == ast.counter_set.back() ? "" : ",");
}
out = fmt::format_to(out, "], \"Reduce_Dimension_Set\":[");
size_t ReduceSetIndex = 0;
for(const auto& ref : ast.reduce_dimension_set)
{
out = fmt::format_to(out,
"\"{}\"{}",
static_cast<int>(ref),
++ReduceSetIndex == ast.reduce_dimension_set.size() ? "" : ",");
}
out = fmt::format_to(out, "], \"Select_Dimension_Set\":[");
size_t SelectSetIndex = 0;
for(const auto& [type, val] : ast.select_dimension_set)
{
out = fmt::format_to(out,
"\"{},{}\"{}",
static_cast<int>(type),
val,
++SelectSetIndex == ast.select_dimension_set.size() ? "" : ",");
}
return fmt::format_to(out, "]}}");
}
};
projects/rocprofiler-sdk/source/lib/rocprofiler/counters/parser/tests/parser_test.cpp
+129 -101
View File
@@ -1,5 +1,6 @@
#include <map>
#include <string>
#include <tuple>
#include <gtest/gtest.h>
@@ -9,26 +10,38 @@
TEST(parser, base_ops)
{
std::map<std::string, std::string> expressionToExpected = {
{"AB + BA",
"{\"Type\":\"ADDITION_NODE\", \"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"AB\",\"ReferenceSet\":[], \"CounterSet\":[]},{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"BA\",\"ReferenceSet\":[], \"CounterSet\":[]}]}"},
{"CD - ZX",
"{\"Type\":\"SUBTRACTION_NODE\", \"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"CD\",\"ReferenceSet\":[], \"CounterSet\":[]},{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"ZX\",\"ReferenceSet\":[], \"CounterSet\":[]}]}"},
{"NM / DB",
"{\"Type\":\"DIVIDE_NODE\", \"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"NM\",\"ReferenceSet\":[], \"CounterSet\":[]},{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"DB\",\"ReferenceSet\":[], \"CounterSet\":[]}]}"},
{"AB * BA",
"{\"Type\":\"MULTIPLY_NODE\", \"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"AB\",\"ReferenceSet\":[], \"CounterSet\":[]},{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"BA\",\"ReferenceSet\":[], \"CounterSet\":[]}]}"}};
"{\"Type\":\"MULTIPLY_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"AB\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]},{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", "
"\"Value\":\"BA\", \"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}"},
{"AB + BA",
"{\"Type\":\"ADDITION_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"AB\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]},{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", "
"\"Value\":\"BA\", \"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}"},
{"CD - ZX",
"{\"Type\":\"SUBTRACTION_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"CD\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]},{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", "
"\"Value\":\"ZX\", \"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}"},
{"NM / DB",
"{\"Type\":\"DIVIDE_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"NM\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]},{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", "
"\"Value\":\"DB\", \"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}"}};
for(auto [op, expected] : expressionToExpected)
{
@@ -46,37 +59,53 @@ TEST(parser, order_of_ops)
{
std::map<std::string, std::string> expressionToExpected = {
{"(AB + BA) / CD",
"{\"Type\":\"DIVIDE_NODE\", \"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"ADDITION_NODE\", \"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"AB\",\"ReferenceSet\":[], \"CounterSet\":[]},{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"BA\",\"ReferenceSet\":[], "
"\"CounterSet\":[]}]},{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"CD\",\"ReferenceSet\":[], \"CounterSet\":[]}]}"},
{"AD / (CD - ZX)",
"{\"Type\":\"DIVIDE_NODE\", \"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"AD\",\"ReferenceSet\":[], \"CounterSet\":[]},{\"Type\":\"SUBTRACTION_NODE\", "
"\"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"CD\",\"ReferenceSet\":[], \"CounterSet\":[]},{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"ZX\",\"ReferenceSet\":[], \"CounterSet\":[]}]}]}"},
{"MN * (NM / DB)",
"{\"Type\":\"MULTIPLY_NODE\", \"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"MN\",\"ReferenceSet\":[], \"CounterSet\":[]},{\"Type\":\"DIVIDE_NODE\", "
"\"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"NM\",\"ReferenceSet\":[], \"CounterSet\":[]},{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"DB\",\"ReferenceSet\":[], \"CounterSet\":[]}]}]}"},
"{\"Type\":\"DIVIDE_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"ADDITION_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"AB\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]},{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", "
"\"Value\":\"BA\", \"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]},{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", "
"\"Value\":\"CD\", \"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}"},
{"(AB / BA) - BN",
"{\"Type\":\"SUBTRACTION_NODE\", \"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"DIVIDE_NODE\", \"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"AB\",\"ReferenceSet\":[], \"CounterSet\":[]},{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"BA\",\"ReferenceSet\":[], "
"\"CounterSet\":[]}]},{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"BN\",\"ReferenceSet\":[], \"CounterSet\":[]}]}"}};
"{\"Type\":\"SUBTRACTION_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"DIVIDE_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"AB\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]},{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", "
"\"Value\":\"BA\", \"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]},{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", "
"\"Value\":\"BN\", \"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}"},
{"AD / (CD - ZX)",
"{\"Type\":\"DIVIDE_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"AD\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]},{\"Type\":\"SUBTRACTION_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"CD\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]},{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", "
"\"Value\":\"ZX\", \"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}"},
{"MN * (NM / DB)",
"{\"Type\":\"MULTIPLY_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"MN\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]},{\"Type\":\"DIVIDE_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"NM\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]},{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", "
"\"Value\":\"DB\", \"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}"}};
for(auto [op, expected] : expressionToExpected)
{
@@ -92,32 +121,34 @@ TEST(parser, order_of_ops)
TEST(parser, reduction)
{
std::map<std::string, std::string> expressionToExpected = {
{"reduce(AB, SUM)",
"{\"Type\":\"REDUCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"SUM\",\"ReferenceSet\":[], \"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"AB\",\"ReferenceSet\":[], \"CounterSet\":[]}]}"},
{"reduce(AB+CD, SUM)",
"{\"Type\":\"REDUCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"SUM\",\"ReferenceSet\":[], \"CounterSet\":[{\"Type\":\"ADDITION_NODE\", "
"\"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"AB\",\"ReferenceSet\":[], \"CounterSet\":[]},{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"CD\",\"ReferenceSet\":[], \"CounterSet\":[]}]}]}"},
{"reduce(AB,DIV)+reduce(DC,SUM)",
"{\"Type\":\"ADDITION_NODE\", \"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"REDUCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"DIV\",\"ReferenceSet\":[], \"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"AB\",\"ReferenceSet\":[], "
"\"CounterSet\":[]}]},{\"Type\":\"REDUCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"SUM\",\"ReferenceSet\":[], \"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"DC\",\"ReferenceSet\":[], \"CounterSet\":[]}]}]}"},
{"reduce(AB, SUM, shader)",
"{\"Type\":\"REDUCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"SUM\",\"ReferenceSet\":[{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"shader\",\"ReferenceSet\":[], \"CounterSet\":[]}], "
"\"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"AB\",\"ReferenceSet\":[], \"CounterSet\":[]}]}"}};
std::vector<std::tuple<std::string, std::string>> expressionToExpected = {
{"reduce(AB, SUM, [DIMENSION_XCC,DIMENSION_SHADER_ENGINE])",
"{\"Type\":\"REDUCE_NODE\", \"REDUCE_OP\":\"SUM\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"AB\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], \"Select_Dimension_Set\":[]}], "
"\"Reduce_Dimension_Set\":[\"2\",\"1\"], \"Select_Dimension_Set\":[]}"},
{"reduce(AB+CD, SUM, [DIMENSION_XCC,DIMENSION_SHADER_ENGINE])",
"{\"Type\":\"REDUCE_NODE\", \"REDUCE_OP\":\"SUM\", "
"\"Counter_Set\":[{\"Type\":\"ADDITION_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"AB\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]},{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", "
"\"Value\":\"CD\", \"Counter_Set\":[], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}], \"Reduce_Dimension_Set\":[], "
"\"Select_Dimension_Set\":[]}], \"Reduce_Dimension_Set\":[\"2\",\"1\"], "
"\"Select_Dimension_Set\":[]}"},
{"reduce(AB,DIV, [DIMENSION_XCC,DIMENSION_SHADER_ENGINE])+reduce(DC,SUM, "
"[DIMENSION_XCC,DIMENSION_SHADER_ENGINE])",
"{\"Type\":\"ADDITION_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REDUCE_NODE\", \"REDUCE_OP\":\"DIV\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"AB\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], \"Select_Dimension_Set\":[]}], "
"\"Reduce_Dimension_Set\":[\"2\",\"1\"], "
"\"Select_Dimension_Set\":[]},{\"Type\":\"REDUCE_NODE\", \"REDUCE_OP\":\"SUM\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"DC\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], \"Select_Dimension_Set\":[]}], "
"\"Reduce_Dimension_Set\":[\"2\",\"1\"], \"Select_Dimension_Set\":[]}], "
"\"Reduce_Dimension_Set\":[], \"Select_Dimension_Set\":[]}"}};
for(auto [op, expected] : expressionToExpected)
{
@@ -131,34 +162,31 @@ TEST(parser, reduction)
}
}
TEST(parser, selection)
TEST(parser, DISABLED_selection)
{
std::map<std::string, std::string> expressionToExpected = {
{"select(AB, SUM)",
"{\"Type\":\"SELECT_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"SUM\",\"ReferenceSet\":[], \"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"AB\",\"ReferenceSet\":[], \"CounterSet\":[]}]}"},
{"select(AB+CD, SUM)",
"{\"Type\":\"SELECT_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"SUM\",\"ReferenceSet\":[], \"CounterSet\":[{\"Type\":\"ADDITION_NODE\", "
"\"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"AB\",\"ReferenceSet\":[], \"CounterSet\":[]},{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"CD\",\"ReferenceSet\":[], \"CounterSet\":[]}]}]}"},
{"select(AB,DIV)+select(DC,SUM)",
"{\"Type\":\"ADDITION_NODE\", \"Operation\":\"NONE\",\"ReferenceSet\":[], "
"\"CounterSet\":[{\"Type\":\"SELECT_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"DIV\",\"ReferenceSet\":[], \"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"AB\",\"ReferenceSet\":[], "
"\"CounterSet\":[]}]},{\"Type\":\"SELECT_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"SUM\",\"ReferenceSet\":[], \"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"DC\",\"ReferenceSet\":[], \"CounterSet\":[]}]}]}"},
{"select(AB, SUM, shader)",
"{\"Type\":\"SELECT_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"SUM\",\"ReferenceSet\":[{\"Type\":\"REFERENCE_NODE\", "
"\"Operation\":\"NONE\", \"Value\":\"shader\",\"ReferenceSet\":[], \"CounterSet\":[]}], "
"\"CounterSet\":[{\"Type\":\"REFERENCE_NODE\", \"Operation\":\"NONE\", "
"\"Value\":\"AB\",\"ReferenceSet\":[], \"CounterSet\":[]}]}"}};
{"select(AB, [SE=1,XCC=0])+select(DC,[SE=2])",
"{\"Type\":\"ADDITION_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"SELECT_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"AB\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], \"Select_Dimension_Set\":[]}], "
"\"Reduce_Dimension_Set\":[], \"Select_Dimension_Set\":[\"(\"XCC\", 0)\",\"(\"SE\", "
"1)\"]},{\"Type\":\"SELECT_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"DC\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], \"Select_Dimension_Set\":[]}], "
"\"Reduce_Dimension_Set\":[], \"Select_Dimension_Set\":[\"(\"SE\", 2)\"]}], "
"\"Reduce_Dimension_Set\":[], \"Select_Dimension_Set\":[]}"},
{"select(AB, [SE=2,XCC=1,WGP=3])",
"{\"Type\":\"SELECT_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"AB\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], \"Select_Dimension_Set\":[]}], "
"\"Reduce_Dimension_Set\":[], \"Select_Dimension_Set\":[\"(\"WGP\", 3)\",\"(\"XCC\", "
"1)\",\"(\"SE\", 2)\"]}"},
{"select(AB, [XCC=0])",
"{\"Type\":\"SELECT_NODE\", \"REDUCE_OP\":\"\", "
"\"Counter_Set\":[{\"Type\":\"REFERENCE_NODE\", \"REDUCE_OP\":\"\", \"Value\":\"AB\", "
"\"Counter_Set\":[], \"Reduce_Dimension_Set\":[], \"Select_Dimension_Set\":[]}], "
"\"Reduce_Dimension_Set\":[], \"Select_Dimension_Set\":[\"(\"XCC\", 0)\"]}"}};
for(auto [op, expected] : expressionToExpected)
{