AI/rocm-systems
2
0
Derivar 0
Código Questões Pedidos de integração Operações Pacotes Planeamentos Lançamentos Wiki Trabalho

adding perfetto-validation-script (#66)

Ver a proveniência 
* adding perfetto-validation-script

* Rename validation script

* Update tests/validate_perfetto.py

Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>

* Update tests/validate_perfetto.py

Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>

* Update tests/validate_perfetto.py

Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>

* Update tests/validate_perfetto.py

Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>

* addressed the edits in the validation script

* addressed the edits in the validation script

* Perfetto validation script (#1)

* Fixed mismatch message in validate-timemory-json

* validate_perfetto.py -> validate-perfetto-proto.py

* Add python-source-validate-perfetto

- python-source-validate-perfetto uses validate-python-proto.py to validate perfetto output
- renamed python-source-check test to python-source-validate timemory

* Moved python-source-validate tests outside of cat command if block

- these tests don't rely on OMNITRACE_CAT_COMMAND

* CMake/CTest OMNITRACE_ADD_PYTHON_VALIDATION_TEST function

- generalized function for performing validation test with validate-{timemory-json,perfetto-proto}.py scripts

* Print perfetto validation

* Install perfetto python package in workflows

* cmake format

* Python formatting

* Python formatting CI

* Install perfetto python package in workflows

* Install dataclasses for perfetto in opensuse

* Install dataclasses for perfetto in ubuntu

- uninstalled dependency for perfetto in Python 3.6

Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>
Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
Este cometimento está contido em:
Tamima Rashid
2022-06-24 03:03:46 -05:00
cometido por GitHub
ascendente be7b03e76f
cometimento cfa16cbe2f
16 ficheiros modificados com 251 adições e 59 eliminações
Descarregar ficheiro patch Descarregar ficheiro diff Expandir todos os ficheiros Colapsar todos os ficheiros
.cmake-format.yaml
+7
Ver ficheiro
@@ -84,6 +84,13 @@ parse:
PASS_REGEX: '*'
FAIL_REGEX: '*'
SKIP_REGEX: '*'
omnitrace_add_python_validation_test:
kwargs:
NAME: '*'
ARGS: '*'
PERFETTO_FILE: '*'
TIMEMORY_FILE: '*'
TIMEMORY_METRIC: '*'
rocm_version_message:
flags:
- STATUS
.github/workflows/formatting.yml
+21
Ver ficheiro
@@ -10,6 +10,27 @@ on:
branches: [ main, develop ]
jobs:
python-formatting:
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: [3.8]
steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install black
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: black format
run: |
black --diff --check .
cmake-format:
runs-on: ubuntu-20.04
.github/workflows/opensuse.yml
+1 -1
Ver ficheiro
@@ -27,7 +27,7 @@ jobs:
- name: Install Packages
timeout-minutes: 5
run:
for i in 6 7 8 9 10; do /opt/conda/envs/py3.${i}/bin/python -m pip install numpy; done
for i in 6 7 8 9 10; do /opt/conda/envs/py3.${i}/bin/python -m pip install numpy perfetto dataclasses; done
- name: Configure Env
run:
.github/workflows/ubuntu-bionic.yml
+2 -1
Ver ficheiro
@@ -46,8 +46,9 @@ jobs:
apt-get install -y build-essential m4 autoconf libtool python3-pip ${{ matrix.compiler }} ${{ matrix.mpi }} &&
python3 -m pip install --upgrade pip &&
python3 -m pip install numpy &&
python3 -m pip install perfetto &&
python3 -m pip install 'cmake==3.16.3' &&
for i in 6 7 8 9; do /opt/conda/envs/py3.${i}/bin/python -m pip install numpy; done
for i in 6 7 8 9; do /opt/conda/envs/py3.${i}/bin/python -m pip install numpy perfetto dataclasses; done
- name: Install Kokkos
timeout-minutes: 5
.github/workflows/ubuntu-focal-external-rocm.yml
+1 -1
Ver ficheiro
@@ -37,7 +37,7 @@ jobs:
apt-get install -y build-essential m4 autoconf libtool python3-pip clang libomp-dev ${{ matrix.compiler }} libudev-dev libnuma-dev rocm-dev rocm-utils roctracer-dev rocprofiler-dev hip-base hsa-amd-aqlprofile hsa-rocr-dev hsakmt-roct-dev libpapi-dev libopenmpi-dev curl &&
python3 -m pip install --upgrade pip &&
python3 -m pip install 'cmake==3.16.3' &&
for i in 6 7 8 9; do /opt/conda/envs/py3.${i}/bin/python -m pip install numpy; done
for i in 6 7 8 9; do /opt/conda/envs/py3.${i}/bin/python -m pip install numpy perfetto dataclasses; done
- name: Configure Env
run:
.github/workflows/ubuntu-focal-external.yml
+2 -1
Ver ficheiro
@@ -65,8 +65,9 @@ jobs:
apt-get install -y build-essential m4 autoconf libtool python3-pip libiberty-dev clang libomp-dev ${{ matrix.compiler }} &&
python3 -m pip install --upgrade pip &&
python3 -m pip install numpy &&
python3 -m pip install perfetto &&
python3 -m pip install 'cmake==3.16.3' &&
for i in 6 7 8 9; do /opt/conda/envs/py3.${i}/bin/python -m pip install numpy; done
for i in 6 7 8 9; do /opt/conda/envs/py3.${i}/bin/python -m pip install numpy perfetto dataclasses; done
- name: Configure Env
run:
.github/workflows/ubuntu-focal.yml
+1
Ver ficheiro
@@ -30,6 +30,7 @@ jobs:
sudo apt-get install -y build-essential m4 autoconf libtool python3-pip libtbb-dev libboost-{atomic,system,thread,date-time,filesystem,timer}-dev clang libomp-dev ${{ matrix.compiler }} ${{ matrix.mpi }} &&
python3 -m pip install --upgrade pip &&
python3 -m pip install numpy &&
python3 -m pip install perfetto &&
python3 -m pip install 'cmake==3.16.3'
- name: Configure Env
examples/python/builtin.py
+2 -6
Ver ficheiro
@@ -37,12 +37,8 @@ if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser()
parser.add_argument(
"-n", "--num-iterations", help="Number", type=int, default=3
)
parser.add_argument(
"-v", "--value", help="Starting value", type=int, default=20
)
parser.add_argument("-n", "--num-iterations", help="Number", type=int, default=3)
parser.add_argument("-v", "--value", help="Starting value", type=int, default=20)
args = parser.parse_args()
_prefix = os.path.basename(__file__)
examples/python/external.py
+2 -6
Ver ficheiro
@@ -36,12 +36,8 @@ if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser()
parser.add_argument(
"-n", "--num-iterations", help="Number", type=int, default=3
)
parser.add_argument(
"-v", "--value", help="Starting value", type=int, default=20
)
parser.add_argument("-n", "--num-iterations", help="Number", type=int, default=3)
parser.add_argument("-v", "--value", help="Starting value", type=int, default=20)
args = parser.parse_args()
_prefix = os.path.basename(__file__)
examples/python/source.py
+2 -6
Ver ficheiro
@@ -58,12 +58,8 @@ if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser()
parser.add_argument(
"-n", "--num-iterations", help="Number", type=int, default=3
)
parser.add_argument(
"-v", "--value", help="Starting value", type=int, default=20
)
parser.add_argument("-n", "--num-iterations", help="Number", type=int, default=3)
parser.add_argument("-v", "--value", help="Starting value", type=int, default=20)
args = parser.parse_args()
_prefix = os.path.basename(__file__)
pyproject.toml
+31
Ver ficheiro
@@ -0,0 +1,31 @@
[build-system]
requires = [
"setuptools >= 40.0.4",
"setuptools_scm >= 2.0.0",
"wheel >= 0.29.0",
]
build-backend = 'setuptools.build_meta'
[tool.black]
line-length = 90
target-version = ['py38']
include = '\.py'
exclude = '''
(
/(
\.eggs
| \.git
| \.github
| \.tox
| \.venv
| \.misc
| \.vscode
| dist
| external
| .pytest_cache
| build
| build-release
| build-omnitrace
)/
)
'''
source/docs/conf.py
+13 -13
Ver ficheiro
@@ -110,26 +110,26 @@ html_theme = "sphinx_rtd_theme"
html_static_path = ["_static"]
html_theme_options = {
'analytics_id': 'G-1HLBBRSTT9', # Provided by Google in your dashboard
'analytics_anonymize_ip': False,
'logo_only': False,
'display_version': True,
'prev_next_buttons_location': 'bottom',
'style_external_links': False,
'vcs_pageview_mode': '',
"analytics_id": "G-1HLBBRSTT9", # Provided by Google in your dashboard
"analytics_anonymize_ip": False,
"logo_only": False,
"display_version": True,
"prev_next_buttons_location": "bottom",
"style_external_links": False,
"vcs_pageview_mode": "",
# 'style_nav_header_background': 'white',
# Toc options
'collapse_navigation': True,
'sticky_navigation': True,
'navigation_depth': 4,
'includehidden': True,
'titles_only': False
"collapse_navigation": True,
"sticky_navigation": True,
"navigation_depth": 4,
"includehidden": True,
"titles_only": False,
}
# Breathe Configuration
breathe_projects = {"omnitrace": "_doxygen/xml"}
breathe_default_project = "omnitrace"
breathe_default_members = ('members', )
breathe_default_members = ("members",)
breathe_projects_source = {
"auto": (
os.path.join(project_root, "source"),
source/python/omnitrace/profiler.py
+2 -10
Ver ficheiro
@@ -295,11 +295,7 @@ class Profiler:
self.stop()
if (
exec_type is not None
and exec_value is not None
and exec_tb is not None
):
if exec_type is not None and exec_value is not None and exec_tb is not None:
import traceback
traceback.print_exception(exec_type, exec_value, exec_tb, limit=5)
@@ -390,9 +386,5 @@ class FakeProfiler:
import traceback
if (
exec_type is not None
and exec_value is not None
and exec_tb is not None
):
if exec_type is not None and exec_value is not None and exec_tb is not None:
traceback.print_exception(exec_type, exec_value, exec_tb, limit=5)
tests/CMakeLists.txt
+63 -13
Ver ficheiro
@@ -964,24 +964,74 @@ foreach(_VERSION ${OMNITRACE_PYTHON_VERSIONS})
PASS_REGEX "\\\[inefficient\\\]\\\[builtin.py:14\\\]"
DEPENDS python-builtin-${_VERSION}
ENVIRONMENT "${_python_environment}")
omnitrace_add_python_test(
NAME python-source-check
COMMAND
${_PYTHON_EXECUTABLE} ${CMAKE_CURRENT_LIST_DIR}/validate-timemory-json.py
-l run fib fib fib fib fib inefficient _sum -c 5 5 10 20 30 10 5 5 -d 0 1
2 3 4 5 1 2 -m trip_count -i
PYTHON_VERSION ${_VERSION}
FILE omnitrace-tests-output/python-source/${_VERSION}/trip_count.json
DEPENDS python-source-${_VERSION}
PASS_REGEX
"omnitrace-tests-output/python-source/${_VERSION}/trip_count.json validated"
ENVIRONMENT "${_python_environment}")
else()
omnitrace_message(
WARNING
"Neither 'cat' nor 'cmake -E cat' are available. Python source checks are disabled"
)
endif()
function(OMNITRACE_ADD_PYTHON_VALIDATION_TEST)
cmake_parse_arguments(TEST "" "NAME;TIMEMORY_METRIC;TIMEMORY_FILE;PERFETTO_FILE"
"ARGS" ${ARGN})
omnitrace_add_python_test(
NAME ${TEST_NAME}-validate-timemory
COMMAND
${_PYTHON_EXECUTABLE} ${CMAKE_CURRENT_LIST_DIR}/validate-timemory-json.py
-m ${TEST_TIMEMORY_METRIC} ${TEST_ARGS} -i
PYTHON_VERSION ${_VERSION}
FILE omnitrace-tests-output/${TEST_NAME}/${_VERSION}/${TEST_TIMEMORY_FILE}
DEPENDS ${TEST_NAME}-${_VERSION}
PASS_REGEX
"omnitrace-tests-output/${TEST_NAME}/${_VERSION}/${TEST_TIMEMORY_FILE} validated"
ENVIRONMENT "${_python_environment}")
omnitrace_add_python_test(
NAME ${TEST_NAME}-validate-perfetto
COMMAND
${_PYTHON_EXECUTABLE} ${CMAKE_CURRENT_LIST_DIR}/validate-perfetto-proto.py
${TEST_ARGS} -p -i
PYTHON_VERSION ${_VERSION}
FILE omnitrace-tests-output/${TEST_NAME}/${_VERSION}/${TEST_PERFETTO_FILE}
DEPENDS ${TEST_NAME}-${_VERSION}
PASS_REGEX
"omnitrace-tests-output/${TEST_NAME}/${_VERSION}/${TEST_PERFETTO_FILE} validated"
ENVIRONMENT "${_python_environment}")
endfunction()
omnitrace_add_python_validation_test(
NAME python-source
TIMEMORY_METRIC "trip_count"
TIMEMORY_FILE "trip_count.json"
PERFETTO_FILE "perfetto-trace.proto"
ARGS -l
run
fib
fib
fib
fib
fib
inefficient
_sum
-c
5
5
10
20
30
10
5
5
-d
0
1
2
3
4
5
1
2)
math(EXPR _INDEX "${_INDEX} + 1")
endforeach()
tests/validate-perfetto-proto.py
Ficheiro executável
+100
Ver ficheiro
@@ -0,0 +1,100 @@
#!/usr/bin/env python3
import sys
import argparse
from perfetto.trace_processor import TraceProcessor
def validate_perfetto(data, labels, counts, depths):
expected = []
for litr, citr, ditr in zip(labels, counts, depths):
entry = []
_label = litr
if ditr > 0:
_label = "{}".format(litr)
entry = [_label, citr, ditr]
expected.append(entry)
for ditr, eitr in zip(data, expected):
_label = ditr["label"]
_count = ditr["count"]
_depth = ditr["depth"]
if _label != eitr[0]:
raise RuntimeError(f"Mismatched prefix: {_label} vs. {eitr[0]}")
if _count != eitr[1]:
raise RuntimeError(f"Mismatched count: {_count} vs. {eitr[1]}")
if _depth != eitr[2]:
raise RuntimeError(f"Mismatched depth: {_depth} vs. {eitr[2]}")
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument(
"-l", "--labels", nargs="+", type=str, help="Expected labels", default=[]
)
parser.add_argument(
"-c", "--counts", nargs="+", type=int, help="Expected counts", default=[]
)
parser.add_argument(
"-d", "--depths", nargs="+", type=int, help="Expected depths", default=[]
)
parser.add_argument(
"-p", "--print", action="store_true", help="Print the processed perfetto data"
)
parser.add_argument("-i", "--input", type=str, help="Input file", required=True)
args = parser.parse_args()
if len(args.labels) != len(args.counts) or len(args.labels) != len(args.depths):
raise RuntimeError(
"The same number of labels, counts, and depths must be specified"
)
tp = TraceProcessor(trace=(args.input))
pdata = {}
# get data from perfetto
qr_it = tp.query("SELECT name, depth FROM slice")
# loop over data rows from perfetto
for row in qr_it:
if row.name not in pdata:
pdata[row.name] = {}
if row.depth not in pdata[row.name]:
pdata[row.name][row.depth] = 0
# accumulate the call-count per name and per depth
pdata[row.name][row.depth] += 1
perfetto_data = []
for name, itr in pdata.items():
for depth, count in itr.items():
_e = {}
_e["label"] = name
_e["count"] = count
_e["depth"] = depth
perfetto_data.append(_e)
# demo display of data
if args.print:
for itr in perfetto_data:
n = 0 if itr["depth"] < 2 else itr["depth"] - 1
lbl = "{}{}{}".format(
" " * n, "|_" if itr["depth"] > 0 else "", itr["label"]
)
print("| {:40} | {:6} | {:6} |".format(lbl, itr["count"], itr["depth"]))
ret = 0
try:
validate_perfetto(
perfetto_data,
args.labels,
args.counts,
args.depths,
)
except RuntimeError as e:
print(f"{e}")
ret = 1
if ret == 0:
print(f"{args.input} validated")
sys.exit(ret)
tests/validate-timemory-json.py
+1 -1
Ver ficheiro
@@ -25,7 +25,7 @@ def validate_json(data, labels, counts, depths):
if _prefix != eitr[0]:
raise RuntimeError(f"Mismatched prefix: {_prefix} vs. {eitr[0]}")
if _count != eitr[1]:
raise RuntimeError(f"Mismatched depth: {_depth} vs. {eitr[2]}")
raise RuntimeError(f"Mismatched count: {_count} vs. {eitr[1]}")
if _depth != eitr[2]:
raise RuntimeError(f"Mismatched depth: {_depth} vs. {eitr[2]}")