AI/rocm-systems
2
0
Форкнуть 0
Код Задачи Запросы на слияние Действия Пакеты Проекты Релизы Вики Активность

[rocprofv3] SQLite3 database output (rocpd) support + rocprofiler-sdk-rocpd (#403)

Просмотр исходного кода 
* [rocprofv3] rocpd SQLite3 database output support

* Move counters xml and yaml to source/share/rocprofiler-sdk

- more representative of install hierarchy

* Add share/rocprofiler-sdk/rocpd SQL files

* Experimental rocprofiler-sdk SQL API

* rocprofv3 default output format is rocpd

* Fix rocpd event ids for counter collection w/o kernel dispatch

* Remove fktable entries from rocpd_tables.sql

* Fix rocpd schema path

* Fix install component for roctx python bindings

* rocprofiler-sdk-rocpd

- create include/rocprofiler-sdk-rocpd
- create rocprofiler-sdk-rocpd library, package, etc.
- default all "guid" fields to "{{guid}}" in tables
- remove "{{view_uuid}}" support (always unused)

* Migrate rocprofv3 to use rocprofiler-sdk-rocpd

* Fix missing foreign key reference

* Revert change

* Fix cmake comment

* Fix maybe-uninitialized compiler warning

* Fix maybe-uninitialized compiler warning

* Add logging to rocpd_sql_load_schema

* Improve string sanitization when inserting json strings

* Initialize rocpd logging on rocprofiler-sdk-rocpd library load

* Revert lib/output/generatePerfetto.cpp changes

* [temporary] Tweak rocprofv3-test-list-avail-trace-execute test log level

* Update get_install_path for lib/rocprofiler-sdk-rocpd/sql.cpp

- try to resolve issues on RHEL/SLES for dladdr

* Update lib/common/logging.cpp

- enable environ overrides

* dlsym for rocpd_sql_load_schema

* Make dl_info.dli_fname lexically normal

* Implement node_info alternatives if /etc/machine-id does not exist

* Misc include fixes

* SHA256 and UUIDv7 support

* Implement UUIDv7 in generateRocpd.cpp

* Support push/pop environment variables

* Minor tweak

* Fix glog segfaults when unsetting glog env

* Updated CHANGELOG

* Updates tests/pytest-packages

- rocpd_reader.py: RocpdReader

* Update tests / marker_views.sql

- add test_rocpd_data

* Update rocpd_tables.sql

- Use AUTOINCREMENT
- insert "uuid" and "guid" into rocpd_metadata

* Minor updates to generateRocpd.cpp

- don't quote GUID
- use sqlite3_open_v2
- use sqlite3_close_v2

* Update execute_raw_sql_statements_impl

- uses sqlite3_last_insert_rowid for autoincrement

* Update SQL deferred_transaction

- CI check for nullptr to connection

* Apply suggestions from code review

Co-authored-by: Welton, Benjamin <Benjamin.Welton@amd.com>

* Code review updates

- formatting
- replace if with switch
- remove loop for {{uuid}}

* Fix pmc_groups handling in rocprofv3

* Address code review feedback

- Include rocm_version in rocprofv3 version info
- Note `--version` option for `rocprofv3` in CHANGELOG.md
- remove commented out code

* Fix packaging dependencies

* Fix install package step of CI workflow

* Fix install package step of CI workflow

---------

Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>
Co-authored-by: Welton, Benjamin <Benjamin.Welton@amd.com>
Этот коммит содержится в:
Madsen, Jonathan
2025-05-30 00:13:19 -05:00
коммит произвёл GitHub
родитель dbb2e52216
Коммит 7afedc63be
81 изменённых файлов: 7725 добавлений и 993 удалений
Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы
.github/workflows/continuous_integration.yml
+4 -2
Просмотреть файл
@@ -146,7 +146,8 @@ jobs:
ls -la
ls -la ./build
dpkg -i ./build/rocprofiler-sdk-roctx_*.deb
for i in $(ls -S ./build/rocprofiler-sdk*.deb | egrep -v roctx); do dpkg -i ${i}; done;
dpkg -i ./build/rocprofiler-sdk-rocpd_*.deb
for i in $(ls -S ./build/rocprofiler-sdk*.deb | egrep -v 'roctx|rocpd'); do dpkg -i ${i}; done;
- name: Test Installed Packages
if: ${{ contains(matrix.runner, env.CORE_EXT_RUNNER) }}
@@ -271,7 +272,8 @@ jobs:
ls -la
ls -la ./build
dpkg -i ./build/rocprofiler-sdk-roctx_*.deb
for i in $(ls -S ./build/rocprofiler-sdk*.deb | egrep -v roctx); do dpkg -i ${i}; done;
dpkg -i ./build/rocprofiler-sdk-rocpd_*.deb
for i in $(ls -S ./build/rocprofiler-sdk*.deb | egrep -v 'roctx|rocpd'); do dpkg -i ${i}; done;
- name: Test Installed Packages
if: ${{ contains(matrix.runner, env.CORE_EXT_RUNNER) }}
CHANGELOG.md
+8
Просмотреть файл
@@ -176,12 +176,20 @@ Full documentation for ROCprofiler-SDK is available at [rocm.docs.amd.com/projec
- type-relative == logical_node_type_id
- Added MI300 stochastic (hardware-based) PC sampling support in ROCProfiler-SDK and ROCProfV3
- Python bindings for rocprofiler-sdk-roctx
- SQLite3 output support for rocprofv3 (`--output-format rocpd`)
- Added `rocprofiler-sdk-rocpd` package
- public API in `include/rocprofiler-sdk-rocpd/rocpd.h`
- library implementation in `librocprofiler-sdk-rocpd.so`
- support for `find_package(rocprofiler-sdk-rocpd)`
- `rocprofiler-sdk-rocpd` DEB and RPM packages
- Support `--version` option for `rocprofv3`
### Changed
- SDK no longer creates a background thread when every tool returns a nullptr from `rocprofiler_configure`.
- Updated disassembly.hpp's vaddr-to-file-offset mapping to use the dedicated comgr API.
- rocprofv3 shorthand argument for `--collection-period` is now `-P` (upper-case) as `-p` (lower-case) is reserved for later use
- default output format for rocprofv3 is now `rocpd` (SQLite3 database)
### Resolved issues
CMakeLists.txt
+2 -1
Просмотреть файл
@@ -87,7 +87,7 @@ mark_as_advanced(CMAKE_INSTALL_RPATH_USE_LINK_PATH CMAKE_DEBUG_POSTFIX)
set(ROCPROFILER_INTERNAL_BUILD_DOCS
OFF
CACHE INTERNAL "Generates rocprofiler/version.h and exits (no build targets)")
CACHE INTERNAL "Generates rocprofiler-sdk/version.h and exits (no build targets)")
if(ROCPROFILER_INTERNAL_BUILD_DOCS)
add_subdirectory(source/include)
return()
@@ -114,6 +114,7 @@ add_subdirectory(source)
include(rocprofiler_config_install)
include(rocprofiler_config_install_roctx)
include(rocprofiler_config_install_rocpd)
if(ROCPROFILER_BUILD_TESTS)
add_subdirectory(tests)
cmake/Templates/rocprofiler-sdk-rocpd/build-config.cmake.in
+9
Просмотреть файл
@@ -0,0 +1,9 @@
# Config file for @PACKAGE_NAME@ and its component libraries in the build tree
#
list(APPEND @PACKAGE_NAME@_INCLUDE_DIR @CMAKE_BINARY_DIR@/source/include)
foreach(COMP @PROJECT_BUILD_TREE_TARGETS@)
list(APPEND @PACKAGE_NAME@_LIBRARIES ${COMP})
target_link_libraries(@PACKAGE_NAME@::@PACKAGE_NAME@ INTERFACE ${COMP})
endforeach()
cmake/Templates/rocprofiler-sdk-rocpd/config.cmake.in
+73
Просмотреть файл
@@ -0,0 +1,73 @@
# - Config file for @PACKAGE_NAME@ and its component libraries
# It defines the following variables:
#
# @PACKAGE_NAME@_VERSION
# @PACKAGE_NAME@_INCLUDE_DIR
# @PACKAGE_NAME@_LIB_DIR
# @PACKAGE_NAME@_LIBRARIES
#
# It provides the following interface libraries:
#
# @PACKAGE_NAME@::@PACKAGE_NAME@
#
# prevent "target already exists" error
include_guard(DIRECTORY)
# compute paths
get_filename_component(@PACKAGE_NAME@_CMAKE_DIR "${CMAKE_CURRENT_LIST_FILE}" PATH)
@PACKAGE_INIT@
set_and_check(@PACKAGE_NAME@_INCLUDE_DIR "@PACKAGE_INCLUDE_INSTALL_DIR@")
set_and_check(@PACKAGE_NAME@_LIB_DIR "@PACKAGE_LIB_INSTALL_DIR@")
get_filename_component(@PACKAGE_NAME@_ROOT_DIR ${@PACKAGE_NAME@_INCLUDE_DIR} PATH)
set_and_check(@PACKAGE_NAME@_ROOT "${@PACKAGE_NAME@_ROOT_DIR}")
# extra validation
foreach(_@PACKAGE_NAME@_SUBDIR @PROJECT_EXTRA_DIRS@)
set_and_check(_@PACKAGE_NAME@_SUBDIR_CHECK
"${PACKAGE_PREFIX_DIR}/${_@PACKAGE_NAME@_SUBDIR}")
unset(_@PACKAGE_NAME@_SUBDIR_CHECK)
endforeach()
set(@PACKAGE_NAME@_LIBRARIES)
# add interface library
add_library(@PACKAGE_NAME@::@PACKAGE_NAME@ INTERFACE IMPORTED)
if(@PACKAGE_NAME@_BUILD_TREE
AND EXISTS "${CMAKE_CURRENT_LIST_DIR}/@PACKAGE_NAME@-build-config.cmake")
include("${CMAKE_CURRENT_LIST_DIR}/@PACKAGE_NAME@-build-config.cmake")
else()
include("${@PACKAGE_NAME@_CMAKE_DIR}/@PACKAGE_NAME@-targets.cmake")
# Library dependencies
if(@PACKAGE_NAME@_FIND_COMPONENTS)
foreach(COMP ${@PACKAGE_NAME@_FIND_COMPONENTS})
set(TARG @PACKAGE_NAME@::@PACKAGE_NAME@-${COMP})
if(TARGET ${TARG})
set(@PACKAGE_NAME@_${COMP}_FOUND 1)
list(APPEND @PACKAGE_NAME@_LIBRARIES ${TARG})
target_link_libraries(@PACKAGE_NAME@::@PACKAGE_NAME@ INTERFACE ${TARG})
else()
set(@PACKAGE_NAME@_${COMP}_FOUND 0)
endif()
endforeach()
else()
foreach(TARG @PROJECT_BUILD_TARGETS@)
set(TARG @PACKAGE_NAME@::${TARG})
list(APPEND @PACKAGE_NAME@_LIBRARIES ${TARG})
target_link_libraries(@PACKAGE_NAME@::@PACKAGE_NAME@ INTERFACE ${TARG})
endforeach()
endif()
endif()
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(
@PACKAGE_NAME@
FOUND_VAR @PACKAGE_NAME@_FOUND
VERSION_VAR @PACKAGE_NAME@_VERSION
REQUIRED_VARS @PACKAGE_NAME@_ROOT_DIR @PACKAGE_NAME@_INCLUDE_DIR
@PACKAGE_NAME@_LIB_DIR @PACKAGE_NAME@_LIBRARIES @PACKAGE_NAME@_VERSION
HANDLE_COMPONENTS HANDLE_VERSION_RANGE)
cmake/Templates/setup-env.sh.in
+23 -4
Просмотреть файл
@@ -8,16 +8,35 @@ if [ ! -d "${BASEDIR}" ]; then
return 1
fi
@PACKAGE_NAME_UNDERSCORED@_get_python3_path()
{
local PYTHON3_EXECUTABLE=$(command -v python3 2> /dev/null)
if [ -n "${PYTHON3_EXECUTABLE}" ]; then
local PYTHON3_VERSION=$(${PYTHON3_EXECUTABLE} -c "import sys; print(f'{sys.version_info[0]}.{sys.version_info[1]}')" 2> /dev/null)
local @PACKAGE_NAME_UNDERSCORED@_PYTHONPATH=${BASEDIR}/@CMAKE_INSTALL_LIBDIR@/python${PYTHON3_VERSION}/site-packages
if [ -d "${@PACKAGE_NAME_UNDERSCORED@_PYTHONPATH}" ]; then
echo ${BASEDIR}/@CMAKE_INSTALL_LIBDIR@/python${PYTHON3_VERSION}/site-packages
fi
fi
}
@PACKAGE_NAME_UNDERSCORED@_ROOT=${BASEDIR}
@PACKAGE_NAME_UNDERSCORED@_DIR=${BASEDIR}/@CMAKE_INSTALL_LIBDIR@/cmake/@PACKAGE_NAME@
PATH=${BASEDIR}/bin:${PATH}
LD_LIBRARY_PATH=${BASEDIR}/@CMAKE_INSTALL_LIBDIR@:${LD_LIBRARY_PATH}
PYTHONPATH=${BASEDIR}/@CMAKE_INSTALL_PYTHONDIR@:${PYTHONPATH}
CMAKE_PREFIX_PATH=${BASEDIR}:${CMAKE_PREFIX_PATH}
@PACKAGE_NAME_UNDERSCORED@_DIR=${BASEDIR}/@CMAKE_INSTALL_LIBDIR@/cmake/@PACKAGE_NAME@
if [ -z "@CMAKE_INSTALL_PYTHONDIR@" ]; then
PYTHONPATH=$(@PACKAGE_NAME_UNDERSCORED@_get_python3_path):${PYTHONPATH}
else
PYTHONPATH=${BASEDIR}/@CMAKE_INSTALL_PYTHONDIR@:${PYTHONPATH}
fi
unset @PACKAGE_NAME_UNDERSCORED@_get_python3_path
export @PACKAGE_NAME_UNDERSCORED@_ROOT
export @PACKAGE_NAME_UNDERSCORED@_DIR
export PATH
export LD_LIBRARY_PATH
export PYTHONPATH
export CMAKE_PREFIX_PATH
export @PACKAGE_NAME_UNDERSCORED@_DIR
export PYTHONPATH
cmake/rocprofiler_config_install_rocpd.cmake
+106
Просмотреть файл
@@ -0,0 +1,106 @@
# include guard
include_guard(GLOBAL)
include(CMakePackageConfigHelpers)
set(CMAKE_INSTALL_DEFAULT_COMPONENT_NAME rocpd)
set(SDK_PACKAGE_NAME "${PROJECT_NAME}")
set(PACKAGE_NAME "${PROJECT_NAME}-rocpd")
install(
EXPORT ${PACKAGE_NAME}-targets
FILE ${PACKAGE_NAME}-targets.cmake
NAMESPACE ${PACKAGE_NAME}::
DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/${PACKAGE_NAME}
COMPONENT rocpd)
rocprofiler_install_env_setup_files(
NAME ${PACKAGE_NAME}
VERSION ${PROJECT_VERSION}
INSTALL_DIR ${CMAKE_INSTALL_DATAROOTDIR}
COMPONENT rocpd)
# ------------------------------------------------------------------------------#
# install tree
#
set(PROJECT_INSTALL_DIR ${CMAKE_INSTALL_PREFIX})
set(INCLUDE_INSTALL_DIR ${CMAKE_INSTALL_INCLUDEDIR})
set(LIB_INSTALL_DIR ${CMAKE_INSTALL_LIBDIR})
set(PROJECT_BUILD_TARGETS ${PACKAGE_NAME}-shared-library)
set(PROJECT_EXTRA_DIRS "${CMAKE_INSTALL_INCLUDEDIR}/${PACKAGE_NAME}")
configure_package_config_file(
${PROJECT_SOURCE_DIR}/cmake/Templates/${PACKAGE_NAME}/config.cmake.in
${PROJECT_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}/cmake/${PACKAGE_NAME}/${PACKAGE_NAME}-config.cmake
INSTALL_DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/${PACKAGE_NAME}
INSTALL_PREFIX ${CMAKE_INSTALL_PREFIX}
PATH_VARS PROJECT_INSTALL_DIR INCLUDE_INSTALL_DIR LIB_INSTALL_DIR)
write_basic_package_version_file(
${PROJECT_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}/cmake/${PACKAGE_NAME}/${PACKAGE_NAME}-config-version.cmake
VERSION ${PROJECT_VERSION}
COMPATIBILITY SameMinorVersion)
install(
FILES
${PROJECT_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}/cmake/${PACKAGE_NAME}/${PACKAGE_NAME}-config.cmake
${PROJECT_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}/cmake/${PACKAGE_NAME}/${PACKAGE_NAME}-config-version.cmake
DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/${PACKAGE_NAME}
COMPONENT rocpd)
export(PACKAGE ${PACKAGE_NAME})
# ------------------------------------------------------------------------------#
# build tree
#
set(${PACKAGE_NAME}_BUILD_TREE
ON
CACHE BOOL "" FORCE)
set(PROJECT_BUILD_TREE_TARGETS ${SDK_PACKAGE_NAME}::${PACKAGE_NAME}-shared-library
${SDK_PACKAGE_NAME}::${SDK_PACKAGE_NAME}-stack-protector)
configure_file(
${PROJECT_SOURCE_DIR}/cmake/Templates/${PACKAGE_NAME}/build-config.cmake.in
${PROJECT_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}/cmake/${PACKAGE_NAME}/${PACKAGE_NAME}-build-config.cmake
@ONLY)
file(RELATIVE_PATH rocp_bin2src_rel_path ${PROJECT_BINARY_DIR} ${PROJECT_SOURCE_DIR})
string(REPLACE "//" "/" rocp_inc_rel_path "${rocp_bin2src_rel_path}/source/include")
set(_BUILDTREE_EXPORT_DIR
"${PROJECT_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}/cmake/${PACKAGE_NAME}")
execute_process(
COMMAND ${CMAKE_COMMAND} -E create_symlink ${rocp_inc_rel_path}
${PROJECT_BINARY_DIR}/include WORKING_DIRECTORY ${PROJECT_BINARY_DIR})
if(NOT EXISTS "${PROJECT_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}")
file(MAKE_DIRECTORY "${PROJECT_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}")
endif()
if(NOT EXISTS "${PROJECT_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}/${PACKAGE_NAME}")
file(MAKE_DIRECTORY "${PROJECT_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}/${PACKAGE_NAME}")
endif()
if(NOT EXISTS "${_BUILDTREE_EXPORT_DIR}")
file(MAKE_DIRECTORY "${_BUILDTREE_EXPORT_DIR}")
endif()
if(NOT EXISTS "${_BUILDTREE_EXPORT_DIR}/${PACKAGE_NAME}-targets.cmake")
file(TOUCH "${_BUILDTREE_EXPORT_DIR}/${PACKAGE_NAME}-targets.cmake")
endif()
export(
EXPORT ${PACKAGE_NAME}-targets
NAMESPACE ${PACKAGE_NAME}::
FILE "${_BUILDTREE_EXPORT_DIR}/${PACKAGE_NAME}-targets.cmake")
set(${PACKAGE_NAME}_DIR
"${_BUILDTREE_EXPORT_DIR}"
CACHE PATH "${PACKAGE_NAME} build tree install" FORCE)
install(
FILES ${PROJECT_SOURCE_DIR}/LICENSE
DESTINATION ${CMAKE_INSTALL_DATAROOTDIR}/doc/${PACKAGE_NAME}
COMPONENT rocpd)
cmake/rocprofiler_config_packaging.cmake
+13 -5
Просмотреть файл
@@ -58,13 +58,14 @@ list(REMOVE_ITEM ROCPROFILER_PACKAGING_COMPONENTS "Development" "Unspecified")
list(LENGTH ROCPROFILER_PACKAGING_COMPONENTS NUM_ROCPROFILER_PACKAGING_COMPONENTS)
# the packages we will generate
set(ROCPROFILER_COMPONENT_GROUPS "core" "docs" "tests" "roctx")
set(ROCPROFILER_COMPONENT_GROUPS "core" "docs" "tests" "roctx" "rocpd")
set(COMPONENT_GROUP_core_COMPONENTS "core" "development" "samples" "tools" "benchmark"
"Development" "Unspecified")
set(COMPONENT_GROUP_docs_COMPONENTS "docs")
set(COMPONENT_GROUP_tests_COMPONENTS "tests")
set(COMPONENT_GROUP_roctx_COMPONENTS "roctx")
set(COMPONENT_GROUP_rocpd_COMPONENTS "rocpd")
# variables for each component group. Note: eventually we will probably want to separate
# the core to just be the runtime libraries, development to be the headers and cmake
@@ -74,18 +75,25 @@ set(COMPONENT_NAME_core "rocprofiler-sdk")
set(COMPONENT_NAME_docs "rocprofiler-sdk-docs")
set(COMPONENT_NAME_tests "rocprofiler-sdk-tests")
set(COMPONENT_NAME_roctx "rocprofiler-sdk-roctx")
set(COMPONENT_NAME_rocpd "rocprofiler-sdk-rocpd")
set(COMPONENT_DEP_core "rocprofiler-sdk-roctx (>= ${PROJECT_VERSION})")
set(COMPONENT_DEP_core "rocprofiler-sdk-roctx (>= ${PROJECT_VERSION})"
"rocprofiler-sdk-rocpd (>= ${PROJECT_VERSION})")
set(COMPONENT_DEP_docs "")
set(COMPONENT_DEP_tests "rocprofiler-sdk (>= ${PROJECT_VERSION})")
set(COMPONENT_DEP_roctx "")
set(COMPONENT_DEP_tests
"rocprofiler-sdk (>= ${PROJECT_VERSION})"
"rocprofiler-sdk-roctx (>= ${PROJECT_VERSION})"
"rocprofiler-sdk-rocpd (>= ${PROJECT_VERSION})")
set(COMPONENT_DEP_roctx "rocprofiler-register")
set(COMPONENT_DEP_rocpd "")
set(COMPONENT_DESC_core "rocprofiler-sdk libraries, headers, samples, and tools")
set(COMPONENT_DESC_docs "rocprofiler-sdk documentation")
set(COMPONENT_DESC_tests "rocprofiler-sdk tests")
set(COMPONENT_DESC_roctx "ROCm Tools Extension library and headers")
set(COMPONENT_DESC_rocpd "ROCm Profiling Data library and headers")
set(EXPECTED_PACKAGING_COMPONENTS 6)
set(EXPECTED_PACKAGING_COMPONENTS 7)
if(ROCPROFILER_BUILD_DOCS)
math(EXPR EXPECTED_PACKAGING_COMPONENTS "${EXPECTED_PACKAGING_COMPONENTS} + 1")
endif()
external/CMakeLists.txt
поставляемый
+2 -6
Просмотреть файл
@@ -272,12 +272,8 @@ if(ROCPROFILER_BUILD_SQLITE3)
)
add_dependencies(rocprofiler-sdk-sqlite3 rocprofiler-sdk-sqlite-build)
else()
# make it optional temporarily because ROCm CI does not have SQLite3 dev installed and
# cannot build from source (missing tclsh)
find_package(SQLite3)
if(SQLite3_FOUND)
target_link_libraries(rocprofiler-sdk-sqlite3 INTERFACE SQLite::SQLite3)
endif()
find_package(SQLite3 REQUIRED)
target_link_libraries(rocprofiler-sdk-sqlite3 INTERFACE SQLite::SQLite3)
endif()
#
source/bin/CMakeLists.txt
+1 -2
Просмотреть файл
@@ -5,8 +5,7 @@
rocprofiler_activate_clang_tidy()
# Adding main rocprofv3
configure_file(rocprofv3.py ${PROJECT_BINARY_DIR}/${CMAKE_INSTALL_BINDIR}/rocprofv3
COPYONLY)
configure_file(rocprofv3.py ${PROJECT_BINARY_DIR}/${CMAKE_INSTALL_BINDIR}/rocprofv3 @ONLY)
install(
FILES ${PROJECT_BINARY_DIR}/${CMAKE_INSTALL_BINDIR}/rocprofv3
source/bin/rocprofv3.py
+51 -9
Просмотреть файл
@@ -22,11 +22,24 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
import argparse
import os
import subprocess
import textwrap
import sys
import argparse
import textwrap
import subprocess
# version info for rocprofiler-sdk / rocprofv3
CONST_VERSION_INFO = {
"version": "@FULL_VERSION_STRING@",
"git_revision": "@ROCPROFILER_SDK_GIT_REVISION@",
"library_arch": "@CMAKE_LIBRARY_ARCHITECTURE@",
"system_name": "@CMAKE_SYSTEM_NAME@",
"system_processor": "@CMAKE_SYSTEM_PROCESSOR@",
"system_version": "@CMAKE_SYSTEM_VERSION@",
"compiler_id": "@CMAKE_CXX_COMPILER_ID@",
"compiler_version": "@CMAKE_CXX_COMPILER_VERSION@",
"rocm_version": "@rocm_version_FULL_VERSION@",
}
class dotdict(dict):
@@ -171,6 +184,13 @@ For MPI applications (or other job launchers such as SLURM), place rocprofv3 ins
metavar="BOOL",
)
parser.add_argument(
"-v",
"--version",
action="store_true",
help="Print the version information and exit",
)
io_options = parser.add_argument_group("I/O options")
io_options.add_argument(
@@ -196,11 +216,12 @@ For MPI applications (or other job launchers such as SLURM), place rocprofv3 ins
required=False,
)
io_options.add_argument(
"-f",
"--output-format",
help="For adding output format (supported formats: csv, json, pftrace, otf2)",
help="For adding output format (supported formats: csv, json, pftrace, otf2, rocpd)",
nargs="+",
default=None,
choices=("csv", "json", "pftrace", "otf2"),
choices=("csv", "json", "pftrace", "otf2", "rocpd"),
type=str.lower,
)
add_parser_bool_argument(
@@ -1018,7 +1039,7 @@ def run(app_args, args, **kwargs):
update_env("ROCPROF_OUTPUT_LIST_AVAIL_FILE", True)
if not args.output_format:
args.output_format = ["csv"]
args.output_format = ["rocpd"]
update_env(
"ROCPROF_OUTPUT_FORMAT", ",".join(args.output_format), append=True, join_char=","
@@ -1102,6 +1123,9 @@ def run(app_args, args, **kwargs):
trace_count += 1 if val else 0
trace_opts += ["--{}".format(opt.replace("_", "-"))]
if args.pmc_groups:
trace_count += 1
# if marker tracing was requested, LD_PRELOAD the rocprofiler-sdk-roctx library
# to override the roctx symbols of an app linked to the old roctracer roctx
if args.marker_trace and not args.suppress_marker_preload:
@@ -1144,6 +1168,12 @@ def run(app_args, args, **kwargs):
"\n ".join(trace_opts),
)
rocprofiler_ci_env = os.environ.get("ROCPROFILER_CI", "0")
if strtobool(rocprofiler_ci_env):
fatal_error(
f"rocprofv3 tracing options are required when ROCPROFILER_CI={rocprofiler_ci_env}"
)
_summary_groups = "##@@##".join(args.summary_groups) if args.summary_groups else None
_summary_output_fname = args.summary_output_file
if args.summary and _summary_output_fname is None:
@@ -1325,7 +1355,7 @@ def run(app_args, args, **kwargs):
update_env("ROCPROF_EXTRA_COUNTERS_CONTENTS", e_file_contents, overwrite=True)
if args.pmc and args.pmc_groups:
fatal_error("Cannot specify both --pmc and --pmc-groups")
fatal_error("Cannot specify both --pmc and (input file) pmc_groups")
if args.pmc:
update_env("ROCPROF_COUNTER_COLLECTION", True, overwrite=True)
@@ -1522,6 +1552,12 @@ def run(app_args, args, **kwargs):
def main(argv=None):
cmd_args, app_args = parse_arguments(argv)
if cmd_args.version:
for key, itr in CONST_VERSION_INFO.items():
print(f" {key:>16}: {itr}")
return 0
inp_args = (
parse_input(cmd_args.input) if getattr(cmd_args, "input") else [dotdict({})]
)
@@ -1531,16 +1567,22 @@ def main(argv=None):
pass_idx = None
if has_set_attr(args, "pmc") and len(args.pmc) > 0:
pass_idx = 1
run(app_args, args, pass_id=pass_idx)
ec = run(app_args, args, pass_id=pass_idx)
else:
ec = 0
for idx, itr in enumerate(inp_args):
args = get_args(cmd_args, itr)
run(
_ec = run(
app_args,
args,
pass_id=(idx + 1),
use_execv=False,
)
if _ec is not None and _ec != 0:
ec = _ec
# return error code
return ec if not None else 0
if __name__ == "__main__":
source/include/CMakeLists.txt
+1
Просмотреть файл
@@ -5,3 +5,4 @@ set(CMAKE_INSTALL_DEFAULT_COMPONENT_NAME "development")
add_subdirectory(rocprofiler-sdk)
add_subdirectory(rocprofiler-sdk-roctx)
add_subdirectory(rocprofiler-sdk-rocpd)
source/include/rocprofiler-sdk-rocpd/CMakeLists.txt
+18
Просмотреть файл
@@ -0,0 +1,18 @@
#
#
# Installation of public headers
#
#
configure_file(${CMAKE_CURRENT_LIST_DIR}/version.h.in
${CMAKE_CURRENT_BINARY_DIR}/version.h @ONLY)
set(ROCPD_HEADER_FILES
# core headers
rocpd.h
# secondary headers
defines.h types.h ${CMAKE_CURRENT_BINARY_DIR}/version.h)
install(
FILES ${ROCPD_HEADER_FILES}
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/rocprofiler-sdk-rocpd
COMPONENT rocpd)
source/include/rocprofiler-sdk-rocpd/defines.h
+130
Просмотреть файл
@@ -0,0 +1,130 @@
// MIT License
//
// Copyright (c) 2023-2025 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#pragma once
/**
* @defgroup SYMBOL_VERSIONING_GROUP Symbol Versions
*
* @brief The names used for the shared library versioned symbols.
*
* Every function is annotated with one of the version macros defined in this
* section. Each macro specifies a corresponding symbol version string. After
* dynamically loading the shared library with @p dlopen, the address of each
* function can be obtained using @p dlsym with the name of the function and
* its corresponding symbol version string. An error will be reported by @p
* dlvsym if the installed library does not support the version for the
* function specified in this version of the interface.
*
* @{
*/
/**
* @brief The function was introduced in version 0.0 of the interface and has the
* symbol version string of ``"ROCPROFILER_SDK_ROCPD_0.0"``.
*/
#define ROCPROFILER_SDK_ROCPD_VERSION_0_0
/** @} */
#if !defined(ROCPD_ATTRIBUTE)
# if defined(_MSC_VER)
# define ROCPD_ATTRIBUTE(...) __declspec(__VA_ARGS__)
# else
# define ROCPD_ATTRIBUTE(...) __attribute__((__VA_ARGS__))
# endif
#endif
#if !defined(ROCPD_PUBLIC_API)
# if defined(_MSC_VER)
# define ROCPD_PUBLIC_API ROCPD_ATTRIBUTE(dllexport)
# else
# define ROCPD_PUBLIC_API ROCPD_ATTRIBUTE(visibility("default"))
# endif
#endif
#if !defined(ROCPD_HIDDEN_API)
# if defined(_MSC_VER)
# define ROCPD_HIDDEN_API
# else
# define ROCPD_HIDDEN_API ROCPD_ATTRIBUTE(visibility("hidden"))
# endif
#endif
#if !defined(ROCPD_EXPORT_DECORATOR)
# define ROCPD_EXPORT_DECORATOR ROCPD_PUBLIC_API
#endif
#if !defined(ROCPD_IMPORT_DECORATOR)
# if defined(_MSC_VER)
# define ROCPD_IMPORT_DECORATOR ROCPD_ATTRIBUTE(dllimport)
# else
# define ROCPD_IMPORT_DECORATOR
# endif
#endif
#define ROCPD_EXPORT ROCPD_EXPORT_DECORATOR
#define ROCPD_IMPORT ROCPD_IMPORT_DECORATOR
#if !defined(ROCPD_API)
# if defined(rocpd_EXPORTS)
# define ROCPD_API ROCPD_EXPORT
# else
# define ROCPD_API ROCPD_IMPORT
# endif
#endif
#if defined(__has_attribute)
# if __has_attribute(nonnull)
# define ROCPD_NONNULL(...) __attribute__((nonnull(__VA_ARGS__)))
# else
# define ROCPD_NONNULL(...)
# endif
#else
# if defined(__GNUC__)
# define ROCPD_NONNULL(...) __attribute__((nonnull(__VA_ARGS__)))
# else
# define ROCPD_NONNULL(...)
# endif
#endif
#ifdef __cplusplus
# define ROCPD_EXTERN_C_INIT extern "C" {
# define ROCPD_EXTERN_C_FINI }
#else
# define ROCPD_EXTERN_C_INIT
# define ROCPD_EXTERN_C_FINI
#endif
#if !defined(ROCPD_EXPERIMENTAL_WARNINGS)
# define ROCPD_EXPERIMENTAL_WARNINGS 0
#endif
#define ROCPD_EXPERIMENTAL_MESSAGE \
ROCPD_DEPRECATED_MESSAGE("Note: this feature has been marked as experimental. Define " \
"ROCPD_EXPERIMENTAL_WARNINGS=0 to silence this message.")
#if defined(ROCPD_EXPERIMENTAL_WARNINGS) && ROCPD_EXPERIMENTAL_WARNINGS > 0
# define ROCPD_EXPERIMENTAL ROCPD_EXPERIMENTAL_MESSAGE
#else
# define ROCPD_EXPERIMENTAL
#endif
source/include/rocprofiler-sdk-rocpd/rocpd.h
+118
Просмотреть файл
@@ -0,0 +1,118 @@
// MIT License
//
// Copyright (c) 2023-2025 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#pragma once
/**
* @file rocpd.h
* @brief rocPD API interface for AMD profiling data analysis
*
* @mainpage rocPD API Specification
*
*/
#include "rocprofiler-sdk-rocpd/defines.h"
#include "rocprofiler-sdk-rocpd/types.h"
/**
* @defgroup VERSIONING_GROUP Library Versioning
* @brief Version information about the interface and the associated installed library.
*
* The semantic version of the interface following semver.org rules. A context
* that uses this interface is only compatible with the installed library if
* the major version numbers match and the interface minor version number is
* less than or equal to the installed library minor version number.
*
* @{
*/
#include "rocprofiler-sdk-rocpd/version.h"
ROCPD_EXTERN_C_INIT
/**
* @fn rocpd_status_t rocpd_get_version(uint32_t* major, uint32_t* minor, uint32_t*
* patch)
* @brief Query the version of the installed library.
*
* Returns the version of the rocprofiler-sdk library loaded at runtime. This can be used to check
* if the runtime version is equal to or compatible with the version of rocprofiler-sdk used during
* compilation time. This function can be invoked before tool initialization.
*
* @param [out] major The major version number is stored if non-NULL.
* @param [out] minor The minor version number is stored if non-NULL.
* @param [out] patch The patch version number is stored if non-NULL.
* @return ::rocpd_status_t
* @retval ::ROCPD_STATUS_SUCCESS Always returned
*/
rocpd_status_t
rocpd_get_version(uint32_t* major, uint32_t* minor, uint32_t* patch) ROCPD_API;
/**
* @brief Simplified alternative to ::rocpd_get_version
*
* Returns the version of the rocprofiler-sdk library loaded at runtime. This can be used to check
* if the runtime version is equal to or compatible with the version of rocprofiler-sdk used during
* compilation time. This function can be invoked before tool initialization.
*
* @param [out] info Pointer to version triplet struct which will be populated by the function call.
* @return ::rocpd_status_t
* @retval ::ROCPD_STATUS_SUCCESS Always returned
*/
rocpd_status_t
rocpd_get_version_triplet(rocpd_version_triplet_t* info) ROCPD_API ROCPD_NONNULL(1);
ROCPD_EXTERN_C_FINI
/** @} */
#include "rocprofiler-sdk-rocpd/sql.h"
ROCPD_EXTERN_C_INIT
/**
* @defgroup MISCELLANEOUS_GROUP Miscellaneous Utility Functions
* @brief utility functions for library
* @{
*/
/**
* @fn const char* rocpd_get_status_name(rocpd_status_t status)
* @brief Return the string encoding of ::rocpd_status_t value
* @param [in] status error code value
* @return Will return a nullptr if invalid/unsupported ::rocpd_status_t value is provided.
*/
const char*
rocpd_get_status_name(rocpd_status_t status) ROCPD_API;
/**
* @fn const char* rocpd_get_status_string(rocpd_status_t status)
* @brief Return the message associated with ::rocpd_status_t value
* @param [in] status error code value
* @return Will return a nullptr if invalid/unsupported ::rocpd_status_t value is provided.
*/
const char*
rocpd_get_status_string(rocpd_status_t status) ROCPD_API;
/** @} */
ROCPD_EXTERN_C_FINI
source/include/rocprofiler-sdk-rocpd/sql.h
+151
Просмотреть файл
@@ -0,0 +1,151 @@
// MIT License
//
// Copyright (c) 2023-2025 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#pragma once
#include <rocprofiler-sdk-rocpd/defines.h>
#include <rocprofiler-sdk-rocpd/types.h>
#include <stdint.h>
ROCPD_EXTERN_C_INIT
/**
* @defgroup SQL rocPD SQL Utilities
* @brief Functions for reading rocpd SQL schema
* @{
*/
/**
* @brief (experimental) Supported SQL engines. Initial support is for SQLite3, other engines such
* as MySQL may be added
*/
typedef enum ROCPD_EXPERIMENTAL rocpd_sql_engine_t
{
ROCPD_SQL_ENGINE_NONE = 0,
ROCPD_SQL_ENGINE_SQLITE3, ///< Ensure compatibility with SQLite3
ROCPD_SQL_ENGINE_LAST,
} rocpd_sql_engine_t;
/**
* @brief (experimental) Supported SQL schema kinds
*/
typedef enum ROCPD_EXPERIMENTAL rocpd_sql_schema_kind_t
{
ROCPD_SQL_SCHEMA_NONE = 0,
ROCPD_SQL_SCHEMA_ROCPD_TABLES,
ROCPD_SQL_SCHEMA_ROCPD_INDEXES,
ROCPD_SQL_SCHEMA_ROCPD_VIEWS,
ROCPD_SQL_SCHEMA_ROCPD_DATA_VIEWS,
ROCPD_SQL_SCHEMA_ROCPD_SUMMARY_VIEWS,
ROCPD_SQL_SCHEMA_ROCPD_MARKER_VIEWS,
ROCPD_SQL_SCHEMA_LAST,
} rocpd_sql_schema_kind_t;
/**
* @brief (experimental) Supported SQL options
*/
typedef enum ROCPD_EXPERIMENTAL rocpd_sql_options_t
{
ROCPD_SQL_OPTIONS_NONE = 0,
ROCPD_SQL_OPTIONS_PERSIST_STRINGS = 0x01, ///< do not delete strings
ROCPD_SQL_OPTIONS_SQLITE3_PRAGMA_FOREIGN_KEYS = 0x02, ///< enable SQLite3 foreign keys
ROCPD_SQL_OPTIONS_LAST = 0xFFFFFFFF,
} rocpd_sql_options_t;
/**
* @brief (experimental) Schema jinja variable substitution values. In general, the struct member
* variable replaces jinja variables of the same name, e.g. the struct member variable `uuid`
* replaces `{{uuid}}` in the schema definition.
*
* When the schema variable `{{uuid}}` is a non-null and non-empty string, it
* will be prefixed with a leading underscore (`_`) and all hyphens will be replaced with
* underscores. This is because this variable is used in SQL table names. The leading
* underscore improves the readability of the table name and the replacement of hyphens with
* underscores is reduces problems.
*/
typedef struct ROCPD_EXPERIMENTAL rocpd_sql_schema_jinja_variables_t
{
uint64_t size; ///< Size of this struct (minus reserved padding)
const char* uuid; ///< Substitution for `{{uuid}}` (non-empty adds leading underscore)
const char* guid; ///< Substitution for `{{guid}}`
} rocpd_sql_schema_jinja_variables_t;
/**
* @brief (experimental) Callback providing the schema content after variable substitution.
*
* @param [in] engine Schema conforms to this SQL database engine
* @param [in] kind Schema category
* @param [in] options Schema options included in content
* @param [in] variables Jinja variables which were substituted
* @param [in] schema_path Filesystem path to base schema file
* @param [in] schema_content SQL schema content is pass to database
* @param [in] user_data User provided data
*/
ROCPD_EXPERIMENTAL typedef void (*rocpd_sql_load_schema_cb_t)(
rocpd_sql_engine_t engine,
rocpd_sql_schema_kind_t kind,
rocpd_sql_options_t options,
const rocpd_sql_schema_jinja_variables_t* variables,
const char* schema_path,
const char* schema_content,
void* user_data);
/**
* @brief (experimental) Invoke the callback which provides the schema kind definition for the given
* SQL engine + the addition of the requested options + jinja variable substitution.
*
* @param [in] engine SQL schema contents should be compatible with this SQL database engine
* @param [in] kind SQL schema kind (tables, indexes, views, etc.)
* @param [in] options Options for the callback and schema
* @param [in] variables Defines variables for jinja substitution. If nullptr, no jinja variables
* will be substituted. For each member variable that is a nullptr, jinja for that member variable
* will not be substituted. To replace jinja variables with empty strings, assign all member
* variables to empty string.
* @param [in] callback Callback function which provides the schema contents
* @param [in] schema_path_hints Suggests for where to find the schema templates
* @param [in] num_schema_path_hints Number of schema path hints
* @param [in] user_data Pointer to pass back into callback
* @return ::rocpd_status_t
* @retval ROCPD_STATUS_SUCCESS if all parameter specifications were applied and callback
* successfully invoked
* @retval ROCPD_STATUS_ERROR_INVALID_ARGUMENT Invalid SQL engine
* @retval ROCPD_STATUS_ERROR_NOT_AVAILABLE Schema file could not be found
* @retval ROCPD_STATUS_ERROR_PERMISSION_DENIED Schema file could not be found
* @retval ROCPD_STATUS_ERROR_KIND_NOT_FOUND Invalid SQL schema kind
* @retval ROCPD_STATUS_ERROR There was some issue with parameters
* @retval ROCPD_STATUS_ERROR_INVALID_ARGUMENT ::rocpd_sql_schema_jinja_variables_t size parameter
* not set
*/
ROCPD_EXPERIMENTAL rocpd_status_t
rocpd_sql_load_schema(rocpd_sql_engine_t engine,
rocpd_sql_schema_kind_t kind,
rocpd_sql_options_t options,
const rocpd_sql_schema_jinja_variables_t* variables,
rocpd_sql_load_schema_cb_t callback,
const char** schema_path_hints,
uint64_t num_schema_path_hints,
void* user_data) ROCPD_API ROCPD_NONNULL(5);
/** @} */
ROCPD_EXTERN_C_FINI
source/include/rocprofiler-sdk-rocpd/types.h
+81
Просмотреть файл
@@ -0,0 +1,81 @@
// MIT License
//
// Copyright (c) 2023-2025 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#pragma once
#include "rocprofiler-sdk-rocpd/defines.h"
#include <stdint.h>
/** @defgroup DATA_TYPE rocPD Data types
*
* Data types defined or aliased by rocPD
*
* @{
*/
//--------------------------------------------------------------------------------------//
//
// ENUMERATIONS
//
//--------------------------------------------------------------------------------------//
/**
* @defgroup BASIC_DATA_TYPES Basic data types
* @brief Basic data types and typedefs
*
* @{
*/
/**
* @brief Status codes.
*/
typedef enum rocpd_status_t // NOLINT(performance-enum-size)
{
ROCPD_STATUS_SUCCESS = 0, ///< No error occurred
ROCPD_STATUS_ERROR, ///< Generalized error
ROCPD_STATUS_ERROR_INVALID_ARGUMENT, ///< Argument is invalid
ROCPD_STATUS_ERROR_SQL_ERROR, ///< General SQL error
ROCPD_STATUS_ERROR_SQL_INVALID_ENGINE, ///< SQL engine is invalid
ROCPD_STATUS_ERROR_SQL_INVALID_SCHEMA_KIND, ///< SQL schema kind not found
ROCPD_STATUS_ERROR_SQL_SCHEMA_NOT_FOUND, ///< SQL schema does not exist
ROCPD_STATUS_ERROR_SQL_SCHEMA_PERMISSION_DENIED, ///< SQL schema not accessible
ROCPD_STATUS_LAST,
} rocpd_status_t;
//--------------------------------------------------------------------------------------//
//
// STRUCTS
//
//--------------------------------------------------------------------------------------//
/**
* @brief Versioning info.
*/
typedef struct rocpd_version_triplet_t
{
uint32_t major;
uint32_t minor;
uint32_t patch;
} rocpd_version_triplet_t;
/** @} */
source/include/rocprofiler-sdk-rocpd/version.h.in
+115
Просмотреть файл
@@ -0,0 +1,115 @@
// MIT License
//
// Copyright (c) 2023-2025 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#pragma once
/**
* @def ROCPD_IS_ROCPROFILER_SDK
* @brief Preprocessor define indicating the rocpd header is a rocprofiler-sdk project
* @addtogroup VERSIONING_GROUP
*
* @def ROCPD_VERSION_MAJOR
* @brief The major version of the interface as a macro so it can be used
* by the preprocessor.
* @addtogroup VERSIONING_GROUP
*
* @def ROCPD_VERSION_MINOR
* @brief The minor version of the interface as a macro so it can be used
* by the preprocessor.
* @addtogroup VERSIONING_GROUP
*
* @def ROCPD_VERSION_PATCH
* @brief The patch version of the interface as a macro so it can be used
* by the preprocessor.
* @addtogroup VERSIONING_GROUP
*
* @def ROCPD_VERSION
* @brief Numerically increasing version number encoding major, minor, and patch via
computing `((10000 * <MAJOR>) + (100 * <MINOR>) + <PATCH>)`.
* @addtogroup VERSIONING_GROUP
*
* @def ROCPD_SOVERSION
* @brief Shared object versioning value whose value is at least `(10000 * <MAJOR>)`.
* @addtogroup VERSIONING_GROUP
*
* @def ROCPD_VERSION_STRING
* @brief Version string in form: `<MAJOR>.<MINOR>.<PATCH>`.
* @addtogroup VERSIONING_GROUP
*
* @def ROCPD_GIT_DESCRIBE
* @brief String encoding of `git describe --tags` when rocprofiler was built.
* @addtogroup VERSIONING_GROUP
*
* @def ROCPD_GIT_REVISION
* @brief String encoding of `git rev-parse HEAD` when rocprofiler was built.
* @addtogroup VERSIONING_GROUP
*
* @def ROCPD_LIBRARY_ARCH
* @brief Architecture triplet of rocprofiler build.
* @addtogroup VERSIONING_GROUP
*
* @def ROCPD_SYSTEM_NAME
* @brief Target operating system for rocprofiler build, e.g. Linux.
* @addtogroup VERSIONING_GROUP
*
* @def ROCPD_SYSTEM_PROCESSOR
* @brief Target architecture for rocprofiler build.
* @addtogroup VERSIONING_GROUP
*
* @def ROCPD_SYSTEM_VERSION
* @brief Version of the operating system which built rocprofiler
* @addtogroup VERSIONING_GROUP
*
* @def ROCPD_COMPILER_ID
* @brief C++ compiler identifier which built rocprofiler, e.g., GNU
* @addtogroup VERSIONING_GROUP
*
* @def ROCPD_COMPILER_VERSION
* @brief C++ compiler version which built rocprofiler
* @addtogroup VERSIONING_GROUP
*/
#define ROCPD_IS_ROCPROFILER_SDK 1
// clang-format off
#define ROCPD_VERSION_MAJOR @PROJECT_VERSION_MAJOR@
#define ROCPD_VERSION_MINOR @PROJECT_VERSION_MINOR@
#define ROCPD_VERSION_PATCH @PROJECT_VERSION_PATCH@
#define ROCPD_SOVERSION (10000 * @PROJECT_VERSION_MAJOR@)
#define ROCPD_VERSION_STRING "@FULL_VERSION_STRING@"
#define ROCPD_GIT_DESCRIBE "@ROCPROFILER_SDK_GIT_DESCRIBE@"
#define ROCPD_GIT_REVISION "@ROCPROFILER_SDK_GIT_REVISION@"
// system info during compilation
#define ROCPD_LIBRARY_ARCH "@CMAKE_LIBRARY_ARCHITECTURE@"
#define ROCPD_SYSTEM_NAME "@CMAKE_SYSTEM_NAME@"
#define ROCPD_SYSTEM_PROCESSOR "@CMAKE_SYSTEM_PROCESSOR@"
#define ROCPD_SYSTEM_VERSION "@CMAKE_SYSTEM_VERSION@"
// compiler information
#define ROCPD_COMPILER_ID "@CMAKE_CXX_COMPILER_ID@"
#define ROCPD_COMPILER_VERSION "@CMAKE_CXX_COMPILER_VERSION@"
// clang-format on
#define ROCPD_VERSION \
((10000 * ROCPD_VERSION_MAJOR) + (100 * ROCPD_VERSION_MINOR) + ROCPD_VERSION_PATCH)
source/include/rocprofiler-sdk/cxx/serialization/save.hpp
-1
Просмотреть файл
@@ -41,7 +41,6 @@
#include <rocprofiler-sdk/marker.h>
#include <rocprofiler-sdk/pc_sampling.h>
#include <rocprofiler-sdk/cxx/name_info.hpp>
#include <rocprofiler-sdk/cxx/perfetto.hpp>
#include <rocprofiler-sdk/cxx/utility.hpp>
#include <cereal/archives/binary.hpp>
source/lib/CMakeLists.txt
+6 -1
Просмотреть файл
@@ -8,9 +8,14 @@ add_subdirectory(common)
add_subdirectory(output)
add_subdirectory(rocprofiler-sdk)
set(CMAKE_INSTALL_DEFAULT_COMPONENT_NAME "roctx")
add_subdirectory(rocprofiler-sdk-roctx)
set(CMAKE_INSTALL_DEFAULT_COMPONENT_NAME "rocpd")
add_subdirectory(rocprofiler-sdk-rocpd)
set(CMAKE_INSTALL_DEFAULT_COMPONENT_NAME "tools")
add_subdirectory(att-tool)
add_subdirectory(rocprofiler-sdk-roctx)
add_subdirectory(rocprofiler-sdk-tool)
add_subdirectory(python)
source/lib/common/CMakeLists.txt
+6 -2
Просмотреть файл
@@ -9,11 +9,13 @@ set(common_sources
environment.cpp
logging.cpp
md5sum.cpp
sha256.cpp
simple_timer.cpp
static_object.cpp
static_tl_object.cpp
string_entry.cpp
utility.cpp)
utility.cpp
uuid_v7.cpp)
set(common_headers
abi.hpp
defines.hpp
@@ -26,6 +28,7 @@ set(common_headers
md5sum.hpp
mpl.hpp
scope_destructor.hpp
sha256.hpp
simple_timer.hpp
static_object.hpp
static_tl_object.hpp
@@ -33,7 +36,8 @@ set(common_headers
stringize_arg.hpp
synchronized.hpp
units.hpp
utility.hpp)
utility.hpp
uuid_v7.hpp)
add_library(rocprofiler-sdk-common-library STATIC)
add_library(rocprofiler-sdk::rocprofiler-sdk-common-library ALIAS
source/lib/common/environment.cpp
+51
Просмотреть файл
@@ -151,5 +151,56 @@ SPECIALIZE_SET_ENV(std::string_view)
SPECIALIZE_SET_ENV(float)
SPECIALIZE_SET_ENV(double)
} // namespace impl
env_store::env_store(std::initializer_list<env_config>&& _container)
{
for(const auto& itr : _container)
{
m_original.emplace_back(env_config{itr.env_name, get_env(itr.env_name, ""), 1});
m_modified.emplace_back(env_config{itr.env_name, itr.env_value, 1});
}
}
env_store::~env_store() { pop(); }
bool
env_store::push()
{
// not that push ignored bc already pushed
if(m_pushed) return false;
for(const auto& itr : m_modified)
itr();
m_pushed = true;
return true;
}
bool
env_store::pop(bool unset_if_empty)
{
if(!m_pushed) return false;
for(const auto& itr : m_original)
{
auto _current = get_env(itr.env_name, "");
if(!unset_if_empty && itr.env_value.empty())
continue;
else if(_current == itr.env_value)
continue;
else if(_current != itr.env_value)
{
ROCP_INFO << fmt::format("[rocprofiler][env][pop] {}=\"{}\" => {}=\"{}\"",
itr.env_name,
_current,
itr.env_name,
itr.env_value);
}
itr();
}
m_pushed = false;
return true;
}
} // namespace common
} // namespace rocprofiler
source/lib/common/environment.hpp
+42 -4
Просмотреть файл
@@ -88,11 +88,49 @@ struct env_config
auto operator()(bool _verbose = false) const
{
if(env_name.empty()) return -1;
ROCP_INFO_IF(_verbose) << "[rocprofiler][set_env] setenv(\"" << env_name << "\", \""
<< env_value << "\", " << overwrite << ")\n";
return setenv(env_name.c_str(), env_value.c_str(), overwrite);
if(env_name.empty())
return -1;
else if(_verbose)
{
ROCP_INFO << "[rocprofiler][set_env] setenv(\"" << env_name << "\", \"" << env_value
<< "\", " << overwrite << ")\n";
}
return (env_value.empty() && overwrite > 0)
? unsetenv(env_name.c_str())
: setenv(env_name.c_str(), env_value.c_str(), overwrite);
}
};
struct env_store
{
template <template <typename, typename...> class ContainerT, typename... TailT>
explicit env_store(ContainerT<env_config, TailT...>&& _container);
explicit env_store(std::initializer_list<env_config>&& _container);
~env_store();
env_store(const env_store&) = default;
env_store(env_store&&) noexcept = default;
env_store& operator=(const env_store&) = default;
env_store& operator=(env_store&&) noexcept = default;
bool push();
bool pop(bool unset_if_empty = true);
bool is_pushed() const { return m_pushed; }
private:
bool m_pushed = false;
std::vector<env_config> m_original = {};
std::vector<env_config> m_modified = {};
};
template <template <typename, typename...> class ContainerT, typename... TailT>
env_store::env_store(ContainerT<env_config, TailT...>&& _container)
{
for(const auto& itr : _container)
{
m_original.emplace_back(env_config{itr.env_name, get_env(itr.env_name, ""), 1});
m_modified.emplace_back(env_config{itr.env_name, itr.env_value, 1});
}
}
} // namespace common
} // namespace rocprofiler
source/lib/common/logging.cpp
+35 -18
Просмотреть файл
@@ -53,6 +53,34 @@ struct log_level_info
int32_t google_level = 0;
int32_t verbose_level = 0;
};
env_store
get_glog_env_config(const logging_config& cfg)
{
auto as_env_config = [](std::string_view _var, auto _val) {
return env_config{std::string{_var}, fmt::format("{}", _val), 1};
};
auto _data = std::vector<env_config>{
as_env_config("GLOG_minloglevel", cfg.loglevel),
as_env_config("GLOG_logtostderr", cfg.logtostderr ? 1 : 0),
as_env_config("GLOG_alsologtostderr", cfg.alsologtostderr ? 1 : 0),
as_env_config("GLOG_stderrthreshold", cfg.loglevel),
as_env_config("GLOG_v", cfg.vlog_level),
};
if(!cfg.logdir.empty())
{
_data.emplace_back(as_env_config("GOOGLE_LOG_DIR", cfg.logdir));
_data.emplace_back(as_env_config("GLOG_log_dir", cfg.logdir));
}
if(!cfg.vlog_modules.empty())
{
_data.emplace_back(as_env_config("GLOG_vmodule", cfg.vlog_modules));
}
return env_store{std::move(_data)};
}
} // namespace
void
@@ -139,7 +167,10 @@ init_logging(std::string_view env_prefix, logging_config cfg)
}
}
update_logging(cfg, !google::IsGoogleLoggingInitialized());
auto _env_store = get_glog_env_config(cfg);
update_logging(cfg);
_env_store.push();
if(!google::IsGoogleLoggingInitialized())
{
@@ -158,11 +189,13 @@ init_logging(std::string_view env_prefix, logging_config cfg)
ROCP_INFO << "logging initialized via " << fmt::format("{}_LOG_LEVEL", env_prefix)
<< ". Log Level: " << loglvl << ". Verbose Log Level: " << vlog_level;
_env_store.pop(false);
});
}
void
update_logging(const logging_config& cfg, bool setup_env, int env_override)
update_logging(const logging_config& cfg)
{
static auto _mtx = std::mutex{};
auto _lk = std::unique_lock<std::mutex>{_mtx};
@@ -192,22 +225,6 @@ update_logging(const logging_config& cfg, bool setup_env, int env_override)
}
}
}
if(setup_env)
{
common::set_env("GLOG_minloglevel", cfg.loglevel, env_override);
common::set_env("GLOG_logtostderr", cfg.logtostderr ? 1 : 0, env_override);
common::set_env("GLOG_alsologtostderr", cfg.alsologtostderr ? 1 : 0, env_override);
common::set_env("GLOG_stderrthreshold", cfg.loglevel, env_override);
common::set_env("GLOG_v", cfg.vlog_level, env_override);
if(!cfg.logdir.empty())
{
common::set_env("GOOGLE_LOG_DIR", cfg.logdir, env_override);
common::set_env("GLOG_log_dir", cfg.logdir, env_override);
}
if(!cfg.vlog_modules.empty())
common::set_env("GLOG_vmodule", cfg.vlog_modules, env_override);
}
}
} // namespace common
} // namespace rocprofiler
source/lib/common/logging.hpp
+1 -1
Просмотреть файл
@@ -79,6 +79,6 @@ void
init_logging(std::string_view env_prefix, logging_config cfg = logging_config{});
void
update_logging(const logging_config& cfg, bool setup_env = false, int env_override = 0);
update_logging(const logging_config& cfg);
} // namespace common
} // namespace rocprofiler
source/lib/common/sha256.cpp
+229
Просмотреть файл
@@ -0,0 +1,229 @@
// MIT License
//
// Copyright (c) 2025 Advanced Micro Devices, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
#include "lib/common/sha256.hpp"
#include "lib/common/defines.hpp"
#include "lib/common/logging.hpp"
#include "lib/common/mpl.hpp"
#include <unistd.h>
#include <array>
#include <cstdint>
#include <cstring>
#include <fstream>
#include <iomanip>
#include <sstream>
#include <string>
#include <vector>
namespace rocprofiler
{
namespace common
{
sha256::sha256() { reset(); }
sha256::sha256(const std::string& data)
{
reset();
update(data);
finalize();
}
void
sha256::update(const uint8_t* data, size_t len)
{
ROCP_CI_LOG_IF(INFO, m_finalized) << "attempt to update sha256 after finalized";
if(m_finalized) return;
for(size_t i = 0; i < len; ++i)
{
m_data[m_datalen++] = data[i];
if(m_datalen == 64)
{
transform();
m_bitlen += 512;
m_datalen = 0;
}
}
}
void
sha256::update(const std::string& data)
{
ROCP_CI_LOG_IF(INFO, m_finalized) << "attempt to update sha256 after finalized";
if(m_finalized) return;
update(reinterpret_cast<const uint8_t*>(data.data()), data.size());
}
void
sha256::finalize()
{
if(m_finalized) return;
uint32_t idx = m_datalen;
if(m_datalen < 56)
{
m_data[idx++] = 0x80;
while(idx < 56)
m_data[idx++] = 0x00;
}
else
{
m_data[idx++] = 0x80;
while(idx < 64)
m_data[idx++] = 0x00;
transform();
std::memset(m_data.data(), 0, 56);
}
m_bitlen += m_datalen * 8;
for(int j = 0; j < 8; ++j)
m_data[63 - j] = static_cast<uint8_t>((m_bitlen >> (8 * j)) & 0xFF);
transform();
m_finalized = true;
}
std::string
sha256::hexdigest()
{
finalize();
auto oss = std::ostringstream{};
for(int j = 0; j < 8; ++j)
oss << std::hex << std::setfill('0') << std::setw(8) << m_state[j];
return oss.str();
}
std::array<uint32_t, 8>
sha256::rawdigest()
{
finalize();
return m_state;
}
uint32_t
sha256::rotr(uint32_t x, uint32_t n)
{
return (x >> n) | (x << (32 - n));
}
uint32_t
sha256::ch(uint32_t x, uint32_t y, uint32_t z)
{
return (x & y) ^ (~x & z);
}
uint32_t
sha256::maj(uint32_t x, uint32_t y, uint32_t z)
{
return (x & y) ^ (x & z) ^ (y & z);
}
uint32_t
sha256::sig0(uint32_t x)
{
return rotr(x, 2) ^ rotr(x, 13) ^ rotr(x, 22);
}
uint32_t
sha256::sig1(uint32_t x)
{
return rotr(x, 6) ^ rotr(x, 11) ^ rotr(x, 25);
}
uint32_t
sha256::theta0(uint32_t x)
{
return rotr(x, 7) ^ rotr(x, 18) ^ (x >> 3);
}
uint32_t
sha256::theta1(uint32_t x)
{
return rotr(x, 17) ^ rotr(x, 19) ^ (x >> 10);
}
void
sha256::transform()
{
uint32_t m[64];
for(int i = 0; i < 16; ++i)
{
m[i] = (m_data[i * 4] << 24) | (m_data[i * 4 + 1] << 16) | (m_data[i * 4 + 2] << 8) |
(m_data[i * 4 + 3]);
}
for(int i = 16; i < 64; ++i)
{
m[i] = theta1(m[i - 2]) + m[i - 7] + theta0(m[i - 15]) + m[i - 16];
}
uint32_t a = m_state[0];
uint32_t b = m_state[1];
uint32_t c = m_state[2];
uint32_t d = m_state[3];
uint32_t e = m_state[4];
uint32_t f = m_state[5];
uint32_t g = m_state[6];
uint32_t h = m_state[7];
for(int i = 0; i < 64; ++i)
{
uint32_t t1 = h + sig1(e) + ch(e, f, g) + m_k[i] + m[i];
uint32_t t2 = sig0(a) + maj(a, b, c);
h = g;
g = f;
f = e;
e = d + t1;
d = c;
c = b;
b = a;
a = t1 + t2;
}
m_state[0] += a;
m_state[1] += b;
m_state[2] += c;
m_state[3] += d;
m_state[4] += e;
m_state[5] += f;
m_state[6] += g;
m_state[7] += h;
}
void
sha256::reset()
{
m_state = {0x6a09e667,
0xbb67ae85,
0x3c6ef372,
0xa54ff53a,
0x510e527f,
0x9b05688c,
0x1f83d9ab,
0x5be0cd19};
m_datalen = 0;
m_bitlen = 0;
}
} // namespace common
} // namespace rocprofiler
source/lib/common/sha256.hpp
+81
Просмотреть файл
@@ -0,0 +1,81 @@
// MIT License
//
// Copyright (c) 2025 Advanced Micro Devices, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
#pragma once
#include "lib/common/defines.hpp"
#include "lib/common/mpl.hpp"
#include <array>
#include <cstdint>
#include <string>
namespace rocprofiler
{
namespace common
{
// --- SHA-256 Implementation ---
class sha256
{
public:
sha256();
explicit sha256(const std::string& data);
void update(const uint8_t* data, size_t len);
void update(const std::string& data);
void finalize();
std::string hexdigest();
std::array<uint32_t, 8> rawdigest();
private:
bool m_finalized = false;
std::array<uint8_t, 64> m_data = {};
std::array<uint32_t, 8> m_state = {};
uint32_t m_datalen = 0;
uint64_t m_bitlen = 0;
static constexpr std::array<uint32_t, 64> m_k = {
0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, 0x923f82a4,
0xab1c5ed5, 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, 0x72be5d74, 0x80deb1fe,
0x9bdc06a7, 0xc19bf174, 0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f,
0x4a7484aa, 0x5cb0a9dc, 0x76f988da, 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7,
0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967, 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc,
0x53380d13, 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85, 0xa2bfe8a1, 0xa81a664b,
0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070, 0x19a4c116,
0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3,
0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7,
0xc67178f2};
static uint32_t rotr(uint32_t x, uint32_t n);
static uint32_t ch(uint32_t x, uint32_t y, uint32_t z);
static uint32_t maj(uint32_t x, uint32_t y, uint32_t z);
static uint32_t sig0(uint32_t x);
static uint32_t sig1(uint32_t x);
static uint32_t theta0(uint32_t x);
static uint32_t theta1(uint32_t x);
void transform();
void reset();
};
} // namespace common
} // namespace rocprofiler
source/lib/common/uuid_v7.cpp
+151
Просмотреть файл
@@ -0,0 +1,151 @@
// MIT License
//
// Copyright (c) 2025 Advanced Micro Devices, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
#include "lib/common/uuid_v7.hpp"
#include "lib/common/defines.hpp"
#include "lib/common/logging.hpp"
#include "lib/common/mpl.hpp"
#include "lib/common/sha256.hpp"
#include <fmt/format.h>
#include <unistd.h>
#include <array>
#include <cstdint>
#include <cstring>
#include <fstream>
#include <iomanip>
#include <random>
#include <sstream>
#include <string>
#include <vector>
namespace rocprofiler
{
namespace common
{
uint64_t
get_process_start_ticks_since_boot(pid_t pid)
{
auto line = std::string{};
// Read the stat file
if(auto stat_file = std::ifstream{fmt::format("/proc/{}/stat", pid)}; !stat_file.is_open())
{
ROCP_CI_LOG(WARNING) << fmt::format("failed to open /proc/{}/stat for process start time",
pid);
return 0;
}
else
{
// Read entire line
std::getline(stat_file, line);
}
// Locate the end of the comm field (")")
size_t rparen = line.rfind(')');
if(rparen == std::string::npos)
{
ROCP_CI_LOG(WARNING) << fmt::format("Malformed stat file for pid {}", pid);
return 0;
}
// Tokenize fields after ") "
auto iss = std::istringstream{line.substr(rparen + 2)};
auto token = std::string{};
// Skip fields 3 through 21
for(int i = 0; i < 20; ++i)
{
if(!(iss >> token))
{
ROCP_CI_LOG(WARNING) << fmt::format("Unexpected end of /proc/{}/stat", pid);
return 0;
}
}
// Field 22: starttime in clock ticks since boot
uint64_t start_ticks = 0;
if(!(iss >> start_ticks))
{
ROCP_CI_LOG(WARNING) << fmt::format(
"Unexpected end of /proc/{}/stat. Failed to read start ticks", pid);
return 0;
}
return start_ticks;
}
uint64_t
compute_system_seed(std::string_view machine_id, pid_t pid, pid_t ppid, uint64_t pstart_ticks)
{
// If no machine_id provided, read from /etc/machine-id
ROCP_CI_LOG_IF(WARNING, machine_id.empty())
<< fmt::format("compute_system_seed provided empty machine id");
// Hash for seed value
return std::hash<std::string>{}(
sha256{fmt::format("{}|{}|{}|{}", machine_id, pid, ppid, pstart_ticks)}.hexdigest());
}
std::string
generate_uuid_v7(uint64_t timestamp_ns, uint64_t seed, std::string_view delim)
{
constexpr auto nanosec_per_millisec = std::nano::den / std::milli::den;
auto timestamp_ms = timestamp_ns / nanosec_per_millisec;
auto uuid = std::array<uint8_t, 16>{};
// First 6 bytes = timestamp
for(int i = 0; i < 6; ++i)
{
uuid[i] = static_cast<uint8_t>((timestamp_ms >> (40 - 8 * i)) & 0xFF);
}
// Set version to 7 with ordering based on timestamp.
uuid[6] = static_cast<uint8_t>((timestamp_ms >> 8) & 0x0F);
uuid[6] |= 0x70;
uuid[7] = static_cast<uint8_t>(timestamp_ms & 0xFF);
// Seeded RNG
auto rand64 = std::mt19937_64{seed}();
for(int i = 0; i < 8; ++i)
{
uuid[8 + i] = static_cast<uint8_t>((rand64 >> (56 - 8 * i)) & 0xFF);
}
// Set variant to RFC 4122
uuid[8] = (uuid[8] & 0x3F) | 0x80;
// Format as UUID string
auto oss = std::ostringstream{};
oss << std::hex << std::setfill('0');
for(int i = 0; i < 16; ++i)
{
oss << std::setw(2) << static_cast<int>(uuid[i]);
if(i == 3 || i == 5 || i == 7 || i == 9) oss << delim;
}
return oss.str();
}
} // namespace common
} // namespace rocprofiler
source/lib/common/uuid_v7.hpp
+49
Просмотреть файл
@@ -0,0 +1,49 @@
// MIT License
//
// Copyright (c) 2025 Advanced Micro Devices, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
#pragma once
#include "lib/common/defines.hpp"
#include "lib/common/mpl.hpp"
#include <array>
#include <cstdint>
#include <random>
#include <string>
namespace rocprofiler
{
namespace common
{
uint64_t
get_process_start_ticks_since_boot(pid_t pid);
// use this function to create a deterministic random number seed for the system and process
uint64_t
compute_system_seed(std::string_view machine_id, pid_t pid, pid_t ppid, uint64_t pstart_ticks);
std::string
generate_uuid_v7(uint64_t timestamp_ns,
uint64_t seed = std::random_device{}(),
std::string_view delim = "-");
} // namespace common
} // namespace rocprofiler
source/lib/output/CMakeLists.txt
+8
Просмотреть файл
@@ -17,6 +17,7 @@ set(TOOL_OUTPUT_HEADERS
generateOTF2.hpp
generatePerfetto.hpp
generateStats.hpp
generateRocpd.hpp
generator.hpp
kernel_symbol_info.hpp
host_symbol_info.hpp
@@ -41,6 +42,7 @@ set(TOOL_OUTPUT_SOURCES
generateOTF2.cpp
generatePerfetto.cpp
generateStats.cpp
generateRocpd.cpp
metadata.cpp
node_info.cpp
output_config.cpp
@@ -57,6 +59,7 @@ target_sources(rocprofiler-sdk-output-library PRIVATE ${TOOL_OUTPUT_SOURCES}
${TOOL_OUTPUT_HEADERS})
target_link_libraries(
rocprofiler-sdk-output-library
PUBLIC rocprofiler-sdk::rocprofiler-sdk-rocpd-library
PRIVATE rocprofiler-sdk::rocprofiler-sdk-headers
rocprofiler-sdk::rocprofiler-sdk-build-flags
rocprofiler-sdk::rocprofiler-sdk-memcheck
@@ -68,3 +71,8 @@ target_link_libraries(
rocprofiler-sdk::rocprofiler-sdk-dw
rocprofiler-sdk::rocprofiler-sdk-elf
rocprofiler-sdk::rocprofiler-sdk-sqlite3)
target_compile_definitions(rocprofiler-sdk-output-library
PRIVATE PROJECT_BINARY_DIR="${PROJECT_BINARY_DIR}")
add_subdirectory(sql)
source/lib/output/generateRocpd.cpp
+1398
Просмотреть файл
Разница между файлами не показана из-за своего большого размера Загрузить разницу
source/lib/output/generateRocpd.hpp
+76
Просмотреть файл
@@ -0,0 +1,76 @@
// MIT License
//
// Copyright (c) 2023 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#pragma once
#include "agent_info.hpp"
#include "generator.hpp"
#include "metadata.hpp"
#include "output_config.hpp"
#include "stream_info.hpp"
#include <cstdint>
#include <deque>
namespace rocprofiler
{
namespace tool
{
void
write_rocpd(
const output_config& cfg,
const metadata& tool_metadata,
const std::vector<agent_info>& agent_data,
const generator<rocprofiler_buffer_tracing_hip_api_ext_record_t>& hip_api_gen,
const generator<rocprofiler_buffer_tracing_hsa_api_record_t>& hsa_api_gen,
const generator<tool_buffer_tracing_kernel_dispatch_ext_record_t>& kernel_dispatch_gen,
const generator<tool_buffer_tracing_memory_copy_ext_record_t>& memory_copy_gen,
const generator<rocprofiler_buffer_tracing_marker_api_record_t>& marker_api_gen,
const generator<tool_buffer_tracing_memory_allocation_ext_record_t>& memory_alloc_gen,
const generator<rocprofiler_buffer_tracing_scratch_memory_record_t>& scratch_memory_gen,
const generator<rocprofiler_buffer_tracing_rccl_api_record_t>& rccl_api_gen,
const generator<rocprofiler_buffer_tracing_rocdecode_api_ext_record_t>& rocdecode_api_gen,
const generator<tool_counter_record_t>& counter_collection_gen);
// used in schema generation
struct argument_info
{
uint32_t arg_number = 0;
std::string arg_type = {};
std::string arg_name = {};
std::string arg_value = {};
};
struct track_data
{
uint64_t node_id = 0;
pid_t pid = 0;
pid_t tid = 0;
uint64_t name_id = 0;
size_t hash() const;
};
bool
operator==(const track_data& lhs, const track_data& rhs);
} // namespace tool
} // namespace rocprofiler
source/lib/output/metadata.cpp
+52 -11
Просмотреть файл
@@ -425,6 +425,25 @@ metadata::get_counter_dimension_info() const
return _ret;
}
metadata::string_index_map_t
metadata::get_string_entries() const
{
return string_entries.rlock([](const auto& _inp) {
auto _sorted = std::vector<std::string_view>{};
_sorted.reserve(_inp.size());
for(const auto& itr : _inp)
_sorted.emplace_back(std::string_view{*itr.second});
std::sort(_sorted.begin(), _sorted.end());
auto _ret = string_index_map_t{};
size_t _idx = 1;
for(const auto& itr : _sorted)
_ret.emplace(itr, _idx++);
return _ret;
});
}
bool
metadata::add_marker_message(uint64_t corr_id, std::string&& msg)
{
@@ -607,17 +626,12 @@ metadata::get_agent_index(rocprofiler_agent_id_t id, agent_indexing index) const
return "UNK";
};
switch(index)
{
case agent_indexing::node: return agent_index{"Agent", _agent->node_id, get_type()};
case agent_indexing::logical_node_type:
return agent_index{
get_type(), static_cast<uint32_t>(_agent->logical_node_type_id), get_type()};
case agent_indexing::logical_node:
default:
return agent_index{"Agent", static_cast<uint32_t>(_agent->logical_node_id), get_type()};
}
return create_agent_index(
index,
_agent->node_id, // absolute index
static_cast<uint32_t>(_agent->logical_node_id), // relative index
static_cast<uint32_t>(_agent->logical_node_type_id), // type-relative index
get_type());
}
const std::string*
@@ -691,5 +705,32 @@ metadata::decode_instruction(rocprofiler_pc_t pc)
pc.code_object_offset);
}
agent_index
create_agent_index(const rocprofiler::tool::agent_indexing index,
uint32_t agent_abs_index,
uint32_t agent_log_index,
uint32_t agent_type_index,
const std::string_view agent_type)
{
switch(index)
{
case rocprofiler::tool::agent_indexing::node: // absolute
{
return agent_index{"Agent", agent_abs_index, agent_type};
}
case rocprofiler::tool::agent_indexing::logical_node: // relative (default)
{
return agent_index{"Agent", agent_log_index, agent_type};
}
case rocprofiler::tool::agent_indexing::logical_node_type: // type-relative
{
return agent_index{agent_type, agent_type_index, agent_type};
}
}
ROCP_CI_LOG(WARNING) << fmt::format(
"Unsupported agent indexing {} for agent-{}", static_cast<int>(index), agent_abs_index);
return agent_index{};
}
} // namespace tool
} // namespace rocprofiler
source/lib/output/metadata.hpp
+30
Просмотреть файл
@@ -69,9 +69,26 @@
} \
}
#define ROCPD_CHECK_NESTED(VAR, RESULT, LEVEL) \
{ \
if(rocpd_status_t ROCPROFILER_VARIABLE(CHECKSTATUS, VAR) = RESULT; \
ROCPROFILER_VARIABLE(CHECKSTATUS, VAR) != ROCPD_STATUS_SUCCESS) \
{ \
ROCP_##LEVEL << fmt::format( \
"[{}] {} returned {} :: {}", \
__FUNCTION__, \
#RESULT, \
rocpd_get_status_name(ROCPROFILER_VARIABLE(CHECKSTATUS, VAR)), \
rocpd_get_status_string(ROCPROFILER_VARIABLE(CHECKSTATUS, VAR))); \
} \
}
#define ROCPROFILER_CHECK(RESULT) ROCPROFILER_CHECK_NESTED(__COUNTER__, RESULT, FATAL)
#define ROCPROFILER_CHECK_WARNING(RESULT) ROCPROFILER_CHECK_NESTED(__COUNTER__, RESULT, WARNING)
#define ROCPD_CHECK(RESULT) ROCPD_CHECK_NESTED(__COUNTER__, RESULT, FATAL)
#define ROCPD_CHECK_WARNING(RESULT) ROCPD_CHECK_NESTED(__COUNTER__, RESULT, WARNING)
namespace rocprofiler
{
namespace tool
@@ -118,6 +135,7 @@ struct agent_index
struct metadata
{
using agent_info_ptr_vec_t = common::container::small_vector<const agent_info*, 16>;
using string_index_map_t = std::unordered_map<std::string_view, size_t>;
struct inprocess
{};
@@ -147,6 +165,9 @@ struct metadata
node_info node_data = {};
std::vector<std::string> command_line = {};
// PMC event ids start at this number
uint64_t pmc_event_offset = 1;
metadata() = default;
metadata(inprocess);
@@ -182,6 +203,8 @@ struct metadata
void add_decoder(rocprofiler_code_object_info_t* obj_data_v);
code_object_load_info_vec_t get_code_object_load_info() const;
string_index_map_t get_string_entries() const;
template <typename Tp>
Tp get_marker_messages(Tp&&);
@@ -232,5 +255,12 @@ metadata::get_marker_messages(Tp&& _inp)
},
std::move(_inp));
}
agent_index
create_agent_index(const agent_indexing index,
uint32_t agent_abs_index,
uint32_t agent_log_index,
uint32_t agent_type_index,
const std::string_view agent_type);
} // namespace tool
} // namespace rocprofiler
source/lib/output/node_info.cpp
+113 -8
Просмотреть файл
@@ -21,10 +21,13 @@
// SOFTWARE.
#include "lib/output/node_info.hpp"
#include "lib/common/filesystem.hpp"
#include "lib/common/logging.hpp"
#include "lib/common/sha256.hpp"
#include <rocprofiler-sdk/cxx/details/tokenize.hpp>
#include <fmt/format.h>
#include <sys/utsname.h>
#include <fstream>
@@ -33,19 +36,121 @@ namespace rocprofiler
{
namespace tool
{
namespace
{
using utsname_t = struct utsname;
std::string
sha256_hex(const std::string& input)
{
auto sha = common::sha256{};
sha.update(input);
return sha.hexdigest();
}
// --- Machine ID Utility ---
std::string
read_file_first_line(const std::string& path)
{
if(auto file = std::ifstream{path}; file.is_open())
{
auto line = std::string{};
std::getline(file, line);
return line;
}
return std::string{};
}
std::string
get_mac_address(std::string_view iface)
{
if(auto mac = read_file_first_line(fmt::format("/sys/class/net/{}/address", iface));
!mac.empty())
return mac;
return {};
}
std::string
read_file(const std::string& filePath)
{
auto file = std::ifstream{filePath, std::ios::in | std::ios::binary};
if(file.is_open())
{
auto buffer = std::stringstream{};
buffer << file.rdbuf();
return buffer.str();
}
return std::string{};
}
std::string
get_mac_address(const std::vector<std::string>& interfaces = {"eth0", "enp0s3", "wlan0", "eno1"})
{
namespace fs = ::rocprofiler::common::filesystem;
auto remove_duplicates = [](auto _data) {
std::sort(_data.begin(), _data.end());
_data.erase(std::unique(_data.begin(), _data.end()), _data.end());
return _data;
};
for(std::string_view iface : interfaces)
{
if(auto mac = get_mac_address(iface); !mac.empty()) return mac;
}
for(const auto& itr : fs::directory_iterator{fs::path{"/sys/class/net"}})
{
if(auto path = fs::path{itr}; fs::exists(path / "address"))
{
if(auto mac = get_mac_address(path.filename().string()); !mac.empty())
{
// some network interfaces have generic addresses like 00:00:00:00:00:00 or
// ee:ee:ee:ee:ee:ee and we want to ignore these
if(remove_duplicates(sdk::parse::tokenize(mac, ":")).size() > 1) return mac;
}
}
}
return std::string{};
}
std::string
get_machine_id()
{
// not all Linux distributions have /etc/machine-id so we need to fallback on various
// alternatives to try to uniquely identify the system
if(std::string id = read_file_first_line("/etc/machine-id"); !id.empty()) return id;
if(std::string id = read_file_first_line("/var/lib/dbus/machine-id"); !id.empty()) return id;
//
// for all values beyond this point, encrypt the id with sha256 since this is potentially
// sensitive information. prefix is used for salt separation
//
if(std::string id = read_file_first_line("/sys/class/dmi/id/product_uuid"); !id.empty())
return sha256_hex(fmt::format("product_uuid:{}", id));
if(std::string id = read_file_first_line("/sys/class/dmi/id/board_serial"); !id.empty())
return sha256_hex(fmt::format("board_serial:{}", id));
if(std::string id = read_file("/proc/cpuinfo") + read_file("/proc/version") +
read_file("/proc/devices") + read_file("/proc/filesystems");
!id.empty())
return sha256_hex(fmt::format("procinfo:{}", id));
if(std::string id = get_mac_address(); !id.empty())
return sha256_hex(fmt::format("mac_address:{}", id));
return std::string{};
}
} // namespace
node_info&
read_node_info(node_info& _info)
{
{
if(auto ifs = std::ifstream{"/etc/machine-id"})
{
auto _mach_id = std::string{};
if((ifs >> _mach_id) && !_mach_id.empty())
_info.machine_id = sdk::parse::strip(std::move(_mach_id), "\n\t\r ");
}
}
_info.machine_id = get_machine_id();
auto _sys_info = utsname_t{};
if(uname(&_sys_info) == 0)
source/lib/output/node_info.hpp
-1
Просмотреть файл
@@ -22,7 +22,6 @@
#pragma once
#include <rocprofiler-sdk/agent.h>
#include <rocprofiler-sdk/cxx/serialization.hpp>
#include <unordered_map>
source/lib/output/output_config.cpp
+2 -1
Просмотреть файл
@@ -74,10 +74,11 @@ output_config::parse_env()
for(const auto& itr : sdk::parse::tokenize(output_format, " \t,;:"))
entries.emplace(to_upper(itr));
csv_output = entries.count("CSV") > 0 || entries.empty();
csv_output = entries.count("CSV") > 0;
json_output = entries.count("JSON") > 0;
pftrace_output = entries.count("PFTRACE") > 0;
otf2_output = entries.count("OTF2") > 0;
rocpd_output = entries.count("ROCPD") > 0 || entries.empty();
const auto supported_formats =
std::set<std::string_view>{"CSV", "JSON", "PFTRACE", "OTF2", "ROCPD"};
source/lib/output/output_config.hpp
+2
Просмотреть файл
@@ -67,6 +67,7 @@ struct output_config
bool json_output = false;
bool pftrace_output = false;
bool otf2_output = false;
bool rocpd_output = false;
bool summary_output = false;
bool kernel_rename = false;
bool group_by_queue = false;
@@ -129,6 +130,7 @@ output_config::save(ArchiveT& ar) const
CFG_SERIALIZE_MEMBER(pftrace_output);
CFG_SERIALIZE_MEMBER(otf2_output);
CFG_SERIALIZE_MEMBER(summary_output);
CFG_SERIALIZE_MEMBER(rocpd_output);
CFG_SERIALIZE_MEMBER(kernel_rename);
CFG_SERIALIZE_MEMBER(group_by_queue);
source/lib/output/sql/CMakeLists.txt
+8
Просмотреть файл
@@ -0,0 +1,8 @@
#
# add sql common sources to output library target
#
set(output_sql_headers common.hpp deferred_transaction.hpp extract_data_type.hpp)
set(output_sql_sources common.cpp deferred_transaction.cpp)
target_sources(rocprofiler-sdk-output-library PRIVATE ${output_sql_sources}
${output_sql_headers})
source/lib/output/sql/common.cpp
+205
Просмотреть файл
@@ -0,0 +1,205 @@
// MIT License
//
// Copyright (c) 2025 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#include "lib/output/sql/common.hpp"
#include "lib/output/kernel_symbol_info.hpp"
#include "lib/common/logging.hpp"
#include "lib/common/scope_destructor.hpp"
#include <rocprofiler-sdk/cxx/details/tokenize.hpp>
#include <rocprofiler-sdk/cxx/hash.hpp>
#include <rocprofiler-sdk/cxx/operators.hpp>
#include <fmt/format.h>
#include <sqlite3.h>
#include <iomanip>
#include <sstream>
#include <thread>
namespace rocprofiler
{
namespace tool
{
namespace sql
{
namespace sdk = ::rocprofiler::sdk;
void
check(std::string_view function, int status, std::string_view stmt)
{
if(status != SQLITE_OK)
{
ROCP_FATAL << "[" << function << "] " << stmt << " failed with error code " << status;
}
}
int
busy_handler(void* /*data*/, int count)
{
count = (count < 9) ? count : 8;
std::this_thread::sleep_for(std::chrono::microseconds{1000 * (0x1 << count)});
return 1;
}
// invoked during SELECT operation; unused but kept for reference
int
exec_callback(void* user_data, int ncols, char** coltext, char** colnames)
{
ROCP_INFO << "SQL callback invoked with " << ncols << " columns";
if(!coltext || !colnames) return SQLITE_OK;
auto header = std::stringstream{};
auto div = std::stringstream{};
auto content = std::stringstream{};
header << fmt::format(
"| {} |", fmt::join(std::vector<std::string_view>(colnames, colnames + ncols), " | "));
div << fmt::format("|-{}-|", std::string(header.str().size() - 4, '-'));
content << fmt::format(
"| {} |", fmt::join(std::vector<std::string_view>(coltext, coltext + ncols), " | "));
ROCP_WARNING << "SQL callback for " << ncols << " columns contents: "
<< "\n\t" << header.str() << "\n\t" << div.str() << "\n\t" << content.str();
return SQLITE_OK;
(void) user_data;
}
int64_t
execute_raw_sql_statements_impl(sqlite3* conn,
std::string_view stmts,
exec_callback_t callback,
void* data,
int line)
{
int64_t row_id = -1;
// NOLINTNEXTLINE(performance-for-range-copy)
for(auto stmt : sdk::parse::tokenize(stmts, ";"))
{
stmt = fmt::format("{};", std::move(stmt)); // make sure ends with semi-colon
ROCP_TRACE << "Executing SQLite3 statement: " << stmt;
char* msg = nullptr;
auto ret = sqlite3_exec(conn, stmt.c_str(), callback, data, &msg);
if(ret != SQLITE_OK)
{
// ensure no memory leak
auto dtor = common::scope_destructor{[msg]() {
if(msg) sqlite3_free(msg);
}};
static constexpr auto full_file = std::string_view{__FILE__};
static constexpr auto base_file = full_file.substr(full_file.find_last_of('/') + 1);
ROCP_FATAL << "SQLite3 error " << ret << ": " << ((msg) ? msg : "unknown error")
<< "\n\tSQLite3 error: " << base_file << ":" << line
<< "\n\tSQL Statement: " << stmt;
}
else
{
if(stmt.find("INSERT") != std::string::npos) row_id = sqlite3_last_insert_rowid(conn);
}
}
return row_id;
}
int64_t
execute_raw_sql_statements_impl(sqlite3* conn, std::string_view stmts, int line)
{
return execute_raw_sql_statements_impl(conn, stmts, exec_callback, nullptr, line);
}
std::string
extract_column_name(sqlite3_stmt* stmt, int32_t col)
{
return std::string{sqlite3_column_name(stmt, col)};
}
int64_t
extract_row_count(sqlite3* conn, std::string_view query)
{
auto _pos = query.find(';');
auto _query = (_pos == std::string_view::npos) ? query : query.substr(0, _pos);
auto _count_query = fmt::format("SELECT COUNT(*) AS count FROM ({}) x;", _query);
sqlite3_stmt* _stmt = nullptr;
if(sqlite3_prepare_v2(conn, _count_query.c_str(), -1, &_stmt, nullptr) != SQLITE_OK)
{
ROCP_CI_LOG(ERROR) << fmt::format(
"Error preparing select statement for '{}': {}", _count_query, sqlite3_errmsg(conn));
return 0;
}
ROCP_CI_LOG_IF(ERROR, _stmt == nullptr) << "Error preparing statment: " << query;
int64_t nrows = 0;
if(_stmt && sqlite3_column_count(_stmt) == 1 && sqlite3_step(_stmt) == SQLITE_ROW)
nrows = extract_column<int64_t>(_stmt, 0).value_or(0);
sqlite3_finalize(_stmt); // Finalize statement
ROCP_INFO << fmt::format("SQL query '{}' contains {} rows", query, nrows);
return nrows;
}
namespace
{
template <int Idx>
struct sql_data_type_info;
#define SPECIALIZE_SQL_DATA_TYPE_INFO(VALUE) \
template <> \
struct sql_data_type_info<VALUE> \
{ \
static constexpr auto value = VALUE; \
static constexpr auto name = #VALUE; \
};
SPECIALIZE_SQL_DATA_TYPE_INFO(SQLITE_INTEGER)
SPECIALIZE_SQL_DATA_TYPE_INFO(SQLITE_FLOAT)
SPECIALIZE_SQL_DATA_TYPE_INFO(SQLITE_BLOB)
SPECIALIZE_SQL_DATA_TYPE_INFO(SQLITE_NULL)
SPECIALIZE_SQL_DATA_TYPE_INFO(SQLITE_TEXT)
template <size_t Idx, size_t... Tail>
std::string_view
get_sql_data_type(int val, std::index_sequence<Idx, Tail...>)
{
using info_type = sql_data_type_info<Idx + 1>;
if(val == info_type::value) return std::string_view{info_type::name};
if constexpr(sizeof...(Tail) > 0) return get_sql_data_type(val, std::index_sequence<Tail...>{});
return std::string_view{"SQLITE_UNKNOWN"};
}
} // namespace
std::string_view
get_sql_data_type(int val)
{
return get_sql_data_type(val, std::make_index_sequence<SQLITE_NULL>{});
}
} // namespace sql
} // namespace tool
} // namespace rocprofiler
source/lib/output/sql/common.hpp
+129
Просмотреть файл
@@ -0,0 +1,129 @@
// MIT License
//
// Copyright (c) 2025 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#pragma once
#include "lib/output/sql/extract_data_type.hpp"
#include "lib/common/container/ring_buffer.hpp"
#include "lib/common/mpl.hpp"
#include "lib/common/units.hpp"
#include <fmt/format.h>
#include <sqlite3.h>
#include <chrono>
#include <cstdint>
#include <string_view>
#include <type_traits>
namespace rocprofiler
{
namespace tool
{
namespace sql
{
using exec_callback_t = int (*)(void* user_data, int ncols, char** coltext, char** colnames);
template <typename Tp>
using ring_buffer_t = rocprofiler::common::container::ring_buffer<Tp>;
int
exec_callback(void* user_data, int ncols, char** coltext, char** colnames);
void
check(std::string_view function, int status, std::string_view stmt);
int
busy_handler(void* data, int count);
int64_t
execute_raw_sql_statements_impl(sqlite3* conn, std::string_view stmts, int line);
int64_t
execute_raw_sql_statements_impl(sqlite3* conn,
std::string_view stmts,
exec_callback_t callback,
void* data,
int line);
std::string
extract_column_name(sqlite3_stmt* stmt, int32_t col);
int64_t
extract_row_count(sqlite3* conn, std::string_view query);
std::string_view
get_sql_data_type(int val);
template <typename Tp>
auto
extract_column(sqlite3_stmt* stmt, int32_t col);
template <typename Tp, int ExpectedV>
bool
column_data_is_null(sqlite3_stmt* stmt, int32_t col);
//
//
// Template function definitions
//
//
template <typename Tp>
auto
extract_column(sqlite3_stmt* stmt, int32_t col)
{
return extract_data_type<Tp>{}(stmt, col);
}
template <typename Tp, int ExpectedV>
bool
column_data_is_null(sqlite3_stmt* stmt, int32_t col)
{
auto coltype = sqlite3_column_type(stmt, col);
if(coltype == SQLITE_NULL) return true;
std::string column_name = extract_column_name(stmt, col);
const char* sql_text = sqlite3_sql(stmt);
ROCP_CI_LOG_IF(WARNING, coltype != ExpectedV) << fmt::format(
"Data in SQL column {} ('{}') is neither NULL nor the expected data type ({} == {}). "
"Column data type is: ({} == {}), SQL: {}",
col,
column_name,
ExpectedV,
get_sql_data_type(ExpectedV),
coltype,
get_sql_data_type(coltype),
sql_text);
return false;
}
} // namespace sql
} // namespace tool
} // namespace rocprofiler
#define execute_raw_sql_statements(...) \
::rocprofiler::tool::sql::execute_raw_sql_statements_impl(__VA_ARGS__, __LINE__)
#define SQLITE3_CHECK(RESULT) \
::rocprofiler::tool::sql::check(__FUNCTION__, (RESULT), std::string_view{#RESULT})
source/lib/output/sql/deferred_transaction.cpp
+62
Просмотреть файл
@@ -0,0 +1,62 @@
// MIT License
//
// Copyright (c) 2025 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#include "lib/output/sql/deferred_transaction.hpp"
#include "lib/output/sql/common.hpp"
#include "lib/common/logging.hpp"
#include "lib/common/scope_destructor.hpp"
#include <fmt/format.h>
#include <sqlite3.h>
#include <iomanip>
#include <sstream>
#include <thread>
namespace rocprofiler
{
namespace tool
{
namespace sql
{
deferred_transaction::deferred_transaction(sqlite3* conn)
: m_conn{conn}
{
ROCP_CI_LOG_IF(INFO, m_conn == nullptr) << "rocprofiler::tool::sql::deferred_transaction "
"constructed will nullptr to sqlite3 connection";
if(m_conn)
{
execute_raw_sql_statements(m_conn, "BEGIN DEFERRED TRANSACTION");
}
}
deferred_transaction::~deferred_transaction()
{
if(m_conn)
{
execute_raw_sql_statements(m_conn, "END TRANSACTION");
}
}
} // namespace sql
} // namespace tool
} // namespace rocprofiler
source/lib/output/sql/deferred_transaction.hpp
+45
Просмотреть файл
@@ -0,0 +1,45 @@
// MIT License
//
// Copyright (c) 2025 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#pragma once
#include "lib/output/sql/common.hpp"
#include <sqlite3.h>
namespace rocprofiler
{
namespace tool
{
namespace sql
{
struct deferred_transaction
{
explicit deferred_transaction(sqlite3* conn);
~deferred_transaction();
private:
sqlite3* m_conn = nullptr;
};
} // namespace sql
} // namespace tool
} // namespace rocprofiler
source/lib/output/sql/extract_data_type.hpp
+141
Просмотреть файл
@@ -0,0 +1,141 @@
// MIT License
//
// Copyright (c) 2025 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#pragma once
#include "lib/common/mpl.hpp"
#include <sqlite3.h>
#include <array>
#include <cstdint>
#include <optional>
#include <string>
#include <string_view>
#include <type_traits>
namespace rocprofiler
{
namespace tool
{
namespace sql
{
template <typename Tp, typename CondT = void>
struct extract_data_type;
template <typename Tp, int ExpectedV>
bool
column_data_is_null(sqlite3_stmt* stmt, int32_t col);
template <typename Tp>
struct extract_data_type<Tp, std::enable_if_t<std::is_integral<Tp>::value>>
{
static constexpr auto value = SQLITE_INTEGER;
std::optional<Tp> operator()(sqlite3_stmt* stmt, int32_t col) const
{
if(column_data_is_null<Tp, value>(stmt, col)) return std::nullopt;
if constexpr(std::is_signed<Tp>::value)
{
if constexpr(sizeof(Tp) > sizeof(int32_t))
return Tp{sqlite3_column_int64(stmt, col)};
else
return Tp{sqlite3_column_int(stmt, col)};
}
else
{
auto val = sqlite3_column_int64(stmt, col);
return static_cast<Tp>(val);
}
}
};
template <typename Tp>
struct extract_data_type<Tp, std::enable_if_t<std::is_floating_point<Tp>::value>>
{
static constexpr auto value = SQLITE_FLOAT;
std::optional<Tp> operator()(sqlite3_stmt* stmt, int32_t col) const
{
if(column_data_is_null<Tp, value>(stmt, col)) return std::nullopt;
return Tp{sqlite3_column_double(stmt, col)};
}
};
template <typename Tp>
struct extract_data_type<Tp, std::enable_if_t<common::mpl::is_string_type<Tp>::value>>
{
static constexpr auto value = SQLITE_TEXT;
std::optional<Tp> operator()(sqlite3_stmt* stmt, int32_t col) const
{
if(column_data_is_null<Tp, value>(stmt, col)) return std::nullopt;
const auto* ret = reinterpret_cast<const char*>(sqlite3_column_text(stmt, col));
if constexpr(std::is_constructible<Tp, const char*>::value) return Tp{ret};
return ret;
}
};
template <typename Tp, size_t N>
struct extract_data_type<
std::array<Tp, N>,
std::enable_if_t<std::is_integral<Tp>::value && sizeof(Tp) == sizeof(uint8_t)>>
{
static constexpr auto value = SQLITE_BLOB;
using value_type = std::array<Tp, N>;
std::optional<value_type> operator()(sqlite3_stmt* stmt, int32_t col) const
{
if(column_data_is_null<value_type, value>(stmt, col)) return std::nullopt;
const auto* val = reinterpret_cast<const Tp*>(sqlite3_column_blob(stmt, col));
auto ret = value_type{};
ret.fill(0);
uint64_t nbytes = std::min<uint64_t>(sqlite3_column_bytes(stmt, col), ret.size());
for(uint64_t i = 0; i < nbytes; ++i)
ret.at(i) = val[i];
return ret;
}
};
template <typename Tp>
struct extract_data_type<Tp, std::enable_if_t<std::is_same<Tp, std::nullptr_t>::value>>
{
static constexpr auto value = SQLITE_TEXT;
std::optional<Tp> operator()(sqlite3_stmt* stmt, int32_t col) const
{
if(column_data_is_null<Tp, value>(stmt, col)) return std::nullopt;
return std::nullptr_t{};
}
};
} // namespace sql
} // namespace tool
} // namespace rocprofiler
source/lib/python/utilities.cmake
+2 -2
Просмотреть файл
@@ -101,7 +101,7 @@ function(rocprofiler_roctx_python_bindings _VERSION)
install(
FILES ${roctx_PYTHON_OUTPUT_DIRECTORY}/${_SOURCE}
DESTINATION ${roctx_PYTHON_INSTALL_DIRECTORY}
COMPONENT core)
COMPONENT roctx)
endforeach()
add_library(rocprofiler-sdk-roctx-python-bindings-${_VERSION} MODULE)
@@ -129,7 +129,7 @@ function(rocprofiler_roctx_python_bindings _VERSION)
install(
TARGETS rocprofiler-sdk-roctx-python-bindings-${_VERSION}
DESTINATION ${roctx_PYTHON_INSTALL_DIRECTORY}
COMPONENT core)
COMPONENT roctx)
endfunction()
function(rocprofiler_rocpd_python_bindings_target_sources _VERSION)
source/lib/rocprofiler-sdk-rocpd/CMakeLists.txt
+46
Просмотреть файл
@@ -0,0 +1,46 @@
#
# ROCm Profiling Data (rocpd) Library
#
set(CMAKE_INSTALL_DEFAULT_COMPONENT_NAME "rocpd")
add_library(rocprofiler-sdk-rocpd-shared-library SHARED)
foreach(_NAMESPACE rocprofiler-sdk rocprofiler-sdk-rocpd)
foreach(_ALIAS library shared-library)
add_library(${_NAMESPACE}::rocprofiler-sdk-rocpd-${_ALIAS} ALIAS
rocprofiler-sdk-rocpd-shared-library)
endforeach()
endforeach()
target_sources(rocprofiler-sdk-rocpd-shared-library PRIVATE rocpd.cpp sql.cpp)
target_include_directories(
rocprofiler-sdk-rocpd-shared-library
INTERFACE $<BUILD_INTERFACE:${PROJECT_BINARY_DIR}/source/include>
$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/source/include>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>)
target_link_libraries(
rocprofiler-sdk-rocpd-shared-library
PRIVATE rocprofiler-sdk::rocprofiler-sdk-headers
rocprofiler-sdk::rocprofiler-sdk-build-flags
rocprofiler-sdk::rocprofiler-sdk-memcheck
rocprofiler-sdk::rocprofiler-sdk-common-library
rocprofiler-sdk::rocprofiler-sdk-dl)
set_target_properties(
rocprofiler-sdk-rocpd-shared-library
PROPERTIES OUTPUT_NAME rocprofiler-sdk-rocpd
LIBRARY_OUTPUT_DIRECTORY ${PROJECT_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}
SOVERSION ${PROJECT_VERSION_MAJOR}
VERSION ${PROJECT_VERSION}
SKIP_BUILD_RPATH OFF
BUILD_RPATH "\$ORIGIN"
INSTALL_RPATH "\$ORIGIN"
DEFINE_SYMBOL rocpd_EXPORTS)
install(
TARGETS rocprofiler-sdk-rocpd-shared-library
DESTINATION ${CMAKE_INSTALL_LIBDIR}
COMPONENT rocpd
EXPORT rocprofiler-sdk-rocpd-targets)
source/lib/rocprofiler-sdk-rocpd/rocpd.cpp
+117
Просмотреть файл
@@ -0,0 +1,117 @@
// MIT License
//
// Copyright (c) 2023-2025 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#include "lib/common/logging.hpp"
#include "lib/common/static_object.hpp"
#include "lib/common/utility.hpp"
#include <rocprofiler-sdk-rocpd/defines.h>
#include <rocprofiler-sdk-rocpd/rocpd.h>
#include <rocprofiler-sdk-rocpd/types.h>
#include <array>
#include <atomic>
#include <cassert>
namespace rocpd
{
namespace
{
#define ROCPD_STATUS_STRING(CODE, MSG) \
template <> \
struct status_string<CODE> \
{ \
static constexpr auto name = #CODE; \
static constexpr auto value = MSG; \
};
template <size_t Idx>
struct status_string;
ROCPD_STATUS_STRING(ROCPD_STATUS_SUCCESS, "Success")
ROCPD_STATUS_STRING(ROCPD_STATUS_ERROR, "General error")
ROCPD_STATUS_STRING(ROCPD_STATUS_ERROR_INVALID_ARGUMENT, "Invalid function argument")
ROCPD_STATUS_STRING(ROCPD_STATUS_ERROR_SQL_ERROR, "General SQL error")
ROCPD_STATUS_STRING(ROCPD_STATUS_ERROR_SQL_INVALID_ENGINE, "Invalid SQL engine")
ROCPD_STATUS_STRING(ROCPD_STATUS_ERROR_SQL_INVALID_SCHEMA_KIND, "Invalid SQL schema kind")
ROCPD_STATUS_STRING(ROCPD_STATUS_ERROR_SQL_SCHEMA_NOT_FOUND, "SQL schema not found")
ROCPD_STATUS_STRING(ROCPD_STATUS_ERROR_SQL_SCHEMA_PERMISSION_DENIED, "SQL schema could not be read")
template <size_t Idx, size_t... Tail>
const char*
get_status_name(rocpd_status_t status, std::index_sequence<Idx, Tail...>)
{
if(status == Idx) return status_string<Idx>::name;
// recursion until tail empty
if constexpr(sizeof...(Tail) > 0)
return get_status_name(status, std::index_sequence<Tail...>{});
return nullptr;
}
template <size_t Idx, size_t... Tail>
const char*
get_status_string(rocpd_status_t status, std::index_sequence<Idx, Tail...>)
{
if(status == Idx) return status_string<Idx>::value;
// recursion until tail empty
if constexpr(sizeof...(Tail) > 0)
return get_status_string(status, std::index_sequence<Tail...>{});
return nullptr;
}
} // namespace
// force initialization of logging on library load
bool _rocpd_init_logging = (rocprofiler::common::init_logging("ROCPD"), true);
} // namespace rocpd
ROCPD_EXTERN_C_INIT
rocpd_status_t
rocpd_get_version(uint32_t* major, uint32_t* minor, uint32_t* patch)
{
if(major) *major = ROCPD_VERSION_MAJOR;
if(minor) *minor = ROCPD_VERSION_MINOR;
if(patch) *patch = ROCPD_VERSION_PATCH;
return ROCPD_STATUS_SUCCESS;
}
rocpd_status_t
rocpd_get_version_triplet(rocpd_version_triplet_t* info)
{
*info = {
.major = ROCPD_VERSION_MAJOR, .minor = ROCPD_VERSION_MINOR, .patch = ROCPD_VERSION_PATCH};
return ROCPD_STATUS_SUCCESS;
}
const char*
rocpd_get_status_name(rocpd_status_t status)
{
return rocpd::get_status_name(status, std::make_index_sequence<ROCPD_STATUS_LAST>{});
}
const char*
rocpd_get_status_string(rocpd_status_t status)
{
return rocpd::get_status_string(status, std::make_index_sequence<ROCPD_STATUS_LAST>{});
}
ROCPD_EXTERN_C_FINI
source/lib/rocprofiler-sdk-rocpd/sql.cpp
+247
Просмотреть файл
@@ -0,0 +1,247 @@
// MIT License
//
// Copyright (c) 2023-2025 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#define _GNU_SOURCE 1
#include "lib/common/environment.hpp"
#include "lib/common/filesystem.hpp"
#include "lib/common/logging.hpp"
#include "lib/common/mpl.hpp"
#include "lib/common/string_entry.hpp"
#include "lib/common/utility.hpp"
#include <rocprofiler-sdk-rocpd/rocpd.h>
#include <rocprofiler-sdk-rocpd/sql.h>
#include <rocprofiler-sdk-rocpd/types.h>
#include <rocprofiler-sdk/cxx/details/tokenize.hpp>
#include <fmt/core.h>
#include <fmt/ranges.h>
#include <dlfcn.h>
#include <initializer_list>
#include <unordered_map>
namespace rocpd
{
namespace sql
{
namespace
{
namespace common = ::rocprofiler::common;
namespace fs = ::rocprofiler::common::filesystem;
std::string
get_install_path()
{
auto* _rocpd_sql_load_schema_sym = dlsym(RTLD_DEFAULT, "rocpd_sql_load_schema");
ROCP_CI_LOG_IF(WARNING, !_rocpd_sql_load_schema_sym)
<< "[rocprofiler-sdk-rocpd] dlsym(RTLD_DEFAULT, 'rocpd_sql_load_schema') failed "
"(unexpectedly) from within the rocprofiler-sdk-rocpd library";
if(!_rocpd_sql_load_schema_sym)
_rocpd_sql_load_schema_sym = reinterpret_cast<void*>(&rocpd_sql_load_schema);
if(Dl_info dl_info = {};
dladdr(_rocpd_sql_load_schema_sym, &dl_info) != 0 && dl_info.dli_fname != nullptr)
{
auto _share_path =
fs::path{dl_info.dli_fname}.lexically_normal().parent_path().parent_path() /
std::string{"share/rocprofiler-sdk-rocpd"};
ROCP_INFO << fmt::format("[rocprofiler-sdk-rocpd] resolved rocprofiler-sdk-rocpd SQL "
"schema path as '{}' (dli_fname: {})",
_share_path.string(),
dl_info.dli_fname);
return _share_path;
}
ROCP_CI_LOG(WARNING)
<< "Failed to locate the installation path of rocprofiler-sdk-rocpd via dladdr of the "
"'rocpd_sql_load_schema' symbol (which should be in librocprofiler-sdk-rocpd.so)";
return std::string{};
}
template <typename Tp>
auto
replace_all(std::string val, Tp from, std::string_view to)
{
size_t pos = 0;
while((pos = val.find(from, pos)) != std::string::npos)
{
if constexpr(std::is_same<common::mpl::unqualified_type_t<Tp>, char>::value)
{
val.replace(pos, 1, to);
pos += to.length();
}
else
{
val.replace(pos, std::string_view{from}.length(), to);
pos += to.length();
}
}
return val;
}
} // namespace
} // namespace sql
} // namespace rocpd
extern "C" {
rocpd_status_t
rocpd_sql_load_schema(rocpd_sql_engine_t engine,
rocpd_sql_schema_kind_t kind,
rocpd_sql_options_t options,
const rocpd_sql_schema_jinja_variables_t* variables,
rocpd_sql_load_schema_cb_t callback,
const char** schema_path_hints,
uint64_t num_schema_path_hints,
void* user_data)
{
namespace fs = ::rocpd::sql::fs;
switch(engine)
{
case ROCPD_SQL_ENGINE_SQLITE3:
{
break;
}
case ROCPD_SQL_ENGINE_NONE:
case ROCPD_SQL_ENGINE_LAST:
{
return ROCPD_STATUS_ERROR_SQL_INVALID_ENGINE;
}
}
const auto kind_file_names = std::unordered_map<rocpd_sql_schema_kind_t, std::string_view>{
{ROCPD_SQL_SCHEMA_ROCPD_TABLES, "rocpd_tables.sql"},
{ROCPD_SQL_SCHEMA_ROCPD_INDEXES, "rocpd_indexes.sql"},
{ROCPD_SQL_SCHEMA_ROCPD_VIEWS, "rocpd_views.sql"},
{ROCPD_SQL_SCHEMA_ROCPD_DATA_VIEWS, "data_views.sql"},
{ROCPD_SQL_SCHEMA_ROCPD_SUMMARY_VIEWS, "summary_views.sql"},
{ROCPD_SQL_SCHEMA_ROCPD_MARKER_VIEWS, "marker_views.sql"},
};
const auto _lib_schema_path = rocpd::sql::get_install_path();
const auto _env_schema_path = rocprofiler::common::get_env("ROCPD_SCHEMA_PATH", "");
const auto _usr_schema_path =
(schema_path_hints)
? fmt::format(
"{}",
fmt::join(schema_path_hints, schema_path_hints + num_schema_path_hints, ":"))
: std::string{};
const auto _schema_paths =
fmt::format("{}:{}:{}", _usr_schema_path, _env_schema_path, _lib_schema_path);
if(kind_file_names.count(kind) == 0) return ROCPD_STATUS_ERROR_SQL_INVALID_SCHEMA_KIND;
auto _schema_file = std::optional<std::string>{};
for(const auto& itr : rocprofiler::sdk::parse::tokenize(_schema_paths, ":"))
{
auto _fpath = fs::path{itr} / kind_file_names.at(kind);
ROCP_TRACE << fmt::format("[rocprofiler-sdk-rocpd] Searching for schema file: '{}'",
_fpath.string());
if(fs::exists(_fpath))
{
ROCP_INFO << fmt::format("[rocprofiler-sdk-rocpd] Found schema file: '{}'",
_fpath.string());
_schema_file = _fpath;
break;
}
}
if(!_schema_file) return ROCPD_STATUS_ERROR_SQL_SCHEMA_NOT_FOUND;
auto read_file = [](const std::string& _file_path) -> std::string {
auto _ifs = std::ifstream{_file_path, std::ios::in | std::ios::binary};
if(!_ifs.is_open()) return std::string{};
auto _buffer = std::stringstream{};
_buffer << _ifs.rdbuf();
return _buffer.str();
};
auto _contents = read_file(*_schema_file);
if(_contents.empty()) return ROCPD_STATUS_ERROR_SQL_SCHEMA_PERMISSION_DENIED;
if(engine == ROCPD_SQL_ENGINE_SQLITE3)
{
if((options & ROCPD_SQL_OPTIONS_SQLITE3_PRAGMA_FOREIGN_KEYS) ==
ROCPD_SQL_OPTIONS_SQLITE3_PRAGMA_FOREIGN_KEYS)
_contents = fmt::format("PRAGMA foreign_keys = ON;\n\n{}", _contents);
}
auto _substitutions = std::vector<std::pair<std::string_view, std::string>>{};
using jinja_init_list_t = std::initializer_list<std::pair<std::string_view, const char*>>;
if(variables != nullptr)
{
if(variables->size == 0)
{
return ROCPD_STATUS_ERROR_SQL_INVALID_SCHEMA_KIND;
}
// {{uuid}} is used in table names and require special handling
if(const auto* value = variables->uuid; value != nullptr)
{
auto _value = std::string{value};
// non-empty strings are prefixed with underscore for readability
if(!_value.empty() && _value.find('_') != 0) _value = fmt::format("_{}", _value);
// replace hyphens with underscores since these are used in table/view names
if(_value.find('-') != std::string::npos)
_value = rocpd::sql::replace_all(_value, "-", "_");
// make substitutions
_contents = rocpd::sql::replace_all(_contents, "{{uuid}}", _value);
}
// make substitutions for remaining variables which do not require special handling like
// {{uuid}}
for(auto [key, value] : jinja_init_list_t{{"{{guid}}", variables->guid}})
{
if(value != nullptr)
{
_contents = rocpd::sql::replace_all(_contents, key, std::string_view{value});
}
}
}
const auto* cb_schema_path = _schema_file->c_str();
const auto* cb_schema_contents = _contents.c_str();
if((options & ROCPD_SQL_OPTIONS_SQLITE3_PRAGMA_FOREIGN_KEYS) ==
ROCPD_SQL_OPTIONS_SQLITE3_PRAGMA_FOREIGN_KEYS)
{
cb_schema_path = rocprofiler::common::get_string_entry(cb_schema_path)->c_str();
cb_schema_contents = rocprofiler::common::get_string_entry(cb_schema_contents)->c_str();
}
callback(engine, kind, options, variables, cb_schema_path, cb_schema_contents, user_data);
return ROCPD_STATUS_SUCCESS;
}
}
source/lib/rocprofiler-sdk-tool/rocprofv3-multi-node.md
-335
Просмотреть файл
@@ -1,335 +0,0 @@
# rocprofv3 Multi-Node Profiling Data
## Overview
- rocprofv3 adds supports for a `--output-format rocpd` option which enables writing a SQLite database file (one per process) with the collected data
- Use SQL schema from `rocpd` initially to support the rocpd post-processing analysis support
- In order to visualize the data, users will convert the database(s) to their desired visualization formats
- SQL has a relatively easy way to treat multiple separate databases as one database via views
- rocprofv3 provides some command-line tools built on top of a python package designed for post-processing our databases
### Skills Required for Tasks
1. Rework rocprofv3 tool library output functions
- __C++__: output functions written in C++ (`^/source/lib/rocprofiler-sdk-tool/generate*`)
- __CMake__: move the output functions into stand-alone library
2. Create Python package skeleton in `^/source/lib/python`
- __Python__: organizing a Python package to be importable (`import rocpd`) and executable (i.e. `python -m rocpd --help`)
3. Adding rocprofv3 SQLite support
- __C++__: just a general skill requirement for working with rocprofiler-sdk
- __CMake__: for integrating SQLite and python bindings into rocprofiler-sdk build
- __SQL__: understanding of SQL statement meanings, knowledge of `rocpd` SQL schema
4. Python bindings for output functions
- __C++__: just a general skill requirement for working with rocprofiler-sdk
- __PyBind11__: for writing Python bindings
#### Task #1: Rework `rocprofv3` Tool Library Output Functions
The problems with most of the output functions are:
- Problem: Access global memory via `tool_table` functions
- Global memory access won't work well for invocation of these functions via Python bindings
- Ideally, these functions should be written in the (pseudo-) functional programming style, i.e., function only accesses memory of arguments, communicates via return value, and avoids concepts like shared states but without restrictions such as immutable data arguments
- Problem: Require all the profiling data to be loaded into memory
- During runtime, rocprofv3 writes data to buffer and when buffer is full, writes the binary blob to a temporary intermediate binary file
- During finalization, rocprofv3 reads _all_ of this data back into memory from the intermediate binary file and then writes to various output forms
- This approach will not work when amount of collected data exceeds amount of available RAM, especially on systems with swap disabled; e.g., 1 TB of profiling data on system with 128 GB of RAM
- We need to be able to stream data in chunks to these output functions
- Proposed approach: function which creates a file handle, function which writes chunk of data to file (invoked multiple times), function which closes file handle
> Assigned: Markus, Olha, Jin, Araceli (i.e. onboarding group task) + Jonathan (CMake part)
##### Tasks
1. Move the `source/lib/rocprofiler-sdk-tool/generate*.{hpp,cpp}` functions into standalone (static) library: `source/lib/tool-data`
- May require `source/lib/tool-common` (static) library if something is needed by both `tool-data` and `rocprofiler-sdk-tool` libraries
- Please consult if you have any questions about where to put things and/or naming conventions
- Pay attention to existing CMake and use similar style
- We will link this library into `rocprofiler-sdk-tool` and link it into Python bindings library
2. Solve global memory access problem
- Probably need some additional data structures which represents the data currently stored/accessed from global memory which will be passed into function.
### Python Package for Converting Databases to Other Output Formats
> __Note__: We could potentially reuse `rocpd` for the the python package name since "ROCm Profiling Data" is a pretty appropriate name.
rocprofv3 will need to rework the output functions within the `librocprofiler-sdk-tool.so` library (underlying library used by `rocprofv3`) in order to support Python bindings.
For example, `generateJSON(...)` currently fetches info from global memory stored during the run, we need these functions to be pure: the only memory operated on is from the function arguments.
Furthermore, these output functions need to support partial writes: invocations with only a subset of the data so that all the data need not be loaded into memory at one time.
> __Example__ (workflow): get handle to output format, e.g. a Perfetto session, invoke `generatePerfetto(...)` with some of the data, repeat until all data has been passed, close handle to the output format.
These reworked functions should be moved to another library, e.g. `librocprofiler-sdk-tool-io.(a|so)`.
Once the output functions are isolated and functional, we need to generate python bindings (via PyBind11) so that a python package can be built on top of them.
Various command-line tools can be provided using `__main__.py` file(s) within our python package.
Users can use the python package to write their own scripts.
> __Example__ (two databases, one Perfetto trace): `rocprofv3-merge --output-format pftrace --out mybenchmark.pftrace --in results-1000.db results-1001.db`
### Treating multiple SQL databases as one database
```python
conn = sqlite3.connect('db1.db')
conn.execute("ATTACH DATABASE 'db2.db' AS db2;")
conn.execute("ATTACH DATABASE 'db3.db' AS db3;")
# Create a view that unifies the 'users' table from all three databases
conn.execute("""
CREATE VIEW all_users AS
SELECT * FROM users
UNION ALL
SELECT * FROM db2.users
UNION ALL
SELECT * FROM db3.users;
""")
# Now you can query the view as if it were a single table
cursor = conn.execute("SELECT * FROM all_users;")
for row in cursor:
print(row)
# Close the connection
conn.close()
```
## Proposed SQL Schema
A more comprehensive SQL Schema is proposed below. This schema is intended to be more comprehensive with respect to the
various types of data that profilers can collect (such as Omnitrace/RSP)
The schema consists of multiple interrelated tables to capture different categories of profiling data.
Below is a high-level schema with the primary tables and relationships.
__*Please note, this is a very preliminary sketch of the schema*__.
If you want to weigh in, please restrict comments to the high-level organization, comments that it doesn't contain
fields for correlation IDs or something like that are not particularly helpful at the moment.
```sql
CREATE TABLE strings (
id SERIAL PRIMARY KEY,
value VARCHAR(1024) UNIQUE
);
CREATE TABLE process (
id INT PRIMARY KEY,
pid INT,
process_name_id INT,
executable_path_id INT,
start_time BIGINT,
end_time BIGINT,
FOREIGN KEY (process_name_id) REFERENCES strings(id)
FOREIGN KEY (executable_path_id) REFERENCES strings(id)
);
CREATE TABLE thread (
id INT PRIMARY KEY,
tid INT,
process_id INT,
thread_name_id INT,
start_time BIGINT,
end_time BIGINT,
FOREIGN KEY (process_id) REFERENCES process(process_id),
FOREIGN KEY (thread_name_id) REFERENCES strings(id)
);
CREATE TABLE cpu_info (
id SERIAL PRIMARY KEY,
core_id INT,
socket_id INT,
frequency_hz INT,
model_id INT,
cache_size_kb INT,
FOREIGN KEY (model_id) REFERENCES strings(id)
);
CREATE TABLE gpu_info (
id SERIAL PRIMARY KEY,
device_name_id INT,
compute_capability_id INT,
memory_size_mb INT,
multiprocessor_count INT,
clock_rate_hz INT,
FOREIGN KEY (device_name_id) REFERENCES strings(id)
FOREIGN KEY (compute_capability_id) REFERENCES strings(id)
);
CREATE TABLE instrumentation_regions (
id SERIAL PRIMARY KEY,
process_id INT,
thread_id INT,
region_name_id INT,
start_time BIGINT,
end_time BIGINT,
parent_region_id INT,
duration_ns BIGINT GENERATED ALWAYS AS (end_time - start_time) STORED,
file_id INT,
line_number INT,
additional_info JSONB,
FOREIGN KEY (process_id) REFERENCES process(process_id),
FOREIGN KEY (thread_id) REFERENCES thread(thread_id),
FOREIGN KEY (region_name_id) REFERENCES strings(id),
FOREIGN KEY (file_id) REFERENCES strings(id)
);
CREATE TABLE call_stacks (
id SERIAL PRIMARY KEY,
process_id INT,
thread_id INT,
timestamp BIGINT,
stack_depth INT,
function_id INT,
file_id INT,
line_number INT,
parent_sample_id INT,
call_site VARCHAR(1024),
additional_info JSONB,
FOREIGN KEY (process_id) REFERENCES process(process_id),
FOREIGN KEY (thread_id) REFERENCES thread(thread_id),
FOREIGN KEY (function_id) REFERENCES strings(id),
FOREIGN KEY (file_id) REFERENCES strings(id)
);
CREATE TABLE hardware_counters (
id SERIAL PRIMARY KEY,
process_id INT,
thread_id INT,
timestamp BIGINT,
event_id INT,
value BIGINT,
FOREIGN KEY (process_id) REFERENCES process(process_id),
FOREIGN KEY (thread_id) REFERENCES thread(thread_id),
FOREIGN KEY (event_id) REFERENCES strings(id)
);
CREATE TABLE memory_operations (
id SERIAL PRIMARY KEY,
process_id INT,
thread_id INT,
timestamp BIGINT,
operation_type VARCHAR(50) CHECK (operation_type IN ('ALLOC', 'FREE', 'COPY')),
source_address BIGINT,
destination_address BIGINT,
size_bytes BIGINT,
duration_us BIGINT,
additional_info JSONB,
FOREIGN KEY (process_id) REFERENCES process(process_id),
FOREIGN KEY (thread_id) REFERENCES thread(thread_id)
);
CREATE TABLE gpu_kernel_launches (
id SERIAL PRIMARY KEY,
process_id INT,
thread_id INT,
gpu_id INT,
kernel_id INT,
dispatch_id INT,
launch_time BIGINT,
start_time BIGINT,
end_time BIGINT,
grid_size_x INT,
grid_size_y INT,
grid_size_z INT,
block_size_x INT,
block_size_y INT,
block_size_z INT,
shared_mem_bytes INT,
duration_ns BIGINT GENERATED ALWAYS AS (end_time - start_time) STORED,
FOREIGN KEY (process_id) REFERENCES process(process_id),
FOREIGN KEY (thread_id) REFERENCES thread(thread_id),
FOREIGN KEY (gpu_id) REFERENCES gpu_info(gpu_id),
FOREIGN KEY (kernel_id) REFERENCES strings(id)
);
CREATE TABLE binary_analysis_info (
id SERIAL PRIMARY KEY,
process_id INT,
binary_name VARCHAR(1024),
function_id INT,
start_address BIGINT,
end_address BIGINT,
instruction_count INT,
file_id INT,
line_number INT,
call_sites JSONB,
additional_info JSONB,
FOREIGN KEY (process_id) REFERENCES process(process_id),
FOREIGN KEY (function_id) REFERENCES strings(id),
FOREIGN KEY (file_id) REFERENCES strings(id)
);
```
Explanation of the design considerations:
1. __Separate String Tables__: Created unique string tables for function names, file names, kernel names, and event names to avoid storing redundant copies.
- `function_names`, `file_names`, `kernel_names`, and `event_names` tables are created to hold unique strings. Each table has a surrogate primary key (`function_id`, `file_id`, `kernel_id`, `event_id`) that is referenced by the main tables.
- This avoids storing redundant copies of long or frequently repeating strings in different tables, reducing the storage footprint and improving consistency.
2. __Foreign Key References__: Main tables reference unique strings using foreign keys for consistency and space efficiency.
- Main tables such as `instrumentation_regions`, `call_stacks`, `gpu_kernel_launches`, etc., reference these unique string tables using foreign keys.
- This makes querying for specific function names or kernel names more efficient, as the strings are indexed separately.
3. __Computed Columns__: Used computed columns for duration fields to facilitate quick analysis.
- The `duration_us` columns are computed based on timestamps, providing useful metrics for quick analysis.
4. __Extensibility__: Designed to be easily extensible with additional string categories if needed.
- New string types or categories can be added by creating new tables, and the main tables can reference them with minor schema adjustments.
5. __JSONB for Additional Metadata__:
- JSONB columns (`additional_info`) are used to handle complex or variable metadata that doesn’t fit neatly into the structured schema (e.g., custom annotations, extra debug info).
### Example Data Insertion and Lookup
#### Adding a new function
```sql
INSERT INTO function_names (function_name) VALUES ('my_function') ON CONFLICT (function_name) DO NOTHING;
```
#### Linking a function in a call stack
```sql
INSERT INTO call_stacks (process_id, thread_id, timestamp, stack_depth, function_id, file_id)
VALUES (123, 456, '2024-09-27 10:00:00', 1, (SELECT function_id FROM function_names WHERE function_name = 'my_function'),
(SELECT file_id FROM file_names WHERE file_name = 'my_file.c'));
```
## Q & A
### All global variables are protected with locks in common synchronized library. How are we sending the data from these variables to the pure functions?
There is a new `rocprofiler::tool::metadata` struct in `lib/output/metadata.hpp` which will be populated with data from SQL.
This struct is passed to the output functions.
### If we provide the functionality to flush the trace at regular intervals, do we delete the data in global memory after each flush? If not, how do we keep track of data already read at any given point time during runtime?
We will probably not delete the metadata (agent info, code objects, kernel symbols, etc.) after a flush.
When we flush, we will swap out the temporary binary file with a new temporary binary file and write/append the database with
the contents of the old temporary binary file.
### Can a user collect trace at regular flush interval and ask for counter collection at the end of application?
I am not sure what you mean here. We can write counter collection data when we flush. If the user is asking for periodic
flushing, we will restrict the output format to the database. In other words, I suspect that only `--flush-rate X` will only
be compatible with `--output-format rocpd` -- any additional or alternative data formats and we will throw an error in the
rocprofv3 script. This is for simplicity sake, supporting periodically flushing to CSV, etc. is unnecessary in my opinion.
### I think hardware_counters table in database schema should have a dispatch_id field to represent the kernel it belongs to
Please note, the proposed schema states clearly:
> __*Please note, this is a very preliminary sketch of the schema*__.
> If you want to weigh in, please restrict comments to the high-level organization, comments that it doesn't contain
> fields for correlation IDs or something like that are not particularly helpful at the moment.
However, I will note that the hardware counters table is probably going to be generic, i.e. supporting CPU HW counters, which
do not have dispatch IDs. Lastly, I will also note, device counter collection is not associated with a dispatch so even in
the case of GPU HW counters, including this field is questionable.
### What is binary analysis info table?
More advanced tools such as Omnitrace/Rocprofiler-System do address to line translations. This could also potentally
include the sort of data related to PC sampling
### What is the Key of gpu info table? Node_id/zero based numbering scheme?
That isn't defined. Very preliminary sketch.
### When is user allowed to access the database in case of flushing the trace at regular intervals? Is user allowed to read the database only after tool finalization? Or we create a database file for each interval?
TBD on the exact details but the user will certainly be able to read the database before tool finalization when it is flushed.
source/lib/rocprofiler-sdk-tool/tool.cpp
+19
Просмотреть файл
@@ -49,6 +49,7 @@
#include "lib/output/generateJSON.hpp"
#include "lib/output/generateOTF2.hpp"
#include "lib/output/generatePerfetto.hpp"
#include "lib/output/generateRocpd.hpp"
#include "lib/output/generateStats.hpp"
#include "lib/output/metadata.hpp"
#include "lib/output/output_stream.hpp"
@@ -2444,6 +2445,24 @@ tool_fini(void* /*tool_data*/)
rocjpeg_output.get_generator());
}
if(tool::get_config().rocpd_output && outdata.num_output > 0 &&
outdata.num_bytes >= tool::get_config().minimum_output_bytes)
{
tool::write_rocpd(tool::get_config(),
*tool_metadata,
agents_output,
hip_output.get_generator(),
hsa_output.get_generator(),
kernel_dispatch_output.get_generator(),
memory_copy_output.get_generator(),
marker_output.get_generator(),
memory_allocation_output.get_generator(),
scratch_memory_output.get_generator(),
rccl_output.get_generator(),
rocdecode_output.get_generator(),
counters_output.get_generator());
}
if(tool::get_config().otf2_output && outdata.num_output > 0 &&
outdata.num_bytes >= tool::get_config().minimum_output_bytes)
{
source/lib/rocprofiler-sdk/counters/CMakeLists.txt
-2
Просмотреть файл
@@ -23,9 +23,7 @@ set(ROCPROFILER_LIB_COUNTERS_HEADERS
ioctl.hpp)
target_sources(rocprofiler-sdk-object-library PRIVATE ${ROCPROFILER_LIB_COUNTERS_SOURCES}
${ROCPROFILER_LIB_COUNTERS_HEADERS})
add_subdirectory(xml)
add_subdirectory(parser)
add_subdirectory(yaml)
if(ROCPROFILER_BUILD_TESTS)
add_subdirectory(tests)
source/lib/rocprofiler-sdk/counters/xml/CMakeLists.txt
-10
Просмотреть файл
@@ -1,10 +0,0 @@
configure_file(basic_counters.xml
${PROJECT_BINARY_DIR}/share/rocprofiler-sdk/basic_counters.xml COPYONLY)
configure_file(derived_counters.xml
${PROJECT_BINARY_DIR}/share/rocprofiler-sdk/derived_counters.xml COPYONLY)
install(
FILES ${PROJECT_BINARY_DIR}/share/rocprofiler-sdk/basic_counters.xml
${PROJECT_BINARY_DIR}/share/rocprofiler-sdk/derived_counters.xml
DESTINATION share/rocprofiler-sdk
COMPONENT core)
source/lib/rocprofiler-sdk/counters/yaml/CMakeLists.txt
-6
Просмотреть файл
@@ -1,6 +0,0 @@
configure_file(counter_defs.yaml
${PROJECT_BINARY_DIR}/share/rocprofiler-sdk/counter_defs.yaml COPYONLY)
install(
FILES ${PROJECT_BINARY_DIR}/share/rocprofiler-sdk/counter_defs.yaml
DESTINATION share/rocprofiler-sdk
COMPONENT core)
source/lib/tests/common/CMakeLists.txt
+7 -3
Просмотреть файл
@@ -5,14 +5,18 @@ project(rocprofiler-sdk-tests-common LANGUAGES C CXX)
include(GoogleTest)
set(common_sources demangling.cpp environment.cpp md5sum.cpp mpl.cpp parse.cpp)
set(common_sources demangling.cpp environment.cpp md5sum.cpp mpl.cpp parse.cpp sha256.cpp
uuid_v7.cpp)
add_executable(common-tests)
target_sources(common-tests PRIVATE ${common_sources})
target_link_libraries(
common-tests
PRIVATE rocprofiler-sdk::rocprofiler-sdk-headers
rocprofiler-sdk::rocprofiler-sdk-common-library GTest::gtest
rocprofiler-sdk::rocprofiler-sdk-common-library
rocprofiler-sdk::rocprofiler-sdk-output-library
rocprofiler-sdk::rocprofiler-sdk-cereal
GTest::gtest
GTest::gtest_main)
gtest_add_tests(
@@ -24,4 +28,4 @@ gtest_add_tests(
set_tests_properties(
${common-tests_TESTS}
PROPERTIES TIMEOUT 45 LABELS "unittests" FAIL_REGULAR_EXPRESSION
"${ROCPROFILER_DEFAULT_FAIL_REGEX}")
"${ROCPROFILER_DEFAULT_FAIL_REGEX}" ENVIRONMENT "TEST_LOG_LEVEL=info")
source/lib/tests/common/environment.cpp
+45
Просмотреть файл
@@ -22,8 +22,14 @@
#include "lib/common/environment.hpp"
#include <fmt/format.h>
#include <gtest/gtest.h>
namespace
{
bool _common_environment_test_init_logging = (rocprofiler::common::init_logging("TEST"), true);
}
TEST(common, environment)
{
using rocprofiler::common::env_config;
@@ -113,3 +119,42 @@ TEST(common, environment)
EXPECT_TRUE(get_env("ROCPROFILER_ENV_TEST_BOOL", false));
}
}
TEST(common, environment_push_pop)
{
using rocprofiler::common::env_config;
using rocprofiler::common::env_store;
using rocprofiler::common::get_env;
namespace common = ::rocprofiler::common;
common::set_env("ROCPROFILER_ENV_TEST_A", "0", 1);
common::set_env("ROCPROFILER_ENV_TEST_B", "2", 1);
auto _store = env_store{{env_config{"ROCPROFILER_ENV_TEST_A", "1"},
env_config{"ROCPROFILER_ENV_TEST_B", "2", 1},
env_config{"ROCPROFILER_ENV_TEST_C", "3", 0}}};
for(size_t i = 0; i < 5; ++i)
{
EXPECT_EQ(get_env("ROCPROFILER_ENV_TEST_A", 3), 0) << fmt::format("iteration={}", i);
EXPECT_EQ(get_env("ROCPROFILER_ENV_TEST_B", 0), 2) << fmt::format("iteration={}", i);
EXPECT_EQ(get_env("ROCPROFILER_ENV_TEST_C", 1), 1) << fmt::format("iteration={}", i);
EXPECT_FALSE(_store.is_pushed()) << fmt::format("iteration={}", i);
EXPECT_TRUE(_store.push()) << fmt::format("iteration={}", i);
EXPECT_FALSE(_store.push()) << fmt::format("iteration={}", i);
EXPECT_EQ(get_env("ROCPROFILER_ENV_TEST_A", 3), 1) << fmt::format("iteration={}", i);
EXPECT_EQ(get_env("ROCPROFILER_ENV_TEST_B", 0), 2) << fmt::format("iteration={}", i);
EXPECT_EQ(get_env("ROCPROFILER_ENV_TEST_C", 1), 3) << fmt::format("iteration={}", i);
EXPECT_TRUE(_store.is_pushed()) << fmt::format("iteration={}", i);
EXPECT_TRUE(_store.pop()) << fmt::format("iteration={}", i);
EXPECT_FALSE(_store.pop()) << fmt::format("iteration={}", i);
EXPECT_EQ(get_env("ROCPROFILER_ENV_TEST_A", 3), 0) << fmt::format("iteration={}", i);
EXPECT_EQ(get_env("ROCPROFILER_ENV_TEST_B", 0), 2) << fmt::format("iteration={}", i);
EXPECT_EQ(get_env("ROCPROFILER_ENV_TEST_C", 0), 0) << fmt::format("iteration={}", i);
}
}
source/lib/tests/common/sha256.cpp
+57
Просмотреть файл
@@ -0,0 +1,57 @@
// MIT License
//
// Copyright (c) 2023-2025 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#include "lib/common/sha256.hpp"
#include <fmt/format.h>
#include <gtest/gtest.h>
#include <cstddef>
#include <cstdint>
#include <sstream>
#include <string>
TEST(common, sha256)
{
auto _val = rocprofiler::common::sha256{};
_val.update("rocprofiler-sdk|rocprofiler-sdk-roctx|rocprofiler-sdk-rocpd");
auto _hex_digest = _val.hexdigest();
EXPECT_EQ(_hex_digest, "e58b701acc3c524f881e49fff2833879eb17975b6a072d9ecc27de5a5344aefc");
auto read_hex = [](std::string&& _inp) {
uint32_t _ret = 0;
std::stringstream{_inp} >> std::hex >> _ret;
return _ret;
};
auto _raw_digest = _val.rawdigest();
for(size_t i = 0; i < _raw_digest.size(); ++i)
{
auto _sub = _hex_digest.substr(i * 8, 8);
auto _extract = read_hex(fmt::format("0x{}", _sub));
int _raw = _raw_digest.at(i);
EXPECT_EQ(_raw, _extract) << fmt::format(
"i={}, raw[i]={}, hex={}, hexdigest={}", i, _raw, _sub, _hex_digest);
}
}
source/lib/tests/common/uuid_v7.cpp
+80
Просмотреть файл
@@ -0,0 +1,80 @@
// MIT License
//
// Copyright (c) 2023-2025 Advanced Micro Devices, Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
#include "lib/common/uuid_v7.hpp"
#include "lib/common/utility.hpp"
#include "lib/output/node_info.hpp"
#include <fmt/format.h>
#include <gtest/gtest.h>
#include <cstddef>
#include <cstdint>
#include <sstream>
#include <string>
TEST(common, uuid_v7)
{
namespace common = ::rocprofiler::common;
namespace tool = ::rocprofiler::tool;
auto _node_data = tool::read_node_info();
const auto& _mach_id = _node_data.machine_id;
auto _this_pid = getpid();
auto _this_ppid = getppid();
auto _init_ns = common::timestamp_ns();
auto _ticks = common::get_process_start_ticks_since_boot(_this_pid);
auto get_uuid_v7 = [&_mach_id, &_this_pid, &_this_ppid, &_ticks](auto _timestamp_ns) {
auto _seed = common::compute_system_seed(_mach_id, _this_pid, _this_ppid, _ticks);
return common::generate_uuid_v7(_timestamp_ns, _seed);
};
auto reference_uuid_v7 = get_uuid_v7(_init_ns);
// uuid v7 has millisecond precision
std::this_thread::sleep_for(std::chrono::milliseconds{5});
for(size_t i = 0; i < 1000; ++i)
{
auto varying_ns = common::timestamp_ns();
auto varying_uuid_v7 = get_uuid_v7(varying_ns);
EXPECT_NE(reference_uuid_v7, varying_uuid_v7)
<< fmt::format("machine-id={}, pid={}, ppid={}, ticks={}, init_ns={}, varying_ns={}",
_mach_id,
_this_pid,
_this_ppid,
_ticks,
_init_ns,
varying_ns);
// UUIDv7 is supposed to be lexicographically sortable
EXPECT_LT(reference_uuid_v7, varying_uuid_v7)
<< fmt::format("machine-id={}, pid={}, ppid={}, ticks={}, init_ns={}, varying_ns={}",
_mach_id,
_this_pid,
_this_ppid,
_ticks,
_init_ns,
varying_ns);
}
}
source/scripts/generate-rocpd.py
-533
Просмотреть файл
@@ -1,533 +0,0 @@
#!/usr/bin/env python3
# MIT License
#
# Copyright (c) 2023-2025 Advanced Micro Devices, Inc. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import sys
import json
import time
import sqlite3
import argparse
__author__ = "AMD"
__copyright__ = "Copyright 2023, Advanced Micro Devices, Inc."
__license__ = "MIT"
__maintainer__ = "AMD"
__status__ = "Development"
"""
This script converts one or more JSON output files from rocprofv3 into a
single SQLite database conforming to the rocpd SQL Schema.
"""
# this is the list of APIs whose records are inserted into API table which
# needs to be updated whenever tracing support for a new API is added
rocprofv3_apis = ("hip_api", "hsa_api", "marker_api", "rccl_api")
class dotdict(dict):
"""dot.notation access to dictionary attributes"""
__getattr__ = dict.get
__setattr__ = dict.__setitem__
__delattr__ = dict.__delitem__
def __init__(self, d):
super(dotdict, self).__init__(d)
for k, v in self.items():
if isinstance(v, dict):
self.__setitem__(k, dotdict(v))
elif isinstance(v, (list, tuple)):
self.__setitem__(
k,
[dotdict(i) if isinstance(i, (list, tuple, dict)) else i for i in v],
)
def dump_table(table):
cursor.execute(f"SELECT * FROM {table};")
results = cursor.fetchall()
print(f"\n\n##### {table} #####\n")
for itr in results:
print(" | {}".format(" | ".join([f"{val}" for val in list(itr)])))
print("")
def execute_raw_sql_statements(cursor, statements):
"""Helper function for executing a sequence of raw SQL statements"""
for itr in [
"{};".format(itr.strip()) for itr in statements.strip().split(";") if itr
]:
try:
cursor.execute(f"{itr}")
except sqlite3.Error as err:
sys.stderr.write(f"SQLite3 error: {err}\nStatement:\n\t{itr}\n")
sys.stderr.flush()
raise err
def create_schema(cursor):
# Create table
table_schema = """
CREATE TABLE IF NOT EXISTS "rocpd_metadata" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "tag" varchar(4096) NOT NULL, "value" varchar(4096) NOT NULL);
CREATE TABLE IF NOT EXISTS "rocpd_string" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "string" varchar(4096) NOT NULL UNIQUE ON CONFLICT IGNORE);
CREATE TABLE IF NOT EXISTS "rocpd_op" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "gpuId" integer NOT NULL, "queueId" integer NOT NULL, "sequenceId" integer NOT NULL, "completionSignal" varchar(18) NOT NULL, "start" integer NOT NULL, "end" integer NOT NULL, "description_id" integer NOT NULL REFERENCES "rocpd_string" ("id") DEFERRABLE INITIALLY DEFERRED, "opType_id" integer NOT NULL REFERENCES "rocpd_string" ("id") DEFERRABLE INITIALLY DEFERRED);
CREATE TABLE IF NOT EXISTS "rocpd_api" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "pid" integer NOT NULL, "tid" integer NOT NULL, "start" integer NOT NULL, "end" integer NOT NULL, "apiName_id" integer NOT NULL REFERENCES "rocpd_string" ("id") DEFERRABLE INITIALLY DEFERRED, "args_id" integer NOT NULL REFERENCES "rocpd_string" ("id") DEFERRABLE INITIALLY DEFERRED);
CREATE TABLE IF NOT EXISTS "rocpd_api_ops" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "api_id" integer NOT NULL REFERENCES "rocpd_api" ("id") DEFERRABLE INITIALLY DEFERRED, "op_id" integer NOT NULL REFERENCES "rocpd_op" ("id") DEFERRABLE INITIALLY DEFERRED);
-- optional
CREATE TABLE IF NOT EXISTS "rocpd_kernelcodeobject" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "vgpr" integer NOT NULL, "sgpr" integer NOT NULL, "fbar" integer NOT NULL, "kernel_id" integer NOT NULL);
CREATE TABLE IF NOT EXISTS "rocpd_kernelapi" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "api_ptr_id" integer NOT NULL REFERENCES "rocpd_api" ("id") DEFERRABLE INITIALLY DEFERRED, "stream" varchar(18) NOT NULL, "gridX" integer NOT NULL, "gridY" integer NOT NULL, "gridZ" integer NOT NULL, "workgroupX" integer NOT NULL, "workgroupY" integer NOT NULL, "workgroupZ" integer NOT NULL, "groupSegmentSize" integer NOT NULL, "privateSegmentSize" integer NOT NULL, "kernelArgAddress" varchar(18) NOT NULL, "aquireFence" varchar(8) NOT NULL, "releaseFence" varchar(8) NOT NULL, "codeObject_id" integer NOT NULL REFERENCES "rocpd_kernelcodeobject" ("id") DEFERRABLE INITIALLY DEFERRED, "kernelName_id" integer NOT NULL REFERENCES "rocpd_string" ("id") DEFERRABLE INITIALLY DEFERRED);
CREATE TABLE IF NOT EXISTS "rocpd_copyapi" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "api_ptr_id" integer NOT NULL REFERENCES "rocpd_api" ("id") DEFERRABLE INITIALLY DEFERRED, "stream" varchar(18) NOT NULL, "size" integer NOT NULL, "width" integer NOT NULL, "height" integer NOT NULL, "kind" integer NOT NULL, "dst" varchar(18) NOT NULL, "src" varchar(18) NOT NULL, "dstDevice" integer NOT NULL, "srcDevice" integer NOT NULL, "sync" bool NOT NULL, "pinned" bool NOT NULL);
INSERT INTO "rocpd_metadata"(tag, value) VALUES ("schema_version", "2");
--CREATE TABLE IF NOT EXISTS "rocpd_monitor" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "deviceType" varchar(16) NOT NULL, "deviceId" integer NOT NULL, "monitorType" varchar(16) NOT NULL, "start" integer NOT NULL, "end" integer NOT NULL, "value" varchar(255) NOT NULL);
--CREATE TABLE IF NOT EXISTS "rocpd_barrierop" ("op_ptr_id" integer NOT NULL PRIMARY KEY REFERENCES "rocpd_op" ("id") DEFERRABLE INITIALLY DEFERRED, "signalCount" integer NOT NULL, "aquireFence" varchar(8) NOT NULL, "releaseFence" varchar(8) NOT NULL);
--CREATE TABLE IF NOT EXISTS "rocpd_op_inputSignals" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "from_op_id" integer NOT NULL REFERENCES "rocpd_op" ("id") DEFERRABLE INITIALLY DEFERRED, "to_op_id" integer NOT NULL REFERENCES "rocpd_op" ("id") DEFERRABLE INITIALLY DEFERRED);
"""
execute_raw_sql_statements(cursor, table_schema)
def finalize_schema(cursor):
table_schema = """
CREATE VIEW api AS SELECT rocpd_api.id,pid,tid,start,end,A.string AS apiName, B.string AS args FROM rocpd_api
INNER JOIN rocpd_string A ON A.id = rocpd_api.apiName_id
INNER JOIN rocpd_string B ON B.id = rocpd_api.args_id;
CREATE VIEW op AS SELECT rocpd_op.id,gpuId,queueId,sequenceId,start,end,A.string AS description, B.string AS opType FROM rocpd_op
INNER JOIN rocpd_string A ON A.id = rocpd_op.description_id
INNER JOIN rocpd_string B ON B.id = rocpd_op.opType_id;
CREATE VIEW busy AS SELECT A.gpuId, GpuTime, WallTime, GpuTime*1.0/WallTime AS Busy FROM (SELECT gpuId, sum(end-start) AS GpuTime FROM rocpd_op GROUP BY gpuId) A
INNER JOIN (SELECT max(end) - min(start) AS WallTime FROM rocpd_op);
CREATE VIEW top AS SELECT C.string AS Name, count(C.string) AS TotalCalls, sum(A.end-A.start) / 1000 AS TotalDuration, (sum(A.end-A.start)/count(C.string))/ 1000 AS Ave, sum(A.end-A.start) * 100.0 / (SELECT sum(A.end-A.start) FROM rocpd_op A) AS Percentage FROM (SELECT opType_id AS name_id, start, end FROM rocpd_op WHERE description_id in (SELECT id FROM rocpd_string WHERE string='')
UNION SELECT description_id, start, end FROM rocpd_op WHERE description_id not in (SELECT id FROM rocpd_string WHERE string='')) A
JOIN rocpd_string C on C.id = A.name_id GROUP BY Name ORDER BY TotalDuration desc;
CREATE VIEW ktop AS SELECT C.string AS Name, count(C.string) AS TotalCalls, sum(A.end-A.start) / 1000 AS TotalDuration, (sum(A.end-A.start)/count(C.string))/ 1000 AS Ave, sum(A.end-A.start) * 100.0 / (SELECT sum(A.end-A.start) FROM rocpd_api A
JOIN rocpd_kernelapi B on B.api_ptr_id = A.id) AS Percentage FROM rocpd_api A
JOIN rocpd_kernelapi B on B.api_ptr_id = A.id
JOIN rocpd_string C on C.id = B.kernelname_id GROUP BY Name ORDER BY TotalDuration desc;
CREATE VIEW kernel AS SELECT B.id, gpuId, queueId, sequenceId, start, end, (end-start) AS duration, stream, gridX, gridY, gridz, workgroupX, workgroupY, workgroupZ, groupSegmentSize, privateSegmentSize, D.string AS kernelName FROM rocpd_api_ops A
JOIN rocpd_op B on B.id = A.op_id
JOIN rocpd_kernelapi C ON C.api_ptr_id = A.api_id
JOIN rocpd_string D on D.id = kernelName_id;
CREATE VIEW copy AS SELECT B.id, pid, tid, start, end, C.string AS apiName, stream, size, width, height, kind, dst, src, dstDevice, srcDevice, sync, pinned FROM rocpd_copyApi A
JOIN rocpd_api B ON B.id = A.api_ptr_id
JOIN rocpd_string C on C.id = B.apiname_id;
CREATE VIEW copyop AS SELECT B.id, gpuId, queueId, sequenceId, B.start, B.end, (B.end-B.start) AS duration, stream, size, width, height, kind, dst, src, dstDevice, srcDevice, sync, pinned, E.string AS apiName FROM rocpd_api_ops A
JOIN rocpd_op B ON B.id = A.op_id
JOIN rocpd_copyapi C ON C.api_ptr_id = A.api_id
JOIN rocpd_api D on D.id = A.api_id
JOIN rocpd_string E ON E.id = D.apiName_id;
"""
execute_raw_sql_statements(cursor, table_schema)
def normalize_timestamps(itr):
"""Make all timestamps relative to the time of rocprofv3 initialization within the application"""
def _normalize_timestamp_impl(value):
return value - itr.metadata.init_time
min_val = None
for aitr in [
"hip_api",
"hsa_api",
"marker_api",
"rccl_api",
"kernel_dispatch",
"memory_copy",
]:
for ritr in itr.buffer_records[aitr]:
ritr.start_timestamp = _normalize_timestamp_impl(ritr.start_timestamp)
ritr.end_timestamp = _normalize_timestamp_impl(ritr.end_timestamp)
min_val = (
min([ritr.start_timestamp, min_val])
if min_val is not None
else ritr.start_timestamp
)
print(f" - starting timestamp normalized down to a minimum of {min_val} nsec")
sys.stdout.flush()
return itr
def insert_strings(cursor, itr):
"""Populate the strings table with all the strings which will be referenced by various records"""
strings = []
def append_strings(*args):
nonlocal strings
for aitr in args:
if isinstance(aitr, list):
strings += aitr
else:
strings.append(aitr)
append_strings("UserMarker")
for aitr in itr.agents:
append_strings(aitr.name, aitr.vendor_name, aitr.product_name, aitr.model_name)
for ritr in itr.strings.callback_records:
append_strings(ritr.kind, ritr.operations)
for ritr in itr.strings.buffer_records:
append_strings(ritr.kind, ritr.operations)
for ritr in itr.strings.marker_api:
append_strings(ritr.value)
for ritr in itr.strings.counters.dimension_ids:
append_strings(ritr.name)
for ritr in itr.strings.correlation_id.external:
append_strings(ritr.value)
for ritr in itr.kernel_symbols:
append_strings(ritr.kernel_name)
append_strings(ritr.formatted_kernel_name)
append_strings(ritr.demangled_kernel_name)
append_strings(ritr.truncated_kernel_name)
for ritr in itr.code_objects:
append_strings(ritr.uri)
for itr in sorted(list(set(strings))):
cursor.execute(f"""INSERT INTO rocpd_string (string) VALUES ('{itr}')""")
def insert_api_data(cursor, itr, corr_id_offset, **kwargs):
"""Add all the HIP, HSA, marker, and RCCL API records to the database.
Eventually we might want to abstract a way to iterate over the APIs covered
here instead of maintaining an explicit list.
"""
marker_message_strings = dict(
[[eitr.key, eitr.value] for eitr in itr.strings.marker_api]
)
def get_api_name(kind, op):
return itr.strings.buffer_records[kind].operations[op]
def get_marker_message(name, corr_id):
return marker_message_strings.get(corr_id, name)
max_corr_id = 0
for aitr in rocprofv3_apis:
for hitr in itr.buffer_records[aitr]:
corr_id = hitr.correlation_id
corr_id.internal += corr_id_offset
name = None
args = None
if aitr == "marker_api":
apiname = get_api_name(hitr.kind, hitr.operation)
message = get_marker_message(apiname, corr_id.internal)
mode = kwargs.get("marker_mode", "message")
assert mode in ("message", "generic", "api")
if mode == "message":
name = message
args = 1
elif mode == "api":
name = apiname
args = f"(SELECT id FROM rocpd_string WHERE string = '{message}')"
elif mode == "generic":
name = "UserMarker"
args = f"(SELECT id FROM rocpd_string WHERE string = '{message}')"
else:
name = get_api_name(hitr.kind, hitr.operation)
args = 1
assert name is not None
assert args is not None
cursor.execute(
f"""INSERT INTO rocpd_api(id, pid, tid, start, end, apiName_id, args_id)
VALUES ({corr_id.internal},
{itr.metadata.pid},
{hitr.thread_id},
{hitr.start_timestamp},
{hitr.end_timestamp},
(SELECT id FROM rocpd_string WHERE string = '{name}'),
{args});
"""
)
max_corr_id = max([max_corr_id, corr_id.internal])
return max_corr_id
def insert_async_data(cursor, itr, corr_id_offset, op_id_offset):
"""Add all the kernel and memory copy records to the database.
Eventually we might want to handle page-migration, scratch-memory, etc. but,
at present, rocpd_schema does not support it.
"""
external_corr_id_strings = dict(
[[eitr.key, eitr.value] for eitr in itr.strings.correlation_id.external]
)
def get_api_name(kind, op=None):
return (
itr.strings.buffer_records[kind].operations[op]
if op is not None
else itr.strings.buffer_records[kind].kind
)
def get_kernel_symbol(kernid):
return itr.kernel_symbols[kernid]
def get_kernel_name(kernid, externid):
if externid > 0:
return external_corr_id_strings[externid]
return get_kernel_symbol(kernid).formatted_kernel_name
def get_agent_id(agent_id):
for aitr in itr.agents:
if aitr.id.handle == agent_id.handle:
return aitr.node_id
return None
for kitr in itr.kernel_symbols:
sgpr = kitr.sgpr_count if "sgpr_count" in kitr.keys() else 0
arch_vgpr = kitr.arch_vgpr_count if "arch_vgpr_count" in kitr.keys() else 0
accum_vgpr = kitr.accum_vgpr_count if "accum_vgpr_count" in kitr.keys() else 0
vgpr = arch_vgpr + accum_vgpr
cursor.execute(
f"""INSERT INTO rocpd_kernelcodeobject(vgpr, sgpr, fbar, kernel_id)
VALUES ({vgpr}, {sgpr}, 0, {kitr.kernel_id});
"""
)
op_id = op_id_offset
for kitr in itr.buffer_records.kernel_dispatch:
kind_name = get_api_name(kitr.kind)
info = kitr.dispatch_info
kernel_id = info.kernel_id
queue_id = info.queue_id.handle
corr_id = kitr.correlation_id
grid = info.grid_size
workgroup = info.workgroup_size
kern_name = get_kernel_name(kernel_id, corr_id.external)
gpu_id = get_agent_id(info.agent_id)
ksym = get_kernel_symbol(kernel_id)
kernel_arg_addr = "{:#x}".format(ksym.kernel_object)
corr_id.internal += corr_id_offset
cursor.execute(
f"""INSERT INTO rocpd_kernelapi(api_ptr_id,
stream,
gridX, gridY, gridZ,
workgroupX, workgroupY, workgroupZ,
groupSegmentSize, privateSegmentSize,
kernelArgAddress, aquireFence, releaseFence,
codeObject_id, kernelName_id)
VALUES ({corr_id.internal},
{queue_id},
{grid.x}, {grid.y}, {grid.z},
{workgroup.x}, {workgroup.y}, {workgroup.z},
{info.group_segment_size}, {info.private_segment_size},
'{kernel_arg_addr}', '', '',
(SELECT id FROM rocpd_kernelcodeobject WHERE kernel_id = {ksym.kernel_id}),
(SELECT id FROM rocpd_string WHERE string = '{kern_name}'));
"""
)
cursor.execute(
f"""INSERT INTO rocpd_op(id, gpuId, queueId, sequenceId, completionSignal, start, end, description_id, opType_id)
VALUES ({op_id},
{gpu_id},
{queue_id},
{corr_id.internal},
"",
{kitr.start_timestamp},
{kitr.end_timestamp},
(SELECT id FROM rocpd_string WHERE string = '{kern_name}'),
(SELECT id FROM rocpd_string WHERE string = '{kind_name}'));
"""
)
cursor.execute(
f"""INSERT INTO rocpd_api_ops(api_id, op_id)
VALUES ({corr_id.internal},
{op_id});
"""
)
op_id += 1
for mitr in itr.buffer_records.memory_copy:
kind_name = get_api_name(mitr.kind)
op_name = get_api_name(mitr.kind, mitr.operation)
dst_id = get_agent_id(mitr.dst_agent_id)
src_id = get_agent_id(mitr.src_agent_id)
corr_id = mitr.correlation_id
synced = False
pinned = False
corr_id.internal += corr_id_offset
cursor.execute(
f"""INSERT INTO rocpd_copyapi(api_ptr_id, stream, size, width, height, kind, src, dst, srcDevice, dstDevice, sync, pinned)
VALUES ({corr_id.internal},
"",
{mitr.bytes},
{mitr.bytes},
1,
(SELECT id FROM rocpd_string WHERE string = '{op_name}'),
"",
"",
{src_id},
{dst_id},
{synced},
{pinned});
"""
)
cursor.execute(
f"""INSERT INTO rocpd_op(id, gpuId, queueId, sequenceId, completionSignal, start, end, description_id, opType_id)
VALUES ({op_id},
{dst_id},
0,
{corr_id.internal},
"",
{mitr.start_timestamp},
{mitr.end_timestamp},
(SELECT id FROM rocpd_string WHERE string = '{op_name}'),
(SELECT id FROM rocpd_string WHERE string = '{kind_name}'));
"""
)
cursor.execute(
f"""INSERT INTO rocpd_api_ops(api_id, op_id)
VALUES ({corr_id.internal},
{op_id});
"""
)
op_id += 1
return op_id
if __name__ == "__main__":
rocpd_tables = [
"metadata",
"string",
"api",
"op",
"api_ops",
"copyapi",
"kernelapi",
"kernelcodeobject",
]
parser = argparse.ArgumentParser()
parser.add_argument(
"-i",
"--input",
help="Input rocprofv3 JSON files",
type=str,
nargs="+",
)
parser.add_argument(
"-o", "--output", help="Output database name", type=str, default="example.db"
)
parser.add_argument(
"-n",
"--normalize-timestamps",
help="Normalize timestamps relative to the app start time",
action="store_true",
)
parser.add_argument(
"-m",
"--marker-mode",
help="'generic' is classical rocpd behavior: all marker regions have 'UserMarker' name with message in args; 'message' uses the message as the region name; 'api' uses the name of the marker function with the message in args",
choices=("generic", "message", "api"),
type=str,
default="message",
)
parser.add_argument(
"-d",
"--dump-tables",
help="Dump generate rocpd tables to console (for debugging)",
type=str,
default=None,
nargs="*",
choices=set(rocpd_tables),
)
args = parser.parse_args(sys.argv[1:])
start = time.monotonic_ns()
print(f"Opening '{args.output}'...")
# Connect to an SQLite database (or create it if it doesn't exist)
conn = sqlite3.connect(args.output)
# Create a cursor object using the cursor() method
cursor = conn.cursor()
create_schema(cursor)
corr_id_offset = 0
op_id_offset = 0
for itr in args.input:
print(f"Reading '{itr}'...")
with open(itr, "rb") as f:
data = dotdict(json.load(f))["rocprofiler-sdk-tool"]
for ditr in data:
# normalize the timestamps if requested
ditr = normalize_timestamps(ditr) if args.normalize_timestamps else ditr
# create the strings table
insert_strings(cursor, ditr)
# insert the api data
_corr_id_offset = insert_api_data(
cursor, ditr, corr_id_offset, marker_mode=args.marker_mode
)
# insert the kernel and memory copy data
_op_id_offset = insert_async_data(
cursor, ditr, corr_id_offset, op_id_offset
)
# Save (commit) the changes
conn.commit()
# update the offsets
corr_id_offset = _corr_id_offset
op_id_offset = _op_id_offset
if args.dump_tables is not None and len(args.dump_tables) == 0:
args.dump_tables = rocpd_tables
if args.dump_tables is not None:
for itr in args.dump_tables:
dump_table(f"rocpd_{itr}")
finalize_schema(cursor)
conn.commit()
print(f"Closing '{args.output}'...")
# Close the connection
conn.close()
end = time.monotonic_ns()
elapsed_nsec = end - start
elapsed_sec = elapsed_nsec / 1.0e9
print(f"Runtime time (nsec): {elapsed_nsec}")
print(f"Runtime time (sec) : {elapsed_sec}")
source/share/CMakeLists.txt
+5
Просмотреть файл
@@ -1 +1,6 @@
#
#
#
add_subdirectory(rocprofiler-sdk)
add_subdirectory(rocprofiler-sdk-rocpd)
source/share/rocprofiler-sdk-rocpd/CMakeLists.txt
+15
Просмотреть файл
@@ -0,0 +1,15 @@
#
#
#
set(rocpd_schemas data_views.sql marker_views.sql rocpd_indexes.sql rocpd_tables.sql
rocpd_views.sql summary_views.sql)
foreach(_FILE ${rocpd_schemas})
configure_file(${_FILE} ${PROJECT_BINARY_DIR}/share/rocprofiler-sdk-rocpd/${_FILE}
COPYONLY)
install(
FILES ${PROJECT_BINARY_DIR}/share/rocprofiler-sdk-rocpd/${_FILE}
DESTINATION share/rocprofiler-sdk-rocpd
COMPONENT rocpd)
endforeach()
source/share/rocprofiler-sdk-rocpd/data_views.sql
+929
Просмотреть файл
@@ -0,0 +1,929 @@
--
-- Useful views
--
-- Code objects
CREATE VIEW IF NOT EXISTS
`code_objects` AS
SELECT
CO.id,
CO.guid,
CO.nid,
CO.pid,
A.absolute_index AS agent_abs_index,
CO.uri,
CO.load_base,
CO.load_size,
CO.load_delta,
CO.storage_type AS storage_type_str,
JSON_EXTRACT(CO.extdata, '$.size') AS code_object_size,
JSON_EXTRACT(CO.extdata, '$.storage_type') AS storage_type,
JSON_EXTRACT(CO.extdata, '$.memory_base') AS memory_base,
JSON_EXTRACT(CO.extdata, '$.memory_size') AS memory_size
FROM
`rocpd_info_code_object` CO
INNER JOIN `rocpd_info_agent` A ON CO.agent_id = A.id
AND CO.guid = A.guid;
CREATE VIEW IF NOT EXISTS
`kernel_symbols` AS
SELECT
KS.id,
KS.guid,
KS.nid,
KS.pid,
KS.code_object_id,
KS.kernel_name,
KS.display_name,
KS.kernel_object,
KS.kernarg_segment_size,
KS.kernarg_segment_alignment,
KS.group_segment_size,
KS.private_segment_size,
KS.sgpr_count,
KS.arch_vgpr_count,
KS.accum_vgpr_count,
JSON_EXTRACT(KS.extdata, '$.size') AS kernel_symbol_size,
JSON_EXTRACT(KS.extdata, '$.kernel_id') AS kernel_id,
JSON_EXTRACT(KS.extdata, '$.kernel_code_entry_byte_offset') AS kernel_code_entry_byte_offset,
JSON_EXTRACT(KS.extdata, '$.formatted_kernel_name') AS formatted_kernel_name,
JSON_EXTRACT(KS.extdata, '$.demangled_kernel_name') AS demangled_kernel_name,
JSON_EXTRACT(KS.extdata, '$.truncated_kernel_name') AS truncated_kernel_name,
JSON_EXTRACT(KS.extdata, '$.kernel_address.handle') AS kernel_address
FROM
`rocpd_info_kernel_symbol` KS;
-- Processes
CREATE VIEW IF NOT EXISTS
`processes` AS
SELECT
N.id AS nid,
N.machine_id,
N.system_name,
N.hostname,
N.release AS system_release,
N.version AS system_version,
P.guid,
P.ppid,
P.id AS pid,
P.init,
P.start,
P.end,
P.fini,
P.command
FROM
`rocpd_info_process` P
INNER JOIN `rocpd_info_node` N ON N.id = P.nid
AND N.guid = P.guid;
-- Threads
CREATE VIEW IF NOT EXISTS
`threads` AS
SELECT
N.id AS nid,
N.machine_id,
N.system_name,
N.hostname,
N.release AS system_release,
N.version AS system_version,
P.guid,
P.ppid,
P.id AS pid,
T.id AS tid,
T.start,
T.end,
T.name
FROM
`rocpd_info_thread` T
INNER JOIN `rocpd_info_process` P ON P.id = T.pid
AND N.guid = T.guid
INNER JOIN `rocpd_info_node` N ON N.id = T.nid
AND N.guid = T.guid;
-- CPU regions
CREATE VIEW IF NOT EXISTS
`regions` AS
SELECT
R.id,
R.guid,
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) AS category,
S.string AS name,
R.nid,
R.pid,
R.tid,
R.start,
R.end,
(R.end - R.start) AS duration,
R.event_id,
E.stack_id,
E.parent_stack_id,
E.correlation_id AS corr_id,
E.extdata,
E.call_stack,
E.line_info
FROM
`rocpd_region` R
INNER JOIN `rocpd_event` E ON E.id = R.event_id
AND E.guid = R.guid
INNER JOIN `rocpd_string` S ON S.id = R.name_id
AND S.guid = R.guid;
CREATE VIEW IF NOT EXISTS
`region_args` AS
SELECT
R.id,
R.guid,
R.nid,
R.pid,
A.type,
A.name,
A.value
FROM
`rocpd_region` R
INNER JOIN `rocpd_event` E ON E.id = R.event_id
AND E.guid = R.guid
INNER JOIN `rocpd_arg` A ON A.event_id = E.id
AND A.guid = R.guid;
--
-- Samples
CREATE VIEW IF NOT EXISTS
`samples` AS
SELECT
R.id,
R.guid,
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) AS category,
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = T.name_id
AND RS.guid = T.guid
) AS name,
T.nid,
T.pid,
T.tid,
R.timestamp,
E.stack_id AS stack_id,
E.parent_stack_id AS parent_stack_id,
E.correlation_id AS corr_id,
E.extdata AS extdata,
E.call_stack AS call_stack,
E.line_info AS line_info
FROM
`rocpd_sample` R
INNER JOIN `rocpd_track` T ON T.id = R.track_id
AND T.guid = R.guid
INNER JOIN `rocpd_event` E ON E.id = R.event_id
AND E.guid = R.guid;
--
-- Kernel information
CREATE VIEW
`kernels` AS
SELECT
K.id,
K.guid,
K.tid,
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) AS category,
R.string AS region,
S.display_name AS name,
K.nid,
Q.pid,
A.absolute_index AS agent_abs_index,
A.logical_index AS agent_log_index,
A.type_index AS agent_type_index,
A.type AS agent_type,
S.code_object_id AS code_object_id,
K.kernel_id,
K.dispatch_id,
K.stream_id,
K.queue_id,
Q.name AS queue,
ST.name AS stream,
K.start,
K.end,
(K.end - K.start) AS duration,
K.grid_size_x AS grid_x,
K.grid_size_y AS grid_y,
K.grid_size_z AS grid_z,
K.workgroup_size_x AS workgroup_x,
K.workgroup_size_y AS workgroup_y,
K.workgroup_size_z AS workgroup_z,
K.group_segment_size AS lds_size,
K.private_segment_size AS scratch_size,
S.group_segment_size AS static_lds_size,
S.private_segment_size AS static_scratch_size,
E.stack_id,
E.parent_stack_id,
E.correlation_id AS corr_id
FROM
`rocpd_kernel_dispatch` K
INNER JOIN `rocpd_info_agent` A ON A.id = K.agent_id
AND A.guid = K.guid
INNER JOIN `rocpd_event` E ON E.id = K.event_id
AND E.guid = K.guid
INNER JOIN `rocpd_string` R ON R.id = K.region_name_id
AND R.guid = K.guid
INNER JOIN `rocpd_info_kernel_symbol` S ON S.id = K.kernel_id
AND S.guid = K.guid
LEFT JOIN `rocpd_info_stream` ST ON ST.id = K.stream_id
AND ST.guid = K.guid
LEFT JOIN `rocpd_info_queue` Q ON Q.id = K.queue_id
AND Q.guid = K.guid;
CREATE VIEW
`kernels_renamed` AS
SELECT
K.id,
K.guid,
K.tid,
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) AS category,
R.string AS region,
R.string AS name,
K.nid,
Q.pid,
A.absolute_index AS agent_abs_index,
A.logical_index AS agent_log_index,
A.type_index AS agent_type_index,
A.type AS agent_type,
S.code_object_id AS code_object_id,
K.kernel_id,
K.dispatch_id,
K.stream_id,
K.queue_id,
Q.name AS queue,
ST.name AS stream,
K.start,
K.end,
(K.end - K.start) AS duration,
K.grid_size_x AS grid_x,
K.grid_size_y AS grid_y,
K.grid_size_z AS grid_z,
K.workgroup_size_x AS workgroup_x,
K.workgroup_size_y AS workgroup_y,
K.workgroup_size_z AS workgroup_z,
K.group_segment_size AS lds_size,
K.private_segment_size AS scratch_size,
S.group_segment_size AS static_lds_size,
S.private_segment_size AS static_scratch_size,
E.stack_id,
E.parent_stack_id,
E.correlation_id AS corr_id
FROM
`rocpd_kernel_dispatch` K
INNER JOIN `rocpd_info_agent` A ON A.id = K.agent_id
AND A.guid = K.guid
INNER JOIN `rocpd_event` E ON E.id = K.event_id
AND E.guid = K.guid
INNER JOIN `rocpd_string` R ON R.id = K.region_name_id
AND R.guid = K.guid
INNER JOIN `rocpd_info_kernel_symbol` S ON S.id = K.kernel_id
AND S.guid = K.guid
LEFT JOIN `rocpd_info_stream` ST ON ST.id = K.stream_id
AND ST.guid = K.guid
LEFT JOIN `rocpd_info_queue` Q ON Q.id = K.queue_id
AND Q.guid = K.guid;
--
-- Performance Monitoring Counters (PMC)
CREATE VIEW IF NOT EXISTS
`pmc_info` AS
SELECT
PMC_I.id,
PMC_I.guid,
PMC_I.nid,
PMC_I.pid,
A.absolute_index AS agent_abs_index,
PMC_I.is_constant,
PMC_I.is_derived,
PMC_I.name,
PMC_I.description,
PMC_I.block,
PMC_I.expression
FROM
`rocpd_info_pmc` PMC_I
INNER JOIN `rocpd_info_agent` A ON PMC_I.agent_id = A.id
AND PMC_I.guid = A.guid;
CREATE VIEW IF NOT EXISTS
`pmc_events` AS
SELECT
PMC_E.id,
PMC_E.guid,
PMC_E.pmc_id,
E.id AS event_id,
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) AS category,
(
SELECT
display_name
FROM
`rocpd_info_kernel_symbol` KS
WHERE
KS.id = K.kernel_id
AND KS.guid = K.guid
) AS name,
K.nid,
K.pid,
K.dispatch_id,
K.start,
K.end,
(K.end - K.start) AS duration,
PMC_I.name AS counter_name,
PMC_E.value AS counter_value
FROM
`rocpd_pmc_event` PMC_E
INNER JOIN `rocpd_info_pmc` PMC_I ON PMC_I.id = PMC_E.pmc_id
AND PMC_I.guid = PMC_E.guid
INNER JOIN `rocpd_event` E ON E.id = PMC_E.event_id
AND E.guid = PMC_E.guid
INNER JOIN `rocpd_kernel_dispatch` K ON K.event_id = PMC_E.event_id
AND K.guid = PMC_E.guid;
-- events with arguments ---
CREATE VIEW IF NOT EXISTS
`events_args` AS
SELECT
E.id AS event_id,
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) AS category,
E.stack_id,
E.parent_stack_id,
E.correlation_id,
A.position AS arg_position,
A.type AS arg_type,
A.name AS arg_name,
A.value AS arg_value,
E.call_stack,
E.line_info,
A.extdata
FROM
`rocpd_event` E
INNER JOIN `rocpd_arg` A ON A.event_id = E.id
AND A.guid = E.guid;
-- list of astream arguments enriched by the corresponding stream descriptions
CREATE VIEW IF NOT EXISTS
`stream_args` AS
SELECT
A.id AS argument_id,
A.event_id AS event_id,
A.position AS arg_position,
A.type AS arg_type,
A.value AS arg_value,
JSON_EXTRACT(A.extdata, '$.stream_id') AS stream_id,
S.nid,
S.pid,
S.name AS stream_name,
S.extdata AS extdata
FROM
`rocpd_arg` A
INNER JOIN `rocpd_info_stream` S ON JSON_EXTRACT(A.extdata, '$.stream_id') = S.id
AND A.guid = S.guid
WHERE
A.name = 'stream';
--
--
CREATE VIEW IF NOT EXISTS
`memory_copies` AS
SELECT
M.id,
M.guid,
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) AS category,
M.nid,
M.pid,
M.tid,
M.start,
M.end,
(M.end - M.start) AS duration,
S.string AS name,
R.string AS region_name,
M.stream_id,
M.queue_id,
ST.name AS stream_name,
Q.name AS queue_name,
M.size,
dst_agent.name AS dst_device,
dst_agent.absolute_index AS dst_agent_abs_index,
dst_agent.logical_index AS dst_agent_log_index,
dst_agent.type_index AS dst_agent_type_index,
dst_agent.type AS dst_agent_type,
M.dst_address,
src_agent.name AS src_device,
src_agent.absolute_index AS src_agent_abs_index,
src_agent.logical_index AS src_agent_log_index,
src_agent.type_index AS src_agent_type_index,
src_agent.type AS src_agent_type,
M.src_address,
E.stack_id,
E.parent_stack_id,
E.correlation_id AS corr_id
FROM
`rocpd_memory_copy` M
INNER JOIN `rocpd_string` S ON S.id = M.name_id
AND S.guid = M.guid
LEFT JOIN `rocpd_string` R ON R.id = M.region_name_id
AND R.guid = M.guid
INNER JOIN `rocpd_info_agent` dst_agent ON dst_agent.id = M.dst_agent_id
AND dst_agent.guid = M.guid
INNER JOIN `rocpd_info_agent` src_agent ON src_agent.id = M.src_agent_id
AND src_agent.guid = M.guid
LEFT JOIN `rocpd_info_queue` Q ON Q.id = M.queue_id
AND Q.guid = M.guid
LEFT JOIN `rocpd_info_stream` ST ON ST.id = M.stream_id
AND ST.guid = M.guid
INNER JOIN `rocpd_event` E ON E.id = M.event_id
AND E.guid = M.guid;
--
--
CREATE VIEW IF NOT EXISTS
`memory_allocations` AS
SELECT
M.id,
M.guid,
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) AS category,
M.nid,
M.pid,
M.tid,
M.start,
M.end,
(M.end - M.start) AS duration,
M.type,
M.level,
A.name AS agent_name,
A.absolute_index AS agent_abs_index,
A.logical_index AS agent_log_index,
A.type_index AS agent_type_index,
A.type AS agent_type,
M.address,
M.size,
M.queue_id,
Q.name AS queue_name,
M.stream_id,
ST.name AS stream_name,
E.stack_id,
E.parent_stack_id,
E.correlation_id AS corr_id
FROM
`rocpd_memory_allocate` M
LEFT JOIN `rocpd_info_agent` A ON M.agent_id = A.id
AND M.guid = A.guid
LEFT JOIN `rocpd_info_queue` Q ON Q.id = M.queue_id
AND Q.guid = M.guid
LEFT JOIN `rocpd_info_stream` ST ON ST.id = M.stream_id
AND ST.guid = M.guid
INNER JOIN `rocpd_event` E ON E.id = M.event_id
AND E.guid = M.guid;
--
--
CREATE VIEW IF NOT EXISTS
`scratch_memory` AS
SELECT
M.id,
M.guid,
M.nid,
M.pid,
M.type AS operation,
A.name AS agent_name,
A.absolute_index AS agent_abs_index,
A.logical_index AS agent_log_index,
A.type_index AS agent_type_index,
A.type AS agent_type,
M.queue_id,
M.tid,
JSON_EXTRACT(M.extdata, '$.flags') AS alloc_flags,
M.start,
M.end,
M.size,
M.address,
E.correlation_id,
E.stack_id,
E.parent_stack_id,
E.correlation_id AS corr_id,
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) AS category,
E.extdata AS event_extdata
FROM
`rocpd_memory_allocate` M
LEFT JOIN `rocpd_info_agent` A ON M.agent_id = A.id
AND M.guid = A.guid
LEFT JOIN `rocpd_info_queue` Q ON Q.id = M.queue_id
AND Q.guid = M.guid
INNER JOIN `rocpd_event` E ON E.id = M.event_id
AND E.guid = M.guid
WHERE
M.level = 'SCRATCH'
ORDER BY
M.start ASC;
--
--
CREATE VIEW IF NOT EXISTS
`counters_collection` AS
SELECT
MIN(PMC_E.id) AS id,
PMC_E.guid,
K.dispatch_id,
K.kernel_id,
E.id AS event_id,
E.correlation_id,
E.stack_id,
E.parent_stack_id,
K.pid,
K.tid,
K.agent_id,
A.absolute_index AS agent_abs_index,
A.logical_index AS agent_log_index,
A.type_index AS agent_type_index,
A.type AS agent_type,
K.queue_id,
k.grid_size_x AS grid_size_x,
k.grid_size_y AS grid_size_y,
k.grid_size_z AS grid_size_z,
(K.grid_size_x * K.grid_size_y * K.grid_size_z) AS grid_size,
S.display_name AS kernel_name,
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = K.region_name_id
AND RS.guid = K.guid
) AS kernel_region,
K.workgroup_size_x AS workgroup_size_x,
K.workgroup_size_y AS workgroup_size_y,
K.workgroup_size_z AS workgroup_size_z,
(K.workgroup_size_x * K.workgroup_size_y * K.workgroup_size_z) AS workgroup_size,
K.group_segment_size AS lds_block_size,
K.private_segment_size AS scratch_size,
S.arch_vgpr_count AS vgpr_count,
S.accum_vgpr_count,
S.sgpr_count,
PMC_I.name AS counter_name,
PMC_I.symbol AS counter_symbol,
PMC_I.component,
PMC_I.description,
PMC_I.block,
PMC_I.expression,
PMC_I.value_type,
PMC_I.id AS counter_id,
SUM(PMC_E.value) AS value,
K.start,
K.end,
PMC_I.is_constant,
PMC_I.is_derived,
(K.end - K.start) AS duration,
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) AS category,
K.nid,
E.extdata,
S.code_object_id
FROM
`rocpd_pmc_event` PMC_E
INNER JOIN `rocpd_info_pmc` PMC_I ON PMC_I.id = PMC_E.pmc_id
AND PMC_I.guid = PMC_E.guid
INNER JOIN `rocpd_event` E ON E.id = PMC_E.event_id
AND E.guid = PMC_E.guid
INNER JOIN `rocpd_kernel_dispatch` K ON K.event_id = PMC_E.event_id
AND K.guid = PMC_E.guid
INNER JOIN `rocpd_info_agent` A ON A.id = K.agent_id
AND A.guid = K.guid
INNER JOIN `rocpd_info_kernel_symbol` S ON S.id = K.kernel_id
AND S.guid = K.guid
GROUP BY
PMC_E.guid,
K.dispatch_id,
PMC_I.name,
K.agent_id;
--
-- RCCL API calls
CREATE VIEW IF NOT EXISTS
`rccl` AS
SELECT
R.id,
R.guid,
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) AS category,
S.string AS name,
R.nid,
R.pid,
R.tid,
R.start,
R.end,
(R.end - R.start) AS duration,
R.event_id,
E.stack_id,
E.parent_stack_id,
E.correlation_id AS corr_id,
E.extdata,
E.call_stack,
E.line_info
FROM
`rocpd_region` R
INNER JOIN `rocpd_event` E ON E.id = R.event_id
AND E.guid = R.guid
INNER JOIN `rocpd_string` S ON S.id = R.name_id
AND S.guid = R.guid
WHERE
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) LIKE 'RCCL_%'
ORDER BY
R.start ASC;
--
-- ROCJPEG API calls
CREATE VIEW IF NOT EXISTS
`rocjpeg` AS
SELECT
R.id,
R.guid,
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) AS category,
S.string AS name,
R.nid,
R.pid,
R.tid,
R.start,
R.end,
(R.end - R.start) AS duration,
R.event_id,
E.stack_id,
E.parent_stack_id,
E.correlation_id AS corr_id,
E.extdata,
E.call_stack,
E.line_info
FROM
`rocpd_region` R
INNER JOIN `rocpd_event` E ON E.id = R.event_id
AND E.guid = R.guid
INNER JOIN `rocpd_string` S ON S.id = R.name_id
AND S.guid = R.guid
WHERE
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) LIKE 'ROCJPEG_%'
ORDER BY
R.start ASC;
--
-- ROCDECODE API calls
CREATE VIEW IF NOT EXISTS
`rocdecode` AS
SELECT
R.id,
R.guid,
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) AS category,
S.string AS name,
R.nid,
R.pid,
R.tid,
R.start,
R.end,
(R.end - R.start) AS duration,
R.event_id,
E.stack_id,
E.parent_stack_id,
E.correlation_id AS corr_id,
E.extdata,
E.call_stack,
E.line_info
FROM
`rocpd_region` R
INNER JOIN `rocpd_event` E ON E.id = R.event_id
AND E.guid = R.guid
INNER JOIN `rocpd_string` S ON S.id = R.name_id
AND S.guid = R.guid
WHERE
(
SELECT
string
FROM
`rocpd_string` RS
WHERE
RS.id = E.category_id
AND RS.guid = E.guid
) LIKE 'ROCDECODE_%'
ORDER BY
R.start ASC;
-- ALL API call regions in one go
CREATE VIEW IF NOT EXISTS
`api_regions` AS
WITH
category_strings AS (
SELECT
RS.id,
RS.guid,
RS.string AS category_string
FROM
`rocpd_string` RS
)
SELECT
R.id,
R.guid,
CS.category_string AS category,
S.string AS name,
R.nid,
R.pid,
R.tid,
R.start,
R.end,
(R.end - R.start) AS duration,
R.event_id,
E.stack_id,
E.parent_stack_id,
E.correlation_id AS corr_id,
E.extdata,
E.call_stack,
E.line_info
FROM
`rocpd_region` R
INNER JOIN `rocpd_event` E ON E.id = R.event_id
AND E.guid = R.guid
INNER JOIN `rocpd_string` S ON S.id = R.name_id
AND S.guid = R.guid
INNER JOIN category_strings CS ON CS.id = E.category_id
AND CS.guid = E.guid
WHERE
CS.category_string LIKE '%HIP_%'
OR CS.category_string LIKE '%HSA_%'
OR CS.category_string LIKE 'RCCL_%'
OR CS.category_string LIKE 'ROCJPEG_%'
OR CS.category_string LIKE 'ROCDECODE_%'
ORDER BY
R.start ASC;
-- Threads participating in API calls
CREATE VIEW IF NOT EXISTS
`api_threads` AS
SELECT DISTINCT
N.id AS nid,
N.machine_id,
N.system_name,
N.hostname,
N.release AS system_release,
N.version AS system_version,
P.guid,
P.ppid,
P.id AS pid,
T.id AS tid,
T.start,
T.end,
T.name
FROM
`rocpd_info_thread` T
INNER JOIN `rocpd_info_process` P ON P.id = T.pid
AND P.guid = T.guid
INNER JOIN `rocpd_info_node` N ON N.id = T.nid
AND N.guid = T.guid
INNER JOIN (
SELECT DISTINCT
tid,
guid
FROM
`api_regions`
UNION ALL
SELECT DISTINCT
tid,
guid
FROM
`kernels`
UNION ALL
SELECT DISTINCT
tid,
guid
FROM
`memory_copies`
UNION ALL
SELECT DISTINCT
tid,
guid
FROM
`memory_allocations`
UNION ALL
SELECT DISTINCT
tid,
guid
FROM
`markers`
) AS activity ON activity.tid = T.id
AND activity.guid = T.guid;
source/share/rocprofiler-sdk-rocpd/marker_views.sql
+218
Просмотреть файл
@@ -0,0 +1,218 @@
--
-- Views related to markers
--
--
--
CREATE VIEW
`range_markers` AS
SELECT
E.id,
E.guid,
ST.string AS category,
JSON_EXTRACT(E.extdata, '$.message') AS name,
R.start,
R.end,
(R.end - R.start) AS duration,
E.stack_id,
E.parent_stack_id,
E.correlation_id,
E.call_stack,
E.line_info,
R.pid,
R.tid,
R.nid,
ST2.string AS operation
FROM
`rocpd_event` E
INNER JOIN `rocpd_string` ST ON ST.id = E.category_id
AND ST.guid = E.guid
INNER JOIN `rocpd_region` R ON R.event_id = E.id
AND R.guid = E.guid
INNER JOIN `rocpd_string` ST2 ON ST2.id = R.name_id
AND ST2.guid = R.guid
WHERE
ST.string LIKE '%MARKER%'
AND JSON_VALID(E.extdata);
--
--
CREATE VIEW
`single_markers` AS
SELECT
E.id,
E.guid,
ST.string AS category,
JSON_EXTRACT(E.extdata, '$.message') AS name,
S.timestamp AS timestamp,
E.stack_id,
E.parent_stack_id,
E.correlation_id,
E.call_stack,
E.line_info,
T.pid AS pid,
T.tid AS tid,
T.nid
FROM
`rocpd_event` E
INNER JOIN `rocpd_string` ST ON ST.id = E.category_id
AND ST.guid = E.guid
INNER JOIN `rocpd_sample` S ON S.event_id = E.id
AND S.guid = E.guid
INNER JOIN `rocpd_track` T ON T.id = S.track_id
AND S.guid = T.guid
WHERE
ST.string LIKE '%MARKER%';
--
--
CREATE VIEW
`markers` AS
SELECT
*
FROM
`range_markers`
UNION
SELECT
SM.id,
SM.guid,
SM.category,
SM.name,
SM.timestamp AS start,
SM.timestamp AS end,
0 AS duration,
SM.stack_id,
SM.parent_stack_id,
SM.correlation_id,
SM.call_stack,
SM.line_info,
SM.pid,
SM.tid,
SM.nid,
'roctxMarkA' AS operation
FROM
`single_markers` SM;
--
--
CREATE VIEW
`range_marker_summary` AS
WITH
avg_data AS (
SELECT
name,
AVG(duration) AS avg_duration
FROM
`range_markers`
GROUP BY
name
),
aggregated_data AS (
SELECT
RM.name,
COUNT(*) AS calls,
SUM(RM.duration) AS total_duration,
A.avg_duration AS average_duration,
MIN(RM.duration) AS min_duration,
MAX(RM.duration) AS max_duration,
SQRT(AVG((RM.duration - A.avg_duration) * (RM.duration - A.avg_duration))) AS std_dev_duration
FROM
`range_markers` RM
JOIN avg_data A ON RM.name = A.name
GROUP BY
RM.name
),
total_duration AS (
SELECT
SUM(total_duration) AS grand_total_duration
FROM
aggregated_data
)
SELECT
AD.name AS name,
AD.calls,
AD.total_duration AS `DURATION (nsec)`,
AD.average_duration AS `AVERAGE (nsec)`,
(CAST(AD.total_duration AS REAL) / TD.grand_total_duration) * 100 AS `PERCENT (INC)`,
AD.min_duration AS `MIN (nsec)`,
AD.max_duration AS `MAX (nsec)`,
AD.std_dev_duration AS `STD_DEV`
FROM
aggregated_data AD
CROSS JOIN total_duration TD;
--
--
CREATE VIEW
`single_marker_summary` AS
SELECT
SM.name,
COUNT(*) AS calls
FROM
`single_markers` SM
GROUP BY
SM.name;
--
-- Markers summary
CREATE VIEW
`marker_summary` AS
WITH
all_markers AS (
SELECT
name,
duration
FROM
`range_markers`
UNION ALL
SELECT
name,
0 AS duration
FROM
`single_markers`
),
avg_data AS (
SELECT
name,
AVG(duration) AS avg_duration
FROM
all_markers
GROUP BY
name
),
aggregated_data AS (
SELECT
M.name,
COUNT(*) AS calls,
SUM(M.duration) AS total_duration,
CAST(SUM(M.duration * M.duration) AS REAL) AS sqr_duration,
A.avg_duration AS average_duration,
MIN(M.duration) AS min_duration,
MAX(M.duration) AS max_duration,
AVG((M.duration - A.avg_duration) * (M.duration - A.avg_duration)) AS variance_duration,
SQRT(AVG((M.duration - A.avg_duration) * (M.duration - A.avg_duration))) AS std_dev_duration
FROM
all_markers M
JOIN avg_data A ON M.name = A.name
GROUP BY
M.name
),
total_duration AS (
SELECT
SUM(total_duration) AS grand_total_duration
FROM
aggregated_data
)
SELECT
AD.name AS name,
AD.calls,
AD.total_duration AS `DURATION (nsec)`,
AD.sqr_duration AS `SQR (nsec)`,
AD.average_duration AS `AVERAGE (nsec)`,
(CAST(AD.total_duration AS REAL) / NULLIF(TD.grand_total_duration, 0)) * 100 AS `PERCENT (INC)`,
AD.min_duration AS `MIN (nsec)`,
AD.max_duration AS `MAX (nsec)`,
AD.variance_duration AS `VARIANCE`,
AD.std_dev_duration AS `STD_DEV`
FROM
aggregated_data AD
CROSS JOIN total_duration TD;
source/share/rocprofiler-sdk-rocpd/rocpd_indexes.sql
+27
Просмотреть файл
@@ -0,0 +1,27 @@
--
-- Indexes for the various fields
--
-- string field
-- CREATE INDEX "rocpd_string{{uuid}}_string_idx" ON "rocpd_string{{uuid}}" ("string");
-- guid field
-- CREATE INDEX "rocpd_string{{uuid}}_guid_idx" ON "rocpd_string{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_info_node{{uuid}}_guid_idx" ON "rocpd_info_node{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_info_process{{uuid}}_guid_idx" ON "rocpd_info_process{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_info_thread{{uuid}}_guid_idx" ON "rocpd_info_thread{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_info_agent{{uuid}}_guid_idx" ON "rocpd_info_agent{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_info_queue{{uuid}}_guid_idx" ON "rocpd_info_queue{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_info_stream{{uuid}}_guid_idx" ON "rocpd_info_stream{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_info_pmc{{uuid}}_guid_idx" ON "rocpd_info_pmc{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_info_code_object{{uuid}}_guid_idx" ON "rocpd_info_code_object{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_info_kernel_symbol{{uuid}}_guid_idx" ON "rocpd_info_kernel_symbol{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_track{{uuid}}_guid_idx" ON "rocpd_track{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_event{{uuid}}_guid_idx" ON "rocpd_event{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_arg{{uuid}}_guid_idx" ON "rocpd_arg{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_pmc_event{{uuid}}_guid_idx" ON "rocpd_pmc_event{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_region{{uuid}}_guid_idx" ON "rocpd_region{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_sample{{uuid}}_guid_idx" ON "rocpd_sample{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_kernel_dispatch{{uuid}}_guid_idx" ON "rocpd_kernel_dispatch{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_memory_copy{{uuid}}_guid_idx" ON "rocpd_memory_copy{{uuid}}" ("guid");
-- CREATE INDEX "rocpd_memory_allocate{{uuid}}_guid_idx" ON "rocpd_memory_allocate{{uuid}}" ("guid");
source/share/rocprofiler-sdk-rocpd/rocpd_tables.sql
+373
Просмотреть файл
@@ -0,0 +1,373 @@
-- Enable foreign key support for cascading
PRAGMA foreign_keys = ON;
CREATE TABLE IF NOT EXISTS
"rocpd_metadata{{uuid}}" (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"tag" TEXT NOT NULL,
"value" TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS
`rocpd_string{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"string" TEXT NOT NULL UNIQUE ON CONFLICT ABORT
);
CREATE TABLE IF NOT EXISTS
`rocpd_info_node{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"hash" BIGINT NOT NULL UNIQUE,
"machine_id" TEXT NOT NULL UNIQUE,
"system_name" TEXT,
"hostname" TEXT,
"release" TEXT,
"version" TEXT,
"hardware_name" TEXT,
"domain_name" TEXT
);
CREATE TABLE IF NOT EXISTS
`rocpd_info_process{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"nid" INTEGER NOT NULL,
"ppid" INTEGER,
"pid" INTEGER NOT NULL,
"init" BIGINT,
"fini" BIGINT,
"start" BIGINT,
"end" BIGINT,
"command" TEXT,
"environment" JSONB DEFAULT "{}" NOT NULL,
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (nid) REFERENCES `rocpd_info_node{{uuid}}` (id) ON UPDATE CASCADE
);
CREATE TABLE IF NOT EXISTS
`rocpd_info_thread{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"nid" INTEGER NOT NULL,
"ppid" INTEGER,
"pid" INTEGER NOT NULL,
"tid" INTEGER NOT NULL,
"name" TEXT,
"start" BIGINT,
"end" BIGINT,
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (nid) REFERENCES `rocpd_info_node{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (pid) REFERENCES `rocpd_info_process{{uuid}}` (id) ON UPDATE CASCADE
);
CREATE TABLE IF NOT EXISTS
`rocpd_info_agent{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"nid" INTEGER NOT NULL,
"pid" INTEGER NOT NULL,
"type" TEXT CHECK ("type" IN ('CPU', 'GPU')),
"absolute_index" INTEGER,
"logical_index" INTEGER,
"type_index" INTEGER,
"uuid" INTEGER,
"name" TEXT,
"model_name" TEXT,
"vendor_name" TEXT,
"product_name" TEXT,
"user_name" TEXT,
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (nid) REFERENCES `rocpd_info_node{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (pid) REFERENCES `rocpd_info_process{{uuid}}` (id) ON UPDATE CASCADE
);
CREATE TABLE IF NOT EXISTS
`rocpd_info_queue{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"nid" INTEGER NOT NULL,
"pid" INTEGER NOT NULL,
"name" TEXT,
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (nid) REFERENCES `rocpd_info_node{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (pid) REFERENCES `rocpd_info_process{{uuid}}` (id) ON UPDATE CASCADE
);
CREATE TABLE IF NOT EXISTS
`rocpd_info_stream{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"nid" INTEGER NOT NULL,
"pid" INTEGER NOT NULL,
"name" TEXT,
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (nid) REFERENCES `rocpd_info_node{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (pid) REFERENCES `rocpd_info_process{{uuid}}` (id) ON UPDATE CASCADE
);
-- 2993533, 2269219937, 2993533
-- 2993533, 2269219937, 2993533
-- Performance monitoring counters (PMC) descriptions
CREATE TABLE IF NOT EXISTS
`rocpd_info_pmc{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"nid" INTEGER NOT NULL,
"pid" INTEGER NOT NULL,
"agent_id" INTEGER,
"target_arch" TEXT CHECK ("target_arch" IN ('CPU', 'GPU')),
"event_code" INT,
"instance_id" INTEGER,
"name" TEXT NOT NULL,
"symbol" TEXT NOT NULL,
"description" TEXT,
"long_description" TEXT DEFAULT "",
"component" TEXT,
"units" TEXT DEFAULT "",
"value_type" TEXT CHECK ("value_type" IN ('ABS', 'ACCUM', 'RELATIVE')),
"block" TEXT,
"expression" TEXT,
"is_constant" INTEGER,
"is_derived" INTEGER,
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (nid) REFERENCES `rocpd_info_node{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (pid) REFERENCES `rocpd_info_process{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (agent_id) REFERENCES `rocpd_info_agent{{uuid}}` (id) ON UPDATE CASCADE
);
CREATE TABLE IF NOT EXISTS
`rocpd_info_code_object{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"nid" INTEGER NOT NULL,
"pid" INTEGER NOT NULL,
"agent_id" INTEGER,
"uri" TEXT,
"load_base" BIGINT,
"load_size" BIGINT,
"load_delta" BIGINT,
"storage_type" TEXT CHECK ("storage_type" IN ('FILE', 'MEMORY')),
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (nid) REFERENCES `rocpd_info_node{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (pid) REFERENCES `rocpd_info_process{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (agent_id) REFERENCES `rocpd_info_agent{{uuid}}` (id) ON UPDATE CASCADE
);
CREATE TABLE IF NOT EXISTS
`rocpd_info_kernel_symbol{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"nid" INTEGER NOT NULL,
"pid" INTEGER NOT NULL,
"code_object_id" INTEGER NOT NULL,
"kernel_name" TEXT,
"display_name" TEXT,
"kernel_object" INTEGER,
"kernarg_segment_size" INTEGER,
"kernarg_segment_alignment" INTEGER,
"group_segment_size" INTEGER,
"private_segment_size" INTEGER,
"sgpr_count" INTEGER,
"arch_vgpr_count" INTEGER,
"accum_vgpr_count" INTEGER,
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (nid) REFERENCES `rocpd_info_node{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (pid) REFERENCES `rocpd_info_process{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (code_object_id) REFERENCES `rocpd_info_code_object{{uuid}}` (id) ON UPDATE CASCADE
);
-- Stores repetitive info for samples
CREATE TABLE IF NOT EXISTS
`rocpd_track{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"nid" INTEGER NOT NULL,
"pid" INTEGER,
"tid" INTEGER,
"name_id" INTEGER,
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (nid) REFERENCES `rocpd_info_node{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (pid) REFERENCES `rocpd_info_process{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (tid) REFERENCES `rocpd_info_thread{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (name_id) REFERENCES `rocpd_string{{uuid}}` (id) ON UPDATE CASCADE
);
-- Storage for a region, instant, and counter
CREATE TABLE IF NOT EXISTS
`rocpd_event{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"category_id" INTEGER,
"stack_id" INTEGER,
"parent_stack_id" INTEGER,
"correlation_id" INTEGER,
"call_stack" JSONB DEFAULT "{}" NOT NULL,
"line_info" JSONB DEFAULT "{}" NOT NULL,
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (category_id) REFERENCES `rocpd_string{{uuid}}` (id) ON UPDATE CASCADE
);
-- stores arguments for events
CREATE TABLE IF NOT EXISTS
`rocpd_arg{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"event_id" INTEGER NOT NULL,
"position" INTEGER NOT NULL,
"type" TEXT NOT NULL,
"name" TEXT NOT NULL,
"value" TEXT, -- TODO: discuss make it value_id and integer, refer to string table --
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (event_id) REFERENCES `rocpd_event{{uuid}}` (id) ON UPDATE CASCADE
);
-- Region with a start/stop on the same thread (CPU)
CREATE TABLE IF NOT EXISTS
`rocpd_pmc_event{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"event_id" INTEGER,
"pmc_id" INTEGER NOT NULL,
"value" REAL DEFAULT 0.0,
"extdata" JSONB DEFAULT "{}",
FOREIGN KEY (pmc_id) REFERENCES `rocpd_info_pmc{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (event_id) REFERENCES `rocpd_event{{uuid}}` (id) ON UPDATE CASCADE
);
-- Region with a start/stop on the same thread (CPU)
CREATE TABLE IF NOT EXISTS
`rocpd_region{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"nid" INTEGER NOT NULL,
"pid" INTEGER NOT NULL,
"tid" INTEGER NOT NULL,
"start" BIGINT NOT NULL,
"end" BIGINT NOT NULL,
"name_id" INTEGER NOT NULL,
"event_id" INTEGER,
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (nid) REFERENCES `rocpd_info_node{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (pid) REFERENCES `rocpd_info_process{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (tid) REFERENCES `rocpd_info_thread{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (name_id) REFERENCES `rocpd_string{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (event_id) REFERENCES `rocpd_event{{uuid}}` (id) ON UPDATE CASCADE
);
-- Instantaneous sample
CREATE TABLE IF NOT EXISTS
`rocpd_sample{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"track_id" INTEGER NOT NULL,
"timestamp" BIGINT NOT NULL,
"event_id" INTEGER,
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (track_id) REFERENCES `rocpd_track{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (event_id) REFERENCES `rocpd_event{{uuid}}` (id) ON UPDATE CASCADE
);
CREATE TABLE IF NOT EXISTS
`rocpd_kernel_dispatch{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"nid" INTEGER NOT NULL,
"pid" INTEGER NOT NULL,
"tid" INTEGER,
"agent_id" INTEGER NOT NULL,
"kernel_id" INTEGER NOT NULL,
"dispatch_id" INTEGER NOT NULL,
"queue_id" INTEGER NOT NULL,
"stream_id" INTEGER NOT NULL,
"start" BIGINT NOT NULL,
"end" BIGINT NOT NULL,
"private_segment_size" INTEGER,
"group_segment_size" INTEGER,
"workgroup_size_x" INTEGER NOT NULL,
"workgroup_size_y" INTEGER NOT NULL,
"workgroup_size_z" INTEGER NOT NULL,
"grid_size_x" INTEGER NOT NULL,
"grid_size_y" INTEGER NOT NULL,
"grid_size_z" INTEGER NOT NULL,
"region_name_id" INTEGER,
"event_id" INTEGER,
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (nid) REFERENCES `rocpd_info_node{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (pid) REFERENCES `rocpd_info_process{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (tid) REFERENCES `rocpd_info_thread{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (agent_id) REFERENCES `rocpd_info_agent{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (kernel_id) REFERENCES `rocpd_info_kernel_symbol{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (queue_id) REFERENCES `rocpd_info_queue{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (stream_id) REFERENCES `rocpd_info_stream{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (region_name_id) REFERENCES `rocpd_string{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (event_id) REFERENCES `rocpd_event{{uuid}}` (id) ON UPDATE CASCADE
);
CREATE TABLE IF NOT EXISTS
`rocpd_memory_copy{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"nid" INTEGER NOT NULL,
"pid" INTEGER NOT NULL,
"tid" INTEGER,
"start" BIGINT NOT NULL,
"end" BIGINT NOT NULL,
"name_id" INTEGER NOT NULL,
"dst_agent_id" INTEGER,
"dst_address" INTEGER,
"src_agent_id" INTEGER,
"src_address" INTEGER,
"size" INTEGER NOT NULL,
"queue_id" INTEGER,
"stream_id" INTEGER,
"region_name_id" INTEGER,
"event_id" INTEGER,
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (nid) REFERENCES `rocpd_info_node{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (pid) REFERENCES `rocpd_info_process{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (tid) REFERENCES `rocpd_info_thread{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (name_id) REFERENCES `rocpd_string{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (dst_agent_id) REFERENCES `rocpd_info_agent{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (src_agent_id) REFERENCES `rocpd_info_agent{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (stream_id) REFERENCES `rocpd_info_stream{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (queue_id) REFERENCES `rocpd_info_queue{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (region_name_id) REFERENCES `rocpd_string{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (event_id) REFERENCES `rocpd_event{{uuid}}` (id) ON UPDATE CASCADE
);
-- Memory allocations (real memory, virtual memory, and scratch memory)
CREATE TABLE IF NOT EXISTS
`rocpd_memory_allocate{{uuid}}` (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"guid" TEXT DEFAULT "{{guid}}" NOT NULL,
"nid" INTEGER NOT NULL,
"pid" INTEGER NOT NULL,
"tid" INTEGER,
"agent_id" INTEGER,
"type" TEXT CHECK ("type" IN ('ALLOC', 'FREE', 'REALLOC', 'RECLAIM')),
"level" TEXT CHECK ("level" IN ('REAL', 'VIRTUAL', 'SCRATCH')),
"start" BIGINT NOT NULL,
"end" BIGINT NOT NULL,
"address" INTEGER,
"size" INTEGER NOT NULL,
"queue_id" INTEGER,
"stream_id" INTEGER,
"event_id" INTEGER,
"extdata" JSONB DEFAULT "{}" NOT NULL,
FOREIGN KEY (nid) REFERENCES `rocpd_info_node{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (pid) REFERENCES `rocpd_info_process{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (tid) REFERENCES `rocpd_info_thread{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (agent_id) REFERENCES `rocpd_info_agent{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (stream_id) REFERENCES `rocpd_info_stream{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (queue_id) REFERENCES `rocpd_info_queue{{uuid}}` (id) ON UPDATE CASCADE,
FOREIGN KEY (event_id) REFERENCES `rocpd_event{{uuid}}` (id) ON UPDATE CASCADE
);
INSERT INTO
`rocpd_metadata{{uuid}}` ("tag", "value")
VALUES
("schema_version", "3"),
("uuid", "{{uuid}}"),
("guid", "{{guid}}");
source/share/rocprofiler-sdk-rocpd/rocpd_views.sql
+139
Просмотреть файл
@@ -0,0 +1,139 @@
CREATE VIEW IF NOT EXISTS
`rocpd_metadata` AS
SELECT
*
FROM
`rocpd_metadata{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_string` AS
SELECT
*
FROM
`rocpd_string{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_info_node` AS
SELECT
*
FROM
`rocpd_info_node{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_info_process` AS
SELECT
*
FROM
`rocpd_info_process{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_info_thread` AS
SELECT
*
FROM
`rocpd_info_thread{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_info_agent` AS
SELECT
*
FROM
`rocpd_info_agent{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_info_queue` AS
SELECT
*
FROM
`rocpd_info_queue{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_info_stream` AS
SELECT
*
FROM
`rocpd_info_stream{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_info_pmc` AS
SELECT
*
FROM
`rocpd_info_pmc{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_info_code_object` AS
SELECT
*
FROM
`rocpd_info_code_object{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_info_kernel_symbol` AS
SELECT
*
FROM
`rocpd_info_kernel_symbol{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_track` AS
SELECT
*
FROM
`rocpd_track{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_event` AS
SELECT
*
FROM
`rocpd_event{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_arg` AS
SELECT
*
FROM
`rocpd_arg{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_pmc_event` AS
SELECT
*
FROM
`rocpd_pmc_event{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_region` AS
SELECT
*
FROM
`rocpd_region{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_sample` AS
SELECT
*
FROM
`rocpd_sample{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_kernel_dispatch` AS
SELECT
*
FROM
`rocpd_kernel_dispatch{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_memory_copy` AS
SELECT
*
FROM
`rocpd_memory_copy{{uuid}}`;
CREATE VIEW IF NOT EXISTS
`rocpd_memory_allocate` AS
SELECT
*
FROM
`rocpd_memory_allocate{{uuid}}`;
source/share/rocprofiler-sdk-rocpd/summary_views.sql
+1389
Просмотреть файл
Разница между файлами не показана из-за своего большого размера Загрузить разницу
source/share/rocprofiler-sdk/CMakeLists.txt
+22 -10
Просмотреть файл
@@ -20,10 +20,28 @@
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
configure_file(
convert-counters-collection-format.py
${PROJECT_BINARY_DIR}/share/rocprofiler-sdk/convert-counters-collection-format.py
COPYONLY)
# non-executable files
set(share_files basic_counters.xml counter_defs.yaml derived_counters.xml)
# executables
set(share_progs convert-counters-collection-format.py)
foreach(_FILE ${share_files})
configure_file(${_FILE} ${PROJECT_BINARY_DIR}/share/rocprofiler-sdk/${_FILE} COPYONLY)
install(
FILES ${PROJECT_BINARY_DIR}/share/rocprofiler-sdk/${_FILE}
DESTINATION share/rocprofiler-sdk
COMPONENT core)
endforeach()
foreach(_FILE ${share_progs})
configure_file(${_FILE} ${PROJECT_BINARY_DIR}/share/rocprofiler-sdk/${_FILE}
USE_SOURCE_PERMISSIONS COPYONLY)
install(
PROGRAMS ${PROJECT_BINARY_DIR}/share/rocprofiler-sdk/${_FILE}
DESTINATION share/rocprofiler-sdk
COMPONENT core)
endforeach()
set(CONVERT_CC_FORMAT_PATH
${PROJECT_BINARY_DIR}/share/rocprofiler-sdk/convert-counters-collection-format.py)
@@ -31,9 +49,3 @@ set(CONVERT_CC_FORMAT_PATH
add_executable(rocprofiler-sdk::convert-counters-collection-format IMPORTED)
set_property(TARGET rocprofiler-sdk::convert-counters-collection-format
PROPERTY IMPORTED_LOCATION ${CONVERT_CC_FORMAT_PATH})
install(
FILES
${PROJECT_BINARY_DIR}/share/rocprofiler-sdk/convert-counters-collection-format.py
DESTINATION share/rocprofiler-sdk
COMPONENT core)
source/lib/rocprofiler-sdk/counters/xml/basic_counters.xml → source/share/rocprofiler-sdk/basic_counters.xml
Исполняемый файл → Обычный файл
Просмотреть файл
source/lib/rocprofiler-sdk/counters/yaml/counter_defs.yaml → source/share/rocprofiler-sdk/counter_defs.yaml
Просмотреть файл
source/lib/rocprofiler-sdk/counters/xml/derived_counters.xml → source/share/rocprofiler-sdk/derived_counters.xml
Исполняемый файл → Обычный файл
Просмотреть файл
tests/pytest-packages/CMakeLists.txt
+2 -1
Просмотреть файл
@@ -28,7 +28,8 @@ if(NOT EXISTS ${CMAKE_CURRENT_BINARY_DIR}/trace_processor_shell.linux_clang_rele
WORLD_EXECUTE)
endif()
set(PYTEST_UTILS_PYTHON_SOURCES __init__.py dotdict.py perfetto_reader.py otf2_reader.py)
set(PYTEST_UTILS_PYTHON_SOURCES __init__.py dotdict.py perfetto_reader.py otf2_reader.py
rocpd_reader.py)
set(TESTS_PYTHON_SOURCES __init__.py rocprofv3.py)
foreach(_FILE ${PYTEST_UTILS_PYTHON_SOURCES})
tests/pytest-packages/pytest_utils/rocpd_reader.py
+37
Просмотреть файл
@@ -0,0 +1,37 @@
# MIT License
#
# Copyright (c) 2023-2025 Advanced Micro Devices, Inc. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from __future__ import absolute_import
import sqlite3
class RocpdReader:
"""Read in rocpd database output"""
def __init__(self, filename):
self.filename = filename if isinstance(filename, (list, tuple)) else [filename]
def read(self, **kwargs):
"""Read rocpd (sqlite3) database"""
return [sqlite3.connect(itr) for itr in self.filename]
tests/pytest-packages/tests/rocprofv3.py
+77
Просмотреть файл
@@ -146,3 +146,80 @@ def test_otf2_data(
assert len(_otf2_data) == len(
_json_data
), f"{otf2_category} ({len(_otf2_data)}):\n\t{_otf2_data}\n{json_category} ({len(_json_data)}):\n\t{_json_data}"
def test_rocpd_data(
rocpd_data,
json_data,
categories=(
"hip",
"hsa",
"marker",
"kernel",
"memory_copy",
"memory_allocation",
"rocdecode_api",
"rocjpeg_api",
),
):
mapping = {
"hip": (
"hip_api",
(
"HIP_COMPILER_API",
"HIP_COMPILER_API_EXT",
"HIP_RUNTIME_API",
"HIP_RUNTIME_API_EXT",
),
),
"hsa": (
"hsa_api",
(
"HSA_CORE_API",
"HSA_AMD_EXT_API",
"HSA_IMAGE_EXT_API",
"HSA_FINALIZE_EXT_API",
),
),
"marker": (
"marker_api",
("MARKER_CORE_API", "MARKER_CONTROL_API", "MARKER_NAME_API"),
),
"kernel": ("kernel_dispatch", ("KERNEL_DISPATCH")),
"memory_copy": ("memory_copy", ("MEMORY_COPY")),
"memory_allocation": ("memory_allocation", ("MEMORY_ALLOCATION")),
"rocdecode_api": ("rocdecode_api", ("ROCDECODE_API")),
"rocjpeg_api": ("rocjpeg_api", ("ROCJPEG_API")),
}
view_mapping = {
"hip_api": "regions",
"hsa_api": "regions",
"marker_api": "markers",
"rccl_api": "regions",
"rocdecode_api": "regions",
"rocjpeg_api": "regions",
"kernel_dispatch": "kernels",
"memory_copy": "memory_copies",
"memory_allocation": "memory_allocations",
}
# make sure they specified valid categories
for itr in categories:
assert itr in mapping.keys()
for js_category, rpd_category in [
itr for key, itr in mapping.items() if key in categories
]:
_js_data = json_data["rocprofiler-sdk-tool"]["buffer_records"][js_category]
_rpd_cats = (
rpd_category if isinstance(rpd_category, (list, tuple)) else [rpd_category]
)
_rpd_cond = " OR ".join([f"category = '{itr}'" for itr in _rpd_cats])
_rpd_query = f"SELECT * FROM {view_mapping[js_category]} WHERE {_rpd_cond}"
_rpd_data = rocpd_data.execute(_rpd_query).fetchall()
assert len(_rpd_data) == len(
_js_data
), f"query: {_rpd_query}\n{rpd_category} ({len(_rpd_data)}):\n\t{_rpd_data}\n{js_category} ({len(_js_data)}):\n\t{_js_data}"
tests/rocprofv3/counter-collection/input3/CMakeLists.txt
+2 -2
Просмотреть файл
@@ -19,14 +19,14 @@ add_test(
COMMAND
$<TARGET_FILE:rocprofiler-sdk::rocprofv3> -i
${CMAKE_CURRENT_BINARY_DIR}/input.json -d ${CMAKE_CURRENT_BINARY_DIR}/%argt%-cc
-o out_json -- $<TARGET_FILE:simple-transpose>)
-o out_json -f csv -- $<TARGET_FILE:simple-transpose>)
add_test(
NAME rocprofv3-test-counter-collection-yaml-pmc1-execute
COMMAND
$<TARGET_FILE:rocprofiler-sdk::rocprofv3> -i
${CMAKE_CURRENT_BINARY_DIR}/input.yml -d ${CMAKE_CURRENT_BINARY_DIR}/%argt%-cc -o
out_yaml -- $<TARGET_FILE:simple-transpose>)
out_yaml -f csv -- $<TARGET_FILE:simple-transpose>)
string(REPLACE "LD_PRELOAD=" "ROCPROF_PRELOAD=" PRELOAD_ENV
"${ROCPROFILER_MEMCHECK_PRELOAD_ENV}")
tests/rocprofv3/counter-collection/list_metrics/CMakeLists.txt
+2 -2
Просмотреть файл
@@ -30,12 +30,12 @@ add_test(
COMMAND
$<TARGET_FILE:rocprofiler-sdk::rocprofv3> -d
${CMAKE_CURRENT_BINARY_DIR}/out_counter_collection_2 -o metrics --list-avail
--sys-trace -- $<TARGET_FILE:transpose>)
--sys-trace --log-level warning -- $<TARGET_FILE:transpose>)
string(REPLACE "LD_PRELOAD=" "ROCPROF_PRELOAD=" PRELOAD_ENV
"${ROCPROFILER_MEMCHECK_PRELOAD_ENV}")
set(cc-env-list-metrics "${PRELOAD_ENV}")
set(cc-env-list-metrics "${PRELOAD_ENV}" "ROCPD_LOG_LEVEL=trace")
set_tests_properties(
rocprofv3-test-list-avail-execute
tests/rocprofv3/counter-collection/multiplex/CMakeLists.txt
+2 -2
Просмотреть файл
@@ -41,14 +41,14 @@ add_test(
COMMAND
$<TARGET_FILE:rocprofiler-sdk::rocprofv3> -i
${CMAKE_CURRENT_BINARY_DIR}/input.json -d ${CMAKE_CURRENT_BINARY_DIR}/%argt%-cc
-o out_json -- $<TARGET_FILE:simple-transpose>)
-o out_json -f csv -- $<TARGET_FILE:simple-transpose>)
add_test(
NAME rocprofv3-test-counter-collection-multiple-yaml-execute
COMMAND
$<TARGET_FILE:rocprofiler-sdk::rocprofv3> -i
${CMAKE_CURRENT_BINARY_DIR}/input.yml -d ${CMAKE_CURRENT_BINARY_DIR}/%argt%-cc -o
out_yaml -- $<TARGET_FILE:simple-transpose>)
out_yaml -f csv -- $<TARGET_FILE:simple-transpose>)
string(REPLACE "LD_PRELOAD=" "ROCPROF_PRELOAD=" PRELOAD_ENV
"${ROCPROFILER_MEMCHECK_PRELOAD_ENV}")
tests/rocprofv3/tracing/CMakeLists.txt
+14 -6
Просмотреть файл
@@ -29,7 +29,7 @@ add_test(
COMMAND
$<TARGET_FILE:rocprofiler-sdk::rocprofv3> -M --hsa-trace --kernel-trace
--memory-copy-trace --marker-trace -d ${CMAKE_CURRENT_BINARY_DIR}/%argt%-trace -o
cmdl-input/out --output-format pftrace csv json --log-level ${LOG_LEVEL} --
cmdl-input/out --output-format pftrace csv json rocpd --log-level ${LOG_LEVEL} --
$<TARGET_FILE:simple-transpose>)
set_tests_properties(
@@ -82,7 +82,9 @@ add_test(
--json-input
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-trace/cmdl-input/out_results.json
--pftrace-input
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-trace/cmdl-input/out_results.pftrace)
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-trace/cmdl-input/out_results.pftrace
--rocpd-input
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-trace/cmdl-input/out_results.db)
add_test(
NAME rocprofv3-test-trace-input-json-validate
@@ -100,9 +102,12 @@ add_test(
--json-input
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-trace/json-input/out_results.json
--pftrace-input
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-trace/json-input/out_results.pftrace)
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-trace/json-input/out_results.pftrace
--rocpd-input
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-trace/json-input/out_results.db)
set(VALIDATION_FILES
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-trace/cmdl-input/out_results.db
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-trace/cmdl-input/out_results.pftrace
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-trace/cmdl-input/out_results.json
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-trace/cmdl-input/out_memory_copy_trace.csv
@@ -152,7 +157,7 @@ add_test(
COMMAND
$<TARGET_FILE:rocprofiler-sdk::rocprofv3> --sys-trace -d
${CMAKE_CURRENT_BINARY_DIR}/%argt%-systrace -o cmdl-input/out --output-format
pftrace csv json -- $<TARGET_FILE:simple-transpose>)
pftrace csv json rocpd -- $<TARGET_FILE:simple-transpose>)
set_tests_properties(
rocprofv3-test-systrace-execute
@@ -204,7 +209,8 @@ add_test(
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-systrace/cmdl-input/out_results.json
--pftrace-input
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-systrace/cmdl-input/out_results.pftrace
)
--rocpd-input
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-systrace/cmdl-input/out_results.db)
add_test(
NAME rocprofv3-test-systrace-input-json-validate
@@ -224,9 +230,11 @@ add_test(
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-systrace/json-input/out_results.json
--pftrace-input
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-systrace/json-input/out_results.pftrace
)
--rocpd-input
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-systrace/json-input/out_results.db)
set(SYS_VALIDATION_FILES
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-systrace/cmdl-input/out_results.db
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-systrace/cmdl-input/out_results.pftrace
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-systrace/cmdl-input/out_results.json
${CMAKE_CURRENT_BINARY_DIR}/simple-transpose-systrace/cmdl-input/out_memory_copy_trace.csv
tests/rocprofv3/tracing/conftest.py
+12
Просмотреть файл
@@ -29,6 +29,7 @@ import json
from rocprofiler_sdk.pytest_utils.dotdict import dotdict
from rocprofiler_sdk.pytest_utils import collapse_dict_list
from rocprofiler_sdk.pytest_utils.perfetto_reader import PerfettoReader
from rocprofiler_sdk.pytest_utils.rocpd_reader import RocpdReader
def pytest_addoption(parser):
@@ -72,6 +73,11 @@ def pytest_addoption(parser):
action="store",
help="Path to Perfetto trace file.",
)
parser.addoption(
"--rocpd-input",
action="store",
help="Path to rocpd SQLite3 database file.",
)
@pytest.fixture
@@ -157,3 +163,9 @@ def json_data(request):
def pftrace_data(request):
filename = request.config.getoption("--pftrace-input")
return PerfettoReader(filename).read()[0]
@pytest.fixture
def rocpd_data(request):
filename = request.config.getoption("--rocpd-input")
return RocpdReader(filename).read()[0]
tests/rocprofv3/tracing/input_systrace.json
+2 -1
Просмотреть файл
@@ -6,7 +6,8 @@
"output_format": [
"csv",
"json",
"pftrace"
"pftrace",
"rocpd"
]
}
]
tests/rocprofv3/tracing/input_trace.json
+2 -1
Просмотреть файл
@@ -9,7 +9,8 @@
"output_format": [
"csv",
"json",
"pftrace"
"pftrace",
"rocpd"
]
}
]
tests/rocprofv3/tracing/validate.py
+8
Просмотреть файл
@@ -323,6 +323,14 @@ def test_perfetto_data(pftrace_data, json_data):
)
def test_rocpd_data(rocpd_data, json_data):
import rocprofiler_sdk.tests.rocprofv3 as rocprofv3
rocprofv3.test_rocpd_data(
rocpd_data, json_data, ("hip", "hsa", "marker", "kernel", "memory_copy")
)
if __name__ == "__main__":
exit_code = pytest.main(["-x", __file__] + sys.argv[1:])
sys.exit(exit_code)