AI/rocm-systems
2
0
Fork 0
Código Issues Pull Requests Ações Pacotes Projetos Versões Wiki Atividade

Shared Library Constructor (rocprofv3 deadlock fix) (#599)

Ver Código-Fonte 
* Moved tests/apps to tests/bin

* Renamed cmake project in tests/bin

* Update samples

- Use ROCPROFILER_DEFAULT_FAIL_REGEX
- tweaks to stdout messages

* Update tests

- Use ROCPROFILER_DEFAULT_FAIL_REGEX

* Add tests/lib

- libraries with HIP code

* Update PTL submodule

- remove atexit delete of thread_id_map

* Update cmake/rocprofiler_options.cmake

- Set ROCPROFILER_DEFAULT_FAIL_REGEX

* Update common lib: env + logging

- improved customization of logging settings
- default to disabling logging to files
- install failure handler for rocprofv3
- set_env support in environment.*

* Add lib/rocprofiler-sdk/shared_library.cpp

- shared library constructor

* Update lib/rocprofiler-sdk-tool/tool.cpp

- destructor thread safety
- convert callback_name_info and buffered_name_info to pointers
- install failure handler for logging

* Add tests/bin/hip-in-libraries

- hip-in-libraries is an exe which uses two shared libraries where each shared library contains HIP kernels
  - used for testing deadlocking within __hipRegisterFatBinary

* Update bin/rocprofv3

- reorganized the env variables
- use exec to launch command
- set ROCPROFILER_LIBRARY_CTOR=1

* Add tests/rocprofv3/tracing-hip-in-libraries

- uses hip-in-libraries exe for exe which uses shared libraries to launch HIP kernels

* Update bin/rocprofv3

- fix counter collection (no exec)

* Update lib/rocprofiler-sdk-tool/tool.cpp

- replace "Kernel-Name" with "Kernel_Name"

* Update lib/rocprofiler-sdk/registration.cpp

Use RTLD_LOCAL instead of RTLD_GLOBAL for env libraries

* Update tests/rocprofv3

- replace "Kernel-Name" with "Kernel_Name"

* Update tests

- vector-ops (bin) stream syncs + runs with 4 queues per device
- improve counter-collection/input1 validation
- rocprofv3/tracing-hip-in-libraries does not do sys-trace
- improved validation script for tracing-hip-in-libraries
- updated dispatch_callback in json-tool.cpp following reworking of prototypes for counter collection

* Update samples/counter_collection

- updated dispatch_callback(s) and record_callback(s) following reworking of prototypes

* Update bin/rocprofv3

- reorganized help menu
- added options for sub-HSA tables
- added --hip-runtime-trace
- changed --hip-trace to include --hip-compiler-trace

* Update lib/rocprofiler-sdk-tool

- improved kernel filtering
- removed arch_vgpr, accum_vgpr, sgpr code (in rocprofiler-sdk)
- fixed issue with counter-collection w/o tracing
- added support for fine grained HSA API tracing
- removed directly linking to HSA-runtime

* Update lib/rocprofiler-sdk/agent.cpp

- rocp_agents != hsa_agents is non-fatal when ROCPROFILER_BUILD_CI=OFF (CMake option)

* GPR (vector and scalar) info in kernel symbol data

- rocprofiler_callback_tracing_code_object_kernel_symbol_register_data_t contains general purpose register info

* Header include order fix

- Include repo headers first
- Third party library headers next
- standard library headers last

* Update dispatch profiling public API

- introduce rocprofiler_profile_counting_dispatch_data_t
- change signature of rocprofiler_profile_counting_dispatch_callback_t and rocprofiler_profile_counting_record_callback_t
- provide rocprofiler_user_data_t pointer in dispatch callback
- provide rocprofiler_user_data_t value (from dispatch cb) in record callback

* Update tests/bin/CMakeLists.txt

- fix add_subdirectory(hip-in-libraries) order

* Update VERSION

- bump to 0.2.0 in prep for AFAR
Esse commit está contido em:
Jonathan R. Madsen
2024-03-07 22:21:26 -06:00
commit de GitHub
pai 665c546e65
commit 7b6d3c70bd
85 arquivos alterados com 2497 adições e 856 exclusões
Baixar Arquivo de Patch Baixar Arquivo de Diff Expandir todos os arquivos Colapsar todos os arquivos
source/bin/rocprofv3
+84 -39
Ver Arquivo
@@ -5,9 +5,16 @@ set -eo pipefail
ROCPROFV3_DIR=$(dirname -- "$(realpath "${BASH_SOURCE[0]}")")
ROCM_DIR=$(dirname -- "${ROCPROFV3_DIR}")
: ${HSA_TOOLS_LIB:="${ROCM_DIR}/lib/librocprofiler-sdk.so"}
: ${ROCPROFILER_LIBRARY_CTOR:=1}
: ${ROCPROF_OUTPUT_PATH:="."}
: ${ROCPROF_OUTPUT_PATH_INTERNAL:="."}
: ${ROCPROF_OUTPUT_FILE_NAME:=""}
: ${ROCPROF_COUNTERS_PATH:=""}
: ${ROCPROF_PRELOAD:=""}
: ${ROCPROF_TOOL_LIBRARY:="${ROCM_DIR}/lib/rocprofiler-sdk/librocprofiler-sdk-tool.so"}
: ${ROCPROF_SDK_LIBRARY:="${ROCM_DIR}/lib/librocprofiler-sdk.so"}
export HSA_TOOLS_LIB
export ROCPROFILER_LIBRARY_CTOR
# Define color codes
GREEN='\033[0;32m'
@@ -19,27 +26,40 @@ usage() {
if [ -z "${EC}" ]; then EC=1; fi
echo -e "${RESET}ROCProfilerV3 Run Script Usage:"
echo -e "${GREEN}-h | --help ${RESET} For showing this message"
echo -e ""
echo -e "${GREEN}--hip-trace ${RESET} For Collecting HIP Traces (runtime + compiler)"
echo -e "${GREEN}--hip-runtime-trace ${RESET} For Collecting HIP Runtime API Traces"
echo -e "${GREEN}--hip-compiler-trace ${RESET} For Collecting HIP Compiler generated code Traces"
echo -e ""
echo -e "${GREEN}--marker-trace ${RESET} For Collecting Marker (ROCTx) Traces"
echo -e "${GREEN}--kernel-trace ${RESET} For Collecting Kernel Dispatch Traces"
echo -e "${GREEN}--memory-copy-trace ${RESET} For Collecting Memory Copy Traces"
echo -e ""
echo -e "${GREEN}--hsa-trace ${RESET} For Collecting HSA API Traces (core + amd + image + finalizer)"
echo -e "${GREEN}--hsa-core-trace ${RESET} For Collecting HSA API Traces (core API)"
echo -e "${GREEN}--hsa-amd-trace ${RESET} For Collecting HSA API Traces (AMD-extension API)"
echo -e "${GREEN}--hsa-image-trace ${RESET} For Collecting HSA API Traces (Image-extenson API)"
echo -e "${GREEN}--hsa-finalizer-trace ${RESET} For Collecting HSA API Traces (Finalizer-extension API)"
echo -e ""
echo -e "${GREEN}--sys-trace ${RESET} For Collecting HIP,HSA, Memory Copy, (marker)ROCTx and Kernel dispatch traces\n"
echo -e ""
echo -e "${GREEN}-o | --output-file ${RESET} For the output file name"
echo -e "\t#${GREY} usage (with current dir): rocprofv3 --hsa-trace -o <file_name> <executable>"
echo -e "\t#${GREY} usage (with custom dir): rocprofv3 --hsa-trace -d <out_dir> -o <file_name> <executable>${RESET}\n"
echo -e ""
echo -e "${GREEN}-d | --output-directory ${RESET} For adding output path where the output files will be saved"
echo -e "\t#${GREY} usage (with custom dir): rocprofv3 --hsa-trace -d <out_dir> <executable>${RESET}"
echo -e ""
echo -e "${GREEN}-M | --mangled-kernels ${RESET} Do not demangle the kernel names"
echo -e "${GREEN}-T | --truncate-kernels ${RESET} Truncate the demangled kernel names"
echo -e ""
echo -e "${GREEN}-L | --list-metrics ${RESET} List metrics for counter collection"
echo -e "${GREEN}-i | --input ${RESET} For counter collection "
echo -e "\t#${GREY} Input file .txt format, automatically rerun application for every profiling features line"
echo -e "\t# Perf counters group 1"
echo -e "\tpmc : Wavefronts VALUInsts SALUInsts SFetchInsts FlatVMemInsts LDSInsts"
echo -e "\t# Perf counters group 2"
echo -e "\tpmc : WriteSize L2CacheHit ${RESET}"
echo -e "${GREEN}--hsa-trace ${RESET} For Collecting HSA API Traces"
echo -e "${GREEN}--kernel-trace ${RESET} For Collecting Kernel Dispatch Traces"
echo -e "${GREEN}--memory-copy-trace ${RESET} For Collecting Memory Copy Traces"
echo -e "${GREEN}--marker-trace ${RESET} For Collecting Marker (ROCTx) Traces"
echo -e "${GREEN}--hip-trace ${RESET} For Collecting HIP Runtime Traces"
echo -e "${GREEN}--hip-compiler-trace ${RESET} For Collecting HIP Compiler generated code Traces"
echo -e "${GREEN}--sys-trace ${RESET} For Collecting HIP,HSA, Memory Copy, (marker)ROCTx and Kernel dispatch traces\n"
echo -e "${GREEN}-o | --output-file ${RESET} For the output file name"
echo -e "\t#${GREY} usage e.g:(with current dir): rocprofv3 --hsa-trace -o <file_name> <executable>"
echo -e "\t#${GREY} usage e.g:(with custom dir): rocprofv3 --hsa-trace -d <out_dir> -o <file_name> <executable>${RESET}\n"
echo -e "${GREEN}-d | --output-directory ${RESET} For adding output path where the output files will be saved"
echo -e "\t#${GREY} usage e.g:(with custom dir): rocprofv3 --hsa-trace -d <out_dir> <executable>${RESET}"
echo -e "${GREEN}-M | --mangled-kernels ${RESET} Do not demangle the kernel names"
echo -e "${GREEN}-T | --truncate-kernels ${RESET} Truncate the demangled kernel names"
echo -e "${GREEN}-L | --list-metrics ${RESET} List metrics"
echo -e ""
exit ${EC}
}
@@ -48,11 +68,22 @@ if [ -z "$1" ]; then
usage 1
fi
: ${ROCPROF_OUTPUT_PATH:="."}
: ${ROCPROF_OUTPUT_PATH_INTERNAL:="."}
: ${ROCPROF_OUTPUT_FILE_NAME:=""}
: ${ROCPROF_COUNTERS_PATH:=""}
: ${ROCPROF_PRELOAD:=""}
if [ -n "${ROCPROF_PRELOAD}" ]; then
ROCPROF_PRELOAD="${ROCPROF_PRELOAD}:${ROCPROF_TOOL_LIBRARY}:${ROCPROF_SDK_LIBRARY}"
else
ROCPROF_PRELOAD="${ROCPROF_TOOL_LIBRARY}:${ROCPROF_SDK_LIBRARY}"
fi
if [ -n "${ROCP_TOOL_LIBRARIES}" ]; then
ROCP_TOOL_LIBRARIES="${ROCP_TOOL_LIBRARIES}:${ROCPROF_TOOL_LIBRARY}"
else
ROCP_TOOL_LIBRARIES="${ROCPROF_TOOL_LIBRARY}"
fi
LD_LIBRARY_PATH=${ROCM_DIR}/lib:${LD_LIBRARY_PATH}
export ROCP_TOOL_LIBRARIES
export LD_LIBRARY_PATH
while true; do
if [[ "$1" == "-h" || "$1" == "--help" ]]; then
@@ -93,14 +124,26 @@ while true; do
shift
shift
elif [ "$1" == "--hsa-trace" ]; then
export ROCPROF_HSA_API_TRACE=1
export ROCPROF_HSA_CORE_API_TRACE=1
export ROCPROF_HSA_AMD_EXT_API_TRACE=1
export ROCPROF_HSA_IMAGE_EXT_API_TRACE=1
export ROCPROF_HSA_FINALIZER_EXT_API_TRACE=1
shift
elif [ "$1" == "--hsa-core-trace" ]; then
export ROCPROF_HSA_CORE_API_TRACE=1
shift
elif [ "$1" == "--hsa-amd-trace" ]; then
export ROCPROF_HSA_AMD_EXT_API_TRACE=1
shift
elif [ "$1" == "--hsa-image-trace" ]; then
export ROCPROF_HSA_IMAGE_EXT_API_TRACE=1
shift
elif [ "$1" == "--hsa-finalizer-trace" ]; then
export ROCPROF_HSA_FINALIZER_EXT_API_TRACE=1
shift
elif [[ "$1" == "-L" || "$1" == "--list-metrics" ]]; then
export ROCPROF_LIST_METRICS=1
ROCP_TOOL_LIBRARIES="${ROCM_DIR}/lib/rocprofiler-sdk/librocprofiler-sdk-tool.so" \
LD_LIBRARY_PATH=${ROCM_DIR}/lib:${LD_LIBRARY_PATH} \
LD_PRELOAD="${ROCPROF_PRELOAD}:${ROCM_DIR}/lib/librocprofiler-sdk.so" \
exec ${ROCM_DIR}/lib/rocprofiler-sdk/rocprofv3-trigger-list-metrics
LD_PRELOAD="${ROCPROF_PRELOAD}" exec ${ROCM_DIR}/lib/rocprofiler-sdk/rocprofv3-trigger-list-metrics
elif [ "$1" == "--kernel-trace" ]; then
export ROCPROF_KERNEL_TRACE=1
shift
@@ -111,17 +154,25 @@ while true; do
export ROCPROF_MARKER_API_TRACE=1
shift
elif [ "$1" == "--hip-trace" ]; then
export ROCPROF_HIP_API_TRACE=1
export ROCPROF_HIP_RUNTIME_API_TRACE=1
export ROCPROF_HIP_COMPILER_API_TRACE=1
shift
elif [ "$1" == "--hip-runtime-trace" ]; then
export ROCPROF_HIP_RUNTIME_API_TRACE=1
shift
elif [ "$1" == "--hip-compiler-trace" ]; then
export ROCPROF_HIP_COMPILER_API_TRACE=1
shift
elif [ "$1" == "--sys-trace" ]; then
export ROCPROF_HSA_API_TRACE=1
export ROCPROF_HSA_CORE_API_TRACE=1
export ROCPROF_HSA_AMD_EXT_API_TRACE=1
export ROCPROF_HSA_IMAGE_EXT_API_TRACE=1
export ROCPROF_HSA_FINALIZER_EXT_API_TRACE=1
export ROCPROF_HIP_RUNTIME_API_TRACE=1
export ROCPROF_HIP_COMPILER_API_TRACE=1
export ROCPROF_MARKER_API_TRACE=1
export ROCPROF_KERNEL_TRACE=1
export ROCPROF_MEMORY_COPY_TRACE=1
export ROCPROF_MARKER_API_TRACE=1
export ROCPROF_HIP_API_TRACE=1
shift
elif [ "$1" == "--" ]; then
shift
@@ -161,18 +212,12 @@ if [ -n "${PMC_LINES:-}" ]; then
export ROCPROF_OUTPUT_PATH=$RESULT_PATH
fi
((COUNTER++))
ROCP_TOOL_LIBRARIES="${ROCM_DIR}/lib/rocprofiler-sdk/librocprofiler-sdk-tool.so" \
LD_LIBRARY_PATH=${ROCM_DIR}/lib:${LD_LIBRARY_PATH} \
LD_PRELOAD="${ROCPROF_PRELOAD}:${ROCM_DIR}/lib/librocprofiler-sdk.so" \
"${@}"
LD_PRELOAD="${ROCPROF_PRELOAD}" "${@}"
if [ -n "$ROCPROF_OUTPUT_PATH" ]; then
echo -e "\nThe output path for the following counters: $ROCPROF_OUTPUT_PATH"
fi
done
else
# for non counter collection. e.g: tracing
ROCP_TOOL_LIBRARIES="${ROCM_DIR}/lib/rocprofiler-sdk/librocprofiler-sdk-tool.so" \
LD_LIBRARY_PATH=${ROCM_DIR}/lib:${LD_LIBRARY_PATH} \
LD_PRELOAD="${ROCPROF_PRELOAD}:${ROCM_DIR}/lib/librocprofiler-sdk.so" \
"${@}"
LD_PRELOAD="${ROCPROF_PRELOAD}" exec "${@}"
fi