8e1aee62d0
## Motivation Resolved: SWDEV-566226 The current implementation of agents inside of rocprof-systems keeps just the minimal necessary set of information required for populating the `info_agent` table inside of rocpd database. There is a sufficient amount of data that is being left out from database, so this change should fix that and store the additional agent information as an `extdata` row inside of `info_agent` table. ## Technical Details This PR introduces additional filed inside of `agent` structure inside which is representing the JSON formatted string of all the additional information we can acquire about particular agent. This data is processed and added during the initial fetching of agents, and afterwards pushed inside of the database. --------- Co-authored-by: David Galiffi <David.Galiffi@amd.com> * SWDEV-557412 - Incorporate proper chunk offset when remapping virtual memory (#1848) * SWDEV-557412 - Incorporate proper offset when remapping virtual memory * Fix condition to check if VMHeap allocation address matches a chunk address * Move offset calculation outside if/else block --------- Co-authored-by: JeniferC99 <150404595+JeniferC99@users.noreply.github.com> * SWDEV-567852 - Clean-up hip::init() (#1948) * SWDEV-559267 - Use CLPrint to DevLogPrintf with Log Level - detail debug. (#1160) * SWDEV-548892 - Stop using ocml isinf wrapper (#1854) * SWDEV-562708 - change default maximum SVM size to 256GB (#1731) * SWDEV-503089 - Fix and enable disabled HIP tests from math group (#1319) * SWDEV-503089 - Fix and enable disabled HIP tests from math group * SWDEV-503089 - Move single precision reduced run to a common function * SWDEV-548892 - Stop using ockl steadyctr function (#1882) Directly use the builtin * Implement PTL support (#1957) * Implement PTL support Signed-off-by: adapryor <Adam.pryor@amd.com> (cherry picked from commit 45bc31292e7940a3b8fca044ef7df22047b95733) Signed-off-by: Maisam Arif <Maisam.Arif@amd.com> --------- Signed-off-by: adapryor <Adam.pryor@amd.com> Signed-off-by: Maisam Arif <Maisam.Arif@amd.com> Co-authored-by: Maisam Arif <Maisam.Arif@amd.com> * SWDEV-558080 - Add recommended granularity (#1176) * Add recommended granularity * Improve granularity testing * Update based on feedback * Fix and enable VMM tests on cuda (#1855) * Fix and enable VMM tests on cuda * Minor syntax fixes --------- Co-authored-by: Rahul Manocha <rmanocha@amd.com> * [rocprofiler-systems] Add support for ompt_callback_thread_begin (#1681) * Add thread_begin callback * Make OMPT callbacks that are instant have start_ts = end_ts * SWDEV-567514: Remove default stream wait (#1977) - when virtual map command is called - can create deadlock Signed-off-by: sdashmiz <shadi.dashmiz@amd.com> * Fix flaky test Unit_hipStreamAddCallback_StrmSyncTiming (#2022) * Review comments * skip the 3 failing tests to merge hip-tests rocm-systems PR --------- Signed-off-by: Bindhiya Kanangot Balakrishnan <Bindhiya.KanangotBalakrishnan@amd.com> Signed-off-by: adapryor <Adam.pryor@amd.com> Signed-off-by: Maisam Arif <Maisam.Arif@amd.com> Signed-off-by: sdashmiz <shadi.dashmiz@amd.com> Co-authored-by: GunaShekar <agunashe@amd.com> Co-authored-by: agunashe <ajay.gunashekar@amd.com> Co-authored-by: Ethan Trinh <Ethan.Trinh@amd.com> Co-authored-by: JeniferC99 <150404595+JeniferC99@users.noreply.github.com> Co-authored-by: Victor Zhang <111778801+victzhan@users.noreply.github.com> Co-authored-by: German Andryeyev <56892148+gandryey@users.noreply.github.com> Co-authored-by: usrihari123 <srihari.u@amd.com> Co-authored-by: Bindhiya Kanangot Balakrishnan <Bindhiya.KanangotBalakrishnan@amd.com> Co-authored-by: anujshuk-amd <anujshuk@amd.com> Co-authored-by: itrowbri <Ian.Trowbridge@amd.com> Co-authored-by: marantic-amd <marantic@amd.com> Co-authored-by: David Galiffi <David.Galiffi@amd.com> Co-authored-by: cadolphe-amd <chris.adolphe@amd.com> Co-authored-by: Karthik Jayaprakash <54370791+kjayapra-amd@users.noreply.github.com> Co-authored-by: Matt Arsenault <Matthew.Arsenault@amd.com> Co-authored-by: Todd tiantuo Li <88386084+lttamd@users.noreply.github.com> Co-authored-by: amilanov-amd <Aleksandar.Milanov@amd.com> Co-authored-by: Adam Pryor <61172547+adam360x@users.noreply.github.com> Co-authored-by: Maisam Arif <Maisam.Arif@amd.com> Co-authored-by: AidanBeltonS <abeltons@amd.com> Co-authored-by: Rahul Manocha <153310294+manocharahul@users.noreply.github.com> Co-authored-by: Rahul Manocha <rmanocha@amd.com> Co-authored-by: Kian Cossettini <Kian.Cossettini@amd.com> Co-authored-by: Shadi Dashmiz <94885391+shadidashmiz@users.noreply.github.com> Co-authored-by: Ioannis Assiouras <38722728+iassiour@users.noreply.github.com> Co-authored-by: Ajay GunaShekar <86270081+agunashe@users.noreply.github.com>
107 строки
4.9 KiB
Python
107 строки
4.9 KiB
Python
import glob
|
|
import sys
|
|
import os
|
|
import re
|
|
|
|
if len(sys.argv) > 1:
|
|
build_folder = sys.argv[1]
|
|
install_folder = sys.argv[2]
|
|
#print(f"In relative_paths.py [1]: build folder: {build_folder}")
|
|
#print(f"In relative_paths.py [2]: install folder: {install_folder}")
|
|
else:
|
|
print(f"Args not provided. Please provide arg1-build folder-catch_tests, arg2-install folder")
|
|
exit(1)
|
|
|
|
install_folder = os.path.abspath(install_folder)
|
|
install_catch_tests = os.path.join(install_folder, "catch_tests")
|
|
install_script = os.path.join(install_catch_tests, "script")
|
|
|
|
modified_msg = """
|
|
# File has been updated by hip-tests/catch folder for portability
|
|
"""
|
|
ctest_current_str = """
|
|
get_filename_component(FULL_FILE_PATH ${CMAKE_CURRENT_LIST_FILE} REALPATH)
|
|
get_filename_component(CTEST_CURRENT_DIR ${FULL_FILE_PATH} DIRECTORY)
|
|
get_filename_component(EXE_PATH ${CTEST_CURRENT_DIR}/.. REALPATH)
|
|
"""
|
|
if os.name == 'posix':
|
|
library_path_str = """
|
|
set(LIB_PATH ${EXE_PATH}/../../../lib)
|
|
file(GLOB HIP_LIBS "${LIB_PATH}/libamdhip64*")
|
|
if(HIP_LIBS)
|
|
set(LIB_PATH ${LIB_PATH} ${LIB_PATH}/rocm_sysdeps/lib)
|
|
else()
|
|
set(LIB_PATH "/opt/rocm/lib")
|
|
endif()
|
|
"""
|
|
ctest_current_str = ctest_current_str + library_path_str
|
|
|
|
inc_cmake_pattern = "_include.cmake"
|
|
ctesttest_pattern = "CTestTestfile.cmake"
|
|
|
|
def make_test_files_portable(filenames):
|
|
"""
|
|
changes the absolute paths with relative paths
|
|
filenames - list of files to perform the workaround
|
|
"""
|
|
for filename in filenames:
|
|
try:
|
|
filename = os.path.abspath(filename)
|
|
# Read the entire content of the file
|
|
with open(filename, 'r') as file:
|
|
file_content = file.read()
|
|
#print(f"**Done reading now parsing", filename)
|
|
# 1 replace abs path with filename. Make relative
|
|
old_text=os.path.dirname(os.path.abspath(filename))
|
|
abs_dir_path = os.path.join(old_text, '')
|
|
old_text = abs_dir_path.replace("\\", "/")
|
|
if modified_msg not in file_content:
|
|
file_content = modified_msg + file_content
|
|
# Perform the replacement
|
|
modified_content = file_content.replace(old_text, '')
|
|
if inc_cmake_pattern in filename:
|
|
# 2 get folder path relative to the current *_include.cmake
|
|
if r"get_filename_component(FULL_FILE_PATH" not in modified_content:
|
|
modified_content = ctest_current_str + modified_content
|
|
# 3 CTEST_FILE to have parameterized relative file
|
|
test_cmake_pattern = r"\[==\[(\w+-[a-f0-9]+_tests.cmake)\]==\]"
|
|
replace_test_pattern = r"${CTEST_CURRENT_DIR}/\1"
|
|
modified_content = re.sub(test_cmake_pattern, replace_test_pattern, modified_content)
|
|
# 4 script to use CatchAddTests.cmake from build rather than src
|
|
add_test_pattern = r'include\(".*CatchAddTests\.cmake"\)'
|
|
replace_add_test_pattern = 'include("${CTEST_CURRENT_DIR}/CatchAddTests.cmake")'
|
|
modified_content = re.sub(add_test_pattern, replace_add_test_pattern, modified_content)
|
|
# 5 use exe from previous folder
|
|
exe_pattern = r"TEST_EXECUTABLE\s+\[==\[(.*?)\]==\]"
|
|
replace_exe_pattern = r'TEST_EXECUTABLE ${EXE_PATH}/\1'
|
|
modified_content = re.sub(exe_pattern, replace_exe_pattern, modified_content)
|
|
# 6 include _ctest.cmake file with path
|
|
ctest_test_pattern = r'include\("(.*?_tests\.cmake)"\)'
|
|
replace_ctest_pattern = r'include("${CTEST_CURRENT_DIR}/\1")'
|
|
modified_content = re.sub(ctest_test_pattern, replace_ctest_pattern, modified_content)
|
|
# 7 use script folder as cwd
|
|
cwd_pattern = r"TEST_WORKING_DIR\s+\[==\[(.*?)\]==\]"
|
|
replace_cwd_pattern = r'TEST_WORKING_DIR ${EXE_PATH}'
|
|
modified_content = re.sub(cwd_pattern, replace_cwd_pattern, modified_content)
|
|
# 8 modify ld_library_path
|
|
if os.name == 'posix':
|
|
lib_path_pattern = r"TEST_DL_PATHS\s+\[==\[(.*?)\]==\]"
|
|
replace_lib_path_pattern = r'TEST_DL_PATHS ${LIB_PATH}'
|
|
modified_content = re.sub(lib_path_pattern, replace_lib_path_pattern, modified_content)
|
|
|
|
filename = os.path.basename(filename)
|
|
install_path = os.path.join(install_script, filename)
|
|
# Write the modified content back to the file
|
|
with open(install_path, 'w') as file:
|
|
file.write(modified_content)
|
|
#print(f"**Done parsing now writing into", install_path)
|
|
except IOError as e:
|
|
print(f"Error: '{e}'")
|
|
sys.exit({e})
|
|
except Exception as e:
|
|
print(f"An error occurred: {e}")
|
|
sys.exit({e})
|
|
|
|
inccmake_files = glob.glob(build_folder + "/**/*"+inc_cmake_pattern, recursive=True)
|
|
make_test_files_portable(inccmake_files)
|