* Data imputation strategy for iteration multiplexing
* Implement data imputation methodology to handle missing counter values
in case of iteration multiplexing
* Enable dispatch filtering with iteration multiplexing since we are no
longer merging dispatches
* Bugfix to prevent check for missing counter values when using csv
format when profiling with iteration multiplexing
* Move warning and info message in case of iteration multiplexing to
sanitize function which comes earlier in analyze mode
* Address review comments
* Fix typo in documentation
* Move profiling config init. after path check in sanitize()
* Graceful handling of dispatches with all counters empty within data
imputation logic
* Improve info message for iteration multiplexing based analysis
* Ensure proper error message when trying to run iteration multiplexing with attach/detach
* fix test case
* Remove MFMA functionality in rocflop sample since its not supported in MI50
* Add gfx arc based support for MFMA and SMFMAC in rocflop.cpp
* Add --int32 usage doc
* Address review comments
* Update rocprofiler workflows to use new runner naming for mi325
* Add input options to workflow_dispatch for rocprofiler-systems CI workflow
* Update runner name on therock-ci-linux.yml as well
* Fix merging logic for multi process
* Fix dispatch id reset logic in case of rocpd format
* Fix kernel id reset logic in case of csv format
* Revert correlation logic change in csv format
* Do inner join instead of left join
* Added tool for dumping counter and metric values
* Skip Linting
* Added support for iteration multiplexing
* Remove subparser and supress compute options
* Specify output dir
* Add kernel info
* csv name change
* Added comments
* Support dispatch id-less dataframes
* Formatting fix
* Add default for path
* Print help with no args
* Support only single workload
* Faster counter accuracy testing
* Better handle SPI_CSN_* metrics for lesser than MI350 series
* Use metric filtering to collect only relevant counters for comparison
* Ensure all workload folders are deleted after testing is completed
* Dont use clean_existing=False
* Add manual test for all counter accuracy
* Test env. vars. in rocprofiler-sdk backend
* Improve rocprofiler-sdk backend test case to check for env. vars. and
ensure we do not overwrite irrelevant env. vars.
* Remove unnecessary usage of ROCPROF_INDIVIDUAL_XCC_MODE env. var.
* Formatting fixes
* Test fixes
* Remove redundant code in tests
* Remove usage of utils_mod and use utils instead, this prevents
duplicate imports
* Fix for multi process workload profiling
Native counter collection tool updates:
* Do not dump empty counter data for a process
* Use PID instead of UUID for dumped csv files to facilitate correlation
* Handle merging multiple pairs of rocpd (from sdk tool) and csv (from
native tool) files
* Handle merging multiple pairs of csv (from sdk tool) and csv (from
native tool) files
Rocpd output format updates:
* Merge multiple rocpd databases into a single csv
* Reset dispatch id and kernel id for unique dispatches and unique
kernels respectively
* Retain multiple rocpd databases per run for multi process workloads
* Add test case for multiprocess profiling using rocflop workload
* Add rocflop
* Fix native counter csv to rocprofv3 csv conversion
* Use kernel_id instead of dispatch_id to correlate native counter csv
and kernel trace csv
* python formatting using ruff 0.14 instead of 0.13
* Install rocm-dev in rocprofiler-compute-tarball.yml workflow
* Update paths for push and PR for rocprofiler-compute-tarball.yml
* Add ROCm dependencies to disttest job
* cmake fix binary link creation and fix format
* Use python3 instead of python3.9 in RHEL 8 and RHEL 9 workflows
* set default python3 to python3.9 in rhel8
* Try alternatives setup for python3 in RHEL8 env
* Add pip install cmake to debug RHEL8 issue
* Remove python3.11 in RHEL8 workflow
* Add back comment regarding RHEL8
---------
Co-authored-by: Vignesh Edithal <Vignesh.Edithal@amd.com>
* Improve Iteration multiplexing
* Improve iteration multiplexing documentation by adding usage note and
listing caveats
* Bugfixes for iteration mulitplexing
* Use merge iteration multiplexing in analysis webui and db mode
* Do not remove Dispatch_ID column in merge iteration multiplexing
since it is needed for analysis of top dispatches based on
duration
* Bugfixes for analysis logic
* Graceful handling of missing counters in case of iteration
multiplexing
* Improved warnings when metrics could not be calculated due to
missing counter data
* Fix the check to prevent showing table when a column is full of
N/A
* Improve detection of empty values when metric evaludation fails
due to missing counter data
* Bugfixes for profile logic
* Fix kernel filtering during roofline benchmark phase
* Update changelog for bugfixes
* Remove unnecessary columns when merging dispatches for iteration multiplexing
* bugfix
* Better analysis warnings
* fix to_std() in parser
* Use median in merge iteration multiplex
* Address review comments
* Fix cmake formatting
* fix None handling of parser util functions
* Enable stochastic counter accuracy test
* fix cmake formatting
* added graceful errors/exit in profile/analyze roofline.csv
* edit if statement truth
* restore if statement truth (roofline_csv needs at least 2 rows)
* addressed comments and skipped showing roof metrics when data invalid
* fix workload merge
* changed warning to error
* removed redundant variable definition
* added roofline csv validate check in TUI
* add test cases to test validation function
* ruff format
* simplified TUI roofline handling
* Improve amdsmi interface
* Fix issue where max mem clock was being set as max gfx clock
* Handle the case when all device handles might not be usable due to
devices being hidden by ROCR and HIP environment variables
* Fix get gpu vram size to return str in KB
* Improve testing of amdsmi interface functions
* Remove TCP_TCP_LATENCY_sum counter for MI300
* Remove TCP_TCP_LATENCY_sum counter which is unsupported for MI300 per register specification
* Remove VL1 Lat metric from memory chart section (block 3) for MI 300
since it uses TCP_TCP_LATENCY_sum counter which is unsupported
* Remove references to TCP_TCP_LATENCY_sum
* Update CHANGELOG
* reword changelog
* Initial work in progress for compute CI workflow
* Update run-ci.py script location, enable test creation
* Add new lines to files
* Add coverage file argument to run-ci.py
* Remove run-ci.py script usage from rocprofiler-compute-continuous-integration.yml workflow
* Add --break-system-packages parameter
* Add --ignore-installed to pip install
* Checkout specific branch until amdclang issue fixed in develop
* Add missing slash to path for cxx compiler
* Remove specific branch from checkout action
* Use run-ci.py in rocprofiler-compute-continuous-integration.yml
* Update install python requirements step
* Fix typo in build-name
* Update run-ci.py to have toggle for code coverage
* Apply ruff formatting
* Ruff again
* Exclude live attach detach and roofline tests in CI
* Add ctest args
* Revert run-ci.py changes
* Try new run-ci-2.py
* Update type of pytest-numprocs argument
* Try casting arg to str
* Fix typo in arg reference
* upgrade pip before running python installs
* Use jammy instead of noble for CI
* Remove python nproc arg from run-ci-2.py
* Switch to MI325 runners for CI
* Fix spacing issue
* Rename run-ci.py to run-code-coverage.py, add new run-ci.py
* Update to ROCm version 7.1.0 to debug sdk issues
* Testing out tarball install again
* Update regex on tarball version
* Update tarball regex on compute
* ruff formatting
* Revert change to systems CI file
* Switch back to rocm-dev install
* ruff formatting again
* Add ld_lib_path for rocm_sysdeps
* Remove excluded tests temporarily
* Add back excluded tests, add timeout for test step
* Address PR feedback
* Add git safe directory lines
* Revert dependencies change to debug new failures
* Exclude roofline again, rework dependencies
* Add in hip-runtime-amd dependency
* Install hip dev package
* Add TEST_FROM_INSTALL cmake arg to compute CI workflow
* Remove test_from_install for now
* Enable roofline tests again
* Add exception handling for native tool path search
* Fix formatting in roofline benchmark code
* Fix detection of .so files
* include hip code and native tool code in standalone binary
* add fallback path for ROCM_PATH
* Enable running tests from installation only
* Use cmake option -DTEST_FROM_INSTALL=ON to enable running tests from installation folder only
* It is not possible to run tests from build folder in this case
* This option prevents changing working directory to source folder
* Fix SourceFileLoader to import rocprof-compute main module correctly
* Install sample executables in the test folder
* fix num_xcds_cli_output test
* Fix tests
* Skip autogen. config. test and add a TODO task for re-design of this
test
* Add flexible import of source code in test_gpu_specs.py
* Update cmake to install tests/workloads folder when INSTALL_TESTS=ON
* Fix sys.argv[0] for tests
* fix live attach detach test
* Implement AMDGPU driver info and GPU VRAM attributes in system info.
section of analysis report.
* Backward compatibility for rocprofiler-sdk avail module path migration
* Fix roofline calculation where AI data points are N/A
* Removing FP8 from peak VALU datatypes - PEAK_OPS_DATATYPES.
* Similar change for BF16.
* Roofline binaries from rocm-amdgpu-bench generated 10/22.
https://github.com/ROCm/rocm-amdgpu-bench/commit/2113ef1f5eada8a4a6e44e6d07fd6abac9b0a3f8
Bins include change that removes FP8 and BF16 peak VALU benchmarks.
Built and tested on rhel8, azl3, ubuntu22.04, sles15sp6.
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Re-committing the bins
accidentally copied over bins from the wrong folder earlier, caught by Gowthami during testing.
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Updated changelog
* gersemi fix
* Changelog corrected.
* Changelog fix.
* Adding this to the 7.2.0 section to be picked up in an RC build.
* Moving changelog entry into unreleasesd section - team reconfirmed cutoff date after I requested this change so I am just quickly correcting my mistake in my ask.
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
---------
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
Co-authored-by: Carrie Fallows <Carrie.Fallows@amd.com>
Co-authored-by: JeniferC99 <150404595+JeniferC99@users.noreply.github.com>
* Split roofline tests
* Use N/A for missing values
* Test eval_expression for no valid data
* Fixed tests
* Updated Changelog for N/A
* Fixed platform specific test failure
* Use native tool for counter collection
* Add native counter collection tool which uses rocprofiler-sdk C++
library public API to get counter collection data
* This is enabled by default, unless --no-native-tool option is
provided or ROCPROF=rocprofv3 env. var. is provided
* This tool is only supported for ROCm version >=7.x.x
* This tool is not supported for attach/detach scenario
* Build native tool shared object during build time
* If using rocprof-compute without building then runtime compilation of
t push native tool shared object is performed
* rocprofiler-sdk tools is still used for services other than counter
collection and data collected by native tool is merged into the
rocpd/csv output of rocprofiler-sdk tool
* Make `rocpd` choice the default choice for `--format-rocprof-output`
option
* If `rocpd` public API from rocprofiler-sdk library is not present,
then fallback to `csv` choice
* In this case only `pmc_perf.csv` is written in workload folder
instead of multiple `csv` files for each profiling run
* Remove `json` choice from `--format-rocprof-output` option since it
functions identical to `csv` option
* Rename option `--rocprofiler-sdk-library-path` to
`--rocprofiler-sdk-tool-path` since we LD_PRELOAD the
rocprofiler-sdk tool shared object and not the rocprofiler-sdk library
shared object
* Fix the meaning of `--dispatch` option in `profile` mode to mention
dispatch iteration filtering instead of dispatch id filtering
* --dispatch option in analyze mode does dispatch id filtering
* Move standalone binary creation logic from cmake file to docker file
* fix native counter collection tool during attach/detach
* improve logging
* fix attach detach with native tool
* fix attach detach with native tool
* do not support attach/detach in native tool
* Update changelog
* add standalone binary creation functionality in cmake
* address review comments
* address review comments
* fix formatting
* address review comments
* Adding paths for cmake to search. Also updated min. cmake requirement to 3.21 as this was when hip was supported.
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Update hip compiler ID check, sometimes comes up as Clang, sometimes ROCMClang- depends on setup.
Updated formatting.
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* RHEL8.10 unable to compile due to defaulting to old c++ version, need to force c++17
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Updating changelog per docs team recommendations
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Apply suggestions from code review to changelog
Co-authored-by: Pratik Basyal <pratik.basyal@amd.com>
* Do not required HIP complier to build native counter collection tool
* fix cmake
* gersemi formatting on latest cmake change
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* ex ci updated dependencies to include rocprofiler-sdk, but cmake was still not capturing the path- there was a commit that added to the cmake_prefix_path entry that specified rocprof-sdk's cmake location ut was too specific for the search paths in find_package's config mode.
removing the cmake_prefix_path var and adding hints to find_package call instead, and specifying config mode so it knows how to construct the search paths
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* gersemi run for formatting
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Still need prefix path, should not have been removed in last commit but does need to be shortened to just the rocm path to allow for find_package config mode to do the job
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* include cstdint for uint32_t
* Run formatting on helper.cpp
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Remove rocm 7.2 release stuff from version and changelog and handle it in separate pr
* fix version
* fix changelog
* fix changelog
* run ruff formatter
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* fix rocprofiler-sdk attach so path
---------
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
Co-authored-by: Carrie Fallows <Carrie.Fallows@amd.com>
Co-authored-by: Pratik Basyal <pratik.basyal@amd.com>
Force tencentos to use rhel-based bin since tencent is branched off of centos, which is branch of fedora.
Verified rocprof-compute run correctly selects bin to use, and the roofline benchmark values look similar between runs on rhel vs tencentos4 docker images on same system.
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* attach: rename librocprofv3-attach
- Renames library to librocprofiler-sdk-rocattach
- ROCAttach library will be formalized and documented in future commit
* Address review comments
- Rename rocprofv3-attach.py to rocprof-attach.py
- Use common filesystem.hpp in rocattach
* Fix component name typo
* Doc fixup
---------
Co-authored-by: JeniferC99 <150404595+JeniferC99@users.noreply.github.com>
vedithal-amd
Jason Bonnell
jamessiddeley-amd
abchoudh-amd
xuchen-amd
cfallows-amd
ywang103-amd
Ben Richard
Pratik Basyal
Kunal Malviya
jonatluu
ggottipa-amd
Mark Meserve