* Split roofline tests
* Use N/A for missing values
* Test eval_expression for no valid data
* Fixed tests
* Updated Changelog for N/A
* Fixed platform specific test failure
* Use native tool for counter collection
* Add native counter collection tool which uses rocprofiler-sdk C++
library public API to get counter collection data
* This is enabled by default, unless --no-native-tool option is
provided or ROCPROF=rocprofv3 env. var. is provided
* This tool is only supported for ROCm version >=7.x.x
* This tool is not supported for attach/detach scenario
* Build native tool shared object during build time
* If using rocprof-compute without building then runtime compilation of
t push native tool shared object is performed
* rocprofiler-sdk tools is still used for services other than counter
collection and data collected by native tool is merged into the
rocpd/csv output of rocprofiler-sdk tool
* Make `rocpd` choice the default choice for `--format-rocprof-output`
option
* If `rocpd` public API from rocprofiler-sdk library is not present,
then fallback to `csv` choice
* In this case only `pmc_perf.csv` is written in workload folder
instead of multiple `csv` files for each profiling run
* Remove `json` choice from `--format-rocprof-output` option since it
functions identical to `csv` option
* Rename option `--rocprofiler-sdk-library-path` to
`--rocprofiler-sdk-tool-path` since we LD_PRELOAD the
rocprofiler-sdk tool shared object and not the rocprofiler-sdk library
shared object
* Fix the meaning of `--dispatch` option in `profile` mode to mention
dispatch iteration filtering instead of dispatch id filtering
* --dispatch option in analyze mode does dispatch id filtering
* Move standalone binary creation logic from cmake file to docker file
* fix native counter collection tool during attach/detach
* improve logging
* fix attach detach with native tool
* fix attach detach with native tool
* do not support attach/detach in native tool
* Update changelog
* add standalone binary creation functionality in cmake
* address review comments
* address review comments
* fix formatting
* address review comments
* Adding paths for cmake to search. Also updated min. cmake requirement to 3.21 as this was when hip was supported.
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Update hip compiler ID check, sometimes comes up as Clang, sometimes ROCMClang- depends on setup.
Updated formatting.
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* RHEL8.10 unable to compile due to defaulting to old c++ version, need to force c++17
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Updating changelog per docs team recommendations
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Apply suggestions from code review to changelog
Co-authored-by: Pratik Basyal <pratik.basyal@amd.com>
* Do not required HIP complier to build native counter collection tool
* fix cmake
* gersemi formatting on latest cmake change
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* ex ci updated dependencies to include rocprofiler-sdk, but cmake was still not capturing the path- there was a commit that added to the cmake_prefix_path entry that specified rocprof-sdk's cmake location ut was too specific for the search paths in find_package's config mode.
removing the cmake_prefix_path var and adding hints to find_package call instead, and specifying config mode so it knows how to construct the search paths
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* gersemi run for formatting
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Still need prefix path, should not have been removed in last commit but does need to be shortened to just the rocm path to allow for find_package config mode to do the job
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* include cstdint for uint32_t
* Run formatting on helper.cpp
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Remove rocm 7.2 release stuff from version and changelog and handle it in separate pr
* fix version
* fix changelog
* fix changelog
* run ruff formatter
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* fix rocprofiler-sdk attach so path
---------
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
Co-authored-by: Carrie Fallows <Carrie.Fallows@amd.com>
Co-authored-by: Pratik Basyal <pratik.basyal@amd.com>
* Upgrade min python version from 3.8 to 3.9
* Set min version for textual-fspicker for TUI support
* Update workflows to use python 3.9 instead of 3.8
* fix formatting
* fix bug
---------
Co-authored-by: Vignesh Edithal <Vignesh.Edithal@amd.com>
* Replace cmake-format with gersemi in rocprofiler-compute-formatting.yml
* Run gersemi formatting on CMakeLists.txt files
* Remove .cmake-format.yaml, add .gersemirc file
* Add more options to .gersemirc
* Add new line to .gersemirc
* Add new line to CMakeLists.txt
* Run gersemi again with new options
ROCProfiler-Register/Systems/Compute: The license file name in the CMake install module and other locations was originally LICENSE, but it was recently changed to LICENSE.md, requiring an update to the CMake install module and all other relevant locations.
* Remove .git folder and git command check in cmake
* Update docker container to work in monorepo
* Update docker container to mount the top level folder in monorepo
* Show description of metrics during analysis
* Use --include-cols Description show the Description column in analyze mode (this is hidden by default)
* Remove tips field from analysis config
* Align metric names in analysis config and documentation
* Add unified config utils/unified_config.yaml
* Add python script utils/split_config.py to auto generate analysis configuration and documentation metrics description
* Add test case to ensure unified config is older than auto-generated config
* Auto generate analysis config and documentation metrics description
* Update CONTRIBUTING.md to add instructions to build documentation assets
* Add docker image and compose file to build documentation
* Update CHANGELOG and Documentation
* Use jinja template instead of hardcoding metric tables in documentation
[ROCm/rocprofiler-compute commit: bb44e90b2d]
* Analysis report block based filtering is the default now
* Update documentation
* Update CHANGELOG
* Fix tests
* Replace hardware block based filtering tests with report block
based filtering tests
[ROCm/rocprofiler-compute commit: 98bb0f4237]
* Fix roofline rocm version bug
* Fix utils bug
* Remove unnecessary tests
* Do not check textual-fspicker package in cmake build
* Use rocprofv3 to test MI 100 and fix tests
[ROCm/rocprofiler-compute commit: 000fd4f5b2]
* Add test for 9fx942 number of xcds.
* Improve the structure of mi gpu specs, add num_xcds_spec_class test.
* Add to ctest.
---------
Signed-off-by: xuchen-amd <xuchen@amd.com>
[ROCm/rocprofiler-compute commit: 85bfa73e2c]
* Add MI 350 hardware information
* Refactor MI GPU YAML file and corresponding interface
* Add SoC file for gfx950 architecture
* Add analysis report configs for MI 350 containing existing metrics
* Add placeholder None valued metrics for previous architectures to make
baseline comparison work
* Enable testing on MI 350
* Analysis config metric changes
- SPI changes
- Update metric formula for default SPI pipe counter
- Use efficiently collected pipe wise SPI counters
- Add SPI Wave Occupancy
- Add Scheduler-Pipe Wave Utilization
- Update formula for VGPR Writes
- Add Scheduler-Pipe FIFO Full Rate
- CPC changes
- Add CPC SYNC FIFO Full Rate
- Add CPC CANE Stall Rate
- Add CPC ADC Utilization
- SQ changes
- Add VALU co-issue efficiency
- Add F6F4 datatype metrics
- Update formula for total FLOPs by adding F6F4 counters
- Add LDS STORE / LOAD / ATOMIC metrics
- Add LDS STORE / LOAD / ATOMIC bandwidth
- Add LDS FIFO and TA ADDR / CMD / DATA FIFO full rates
* Collect TCP_TCP_LATENCY_sum only for gfx950 (MI 350)
* Do not inject SQ_ACCUM_PREV_HIRES unnecesarily
* Do not hardcode memory and shader clock speeds
* Write num_hbm_channels to sysinfo.csv instead of hbm_bw while profiling
* Move generate sysinfo.csv to pre processing step of profiling
* Add warnings to use --specs-correction for missing sysinfo.csv values during analysis phase
* Update CHANGELOG
* Analysis phase warning to use --specs-correction when needed
[ROCm/rocprofiler-compute commit: f9aa7be97c]
* Analysis report block based filtering for profiling
* Profiling mode changes
- `-b` option now additionally accepts metric id(s), similar to `-b` option in analyze mode (e.g. 6, 6.2, 6.23)
- Only counters mentioned in the selected analysis report blocks will be collected
- Add parsing logic to identify hardware counters from analysis report blocks
- Add filtering logic to only write filtered counters in perfmon files
- Log not collected counters in one line
- `--list-metrics` option added in profile mode to list possible metric id(s) similar to analyze mode
- Write arguments provided during profiling in profiling_configuration.yaml file
* Analysis mode changes
- During analysis mode, only show report blocks selected during profiling
- If `-b` option is provided in analysis mode, then follow provided filters
- Do not show empty tables in analysis report
* Miscellaneous changes
- Update CHANGELOG
- Add test cases
- Instruction mix report block filter
- Instruction mix and Memory chart report block filter
- Instruction mix report block filter and CPC hardware block filter
- TA hardware block filter
- --list-metrics in profile mode should work
- Move binary handler fixtures to conftest.py to avoid importing
fixtures
- cmake file in tests directory has been updated to compile sample/vmem.hip for testing
* Public documentation changes
- Use the term "Hardware report block" instead of "Hardware block"
- Add documentation for "--list-metrics" option in profile mode
- Add example of filtering by hardware report block such as instruction
mix and wavefront launch statistics
- Add deprecation warning for hardware component (sq, tcc) based filtering
[ROCm/rocprofiler-compute commit: 55cf0e237e]
* Fix post analysis gui in standalone binary
* Add post analysis gui assets and required server libraries for GUI
server and web page
* Add port forwarding to docker test compose
* Update README me to use `docker compose up` instead of `docker compose run`
to run containers with port forwarding and to leverage other
functionalities of docker compose
[ROCm/rocprofiler-compute commit: 0b3114fa88]
* Create VERSION.sha file while creating standalone binary
* Do not fail on VERSION.sha not found
* Add git to docker container for testing
[ROCm/rocprofiler-compute commit: 19a9365bb9]
* Add cmake function to create standalone binary
* Mention licenses used by dependencies in the LICENSE file
* Add test cases for standalone binary by adding --call-binary option for pytest
* Docker compose file to create standalone binary in standardized RHEL 8 environment
* Add README instructions on how to create and test standalone binary
* Move docker files from utils to docker folder; Add standalone binary testing instructions
* Add CHANGELOG statement
* Use different service names in docker compose files
* Use volume mounting in docker files
[ROCm/rocprofiler-compute commit: 40f79c28b1]
* External CI: rename pipeline to rocprofiler-compute (#463)
Signed-off-by: Daniel Su <danielsu@amd.com>
* Update webui branding (#459)
* Update name and icon for browser tab to rocprofiler-compute.
Signed-off-by: xuchen-amd <xuchen@amd.com>
* Update name and icon for browser tab to rocprofiler-compute.
Signed-off-by: xuchen-amd <xuchen@amd.com>
---------
Signed-off-by: xuchen-amd <xuchen@amd.com>
* Update branding in documentation (#442)
* find/replace Omniperf to ROCm Compute Profiler
Signed-off-by: Peter Park <peter.park@amd.com>
* update name in Sphinx conf
Signed-off-by: Peter Park <peter.park@amd.com>
* mv what-is-omniperf.rst -> what-is-rocprof-compute.rst
Signed-off-by: Peter Park <peter.park@amd.com>
* update Tutorials section
Signed-off-by: Peter Park <peter.park@amd.com>
* add Omniperf as keyword to Conceptual section for internal search
Signed-off-by: Peter Park <peter.park@amd.com>
* update Reference section
Signed-off-by: Peter Park <peter.park@amd.com>
* black fmt conf.py
Signed-off-by: Peter Park <peter.park@amd.com>
* update profile mode and basic usage subsections
Signed-off-by: Peter Park <peter.park@amd.com>
* update how to use analyze mode subsection
Signed-off-by: Peter Park <peter.park@amd.com>
* update install section
Signed-off-by: Peter Park <peter.park@amd.com>
* fix sphinx warnings
Signed-off-by: Peter Park <peter.park@amd.com>
* fix cmd line examples in profile/mode.rst
Signed-off-by: Peter Park <peter.park@amd.com>
* update install decision tree image
Signed-off-by: Peter Park <peter.park@amd.com>
* fix TOC and index
Signed-off-by: Peter Park <peter.park@amd.com>
fix weird wording
* fix cli text: deriving rocprofiler-compute metrics...
Signed-off-by: Peter Park <peter.park@amd.com>
* update standalone-gui.rst
Signed-off-by: Peter Park <peter.park@amd.com>
* restore removed doc updates from #428
Signed-off-by: Peter Park <peter.park@amd.com>
* update ref to Omniperf in index.rst
Signed-off-by: Peter Park <peter.park@amd.com>
* fix grafana connection name to match image
Signed-off-by: Peter Park <peter.park@amd.com>
* update cmds in tutorials
Signed-off-by: Peter Park <peter.park@amd.com>
---------
Signed-off-by: Peter Park <peter.park@amd.com>
* MI300 roofline enablement in rocprofiler-compute (#470)
* MI300 roofline enablement in rocprofiler-compute
requirements.txt
- running some modules complained about numpy version too new, adding extra requirement that numpy be 1.x
pmc_roof_perf.txt
- adding TCC_BUBBLE_sum counter to profile
soc_gfx940.py
soc_gfx941.py
soc_gfx942.py
- remove console logs reading that roofline is temporarily disabled, uncommenting blocks that check for roofline csv and run roofline post-processing
roofline_calc.py
- add mi300 to supported soc
- add new calculation for hbm_data for MI300 using tcc_bubble_sum, checks if counter > 0 to use
- add to a few comments
roofline-ubuntu-20_04-mi300-rocm6
- binary for the ubuntu systems to enable mi300 roofline calculations from rocm-amdgpu-bench
Note- other distros will get roofline bins to enable mi300, but need to be further tested before putting into branch.
Signed-off-by: Carrie Fallows <carrie.fallows@amd.com>
* Reformatting roofline_calc.py
Signed-off-by: Carrie Fallows <carrie.fallows@amd.com>
---------
Signed-off-by: Carrie Fallows <carrie.fallows@amd.com>
* Update Python format checker (#471)
* Add pre commit hook for Python formatting
Signed-off-by: coleramos425 <colramos@amd.com>
* Update formatting workflow to run on latest Python and add isort formatter
Signed-off-by: coleramos425 <colramos@amd.com>
* Fix caught yaml formatting issues
* Update pyproject file
* Add pre-commit hook instruction to CONTRIBUTING guide
* Remove target-version from black pyproject.toml
* Fixed formatting errors found with black and isort
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
* Run hook: Whitespaces, fix end of file spaces
---------
Signed-off-by: coleramos425 <colramos@amd.com>
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
Co-authored-by: David Galiffi <David.Galiffi@amd.com>
* Bump cryptography from 43.0.0 to 43.0.1 in /docs/sphinx (#473)
Bumps [cryptography](https://github.com/pyca/cryptography) from 43.0.0 to 43.0.1.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/43.0.0...43.0.1)
---
updated-dependencies:
- dependency-name: cryptography
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Fix file permission on MI300 roofline binary (#477)
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
* Removing numpy requirements of <2 (#478)
Checks are failing if version too high and no need for lower version
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Fix crash when loading web UI roofline for gfx942 (#479)
* Fix crash when loading web UI roofline for gfx942
* Fix formatting
Signed-off-by: benrichard-amd <ben.richard@amd.com>
* Make same changs for gfx940, gfx942.
Signed-off-by: benrichard-amd <ben.richard@amd.com>
* Fix formatting in soc_gfx940 and soc_gfx941.
Signed-off-by: benrichard-amd <ben.richard@amd.com>
---------
Signed-off-by: benrichard-amd <ben.richard@amd.com>
* Rebranding name change patch (#469)
* Patch in missed name change for rebranding.
Signed-off-by: xuchen-amd <xuchen@amd.com>
* Patch in missed name change for rebranding.
Signed-off-by: xuchen-amd <xuchen@amd.com>
---------
Signed-off-by: xuchen-amd <xuchen@amd.com>
* Move dependabot.yml to .github/ and bump rocm-docs-core (#481)
* Move dependabot.yml to .github/
* Bump rocm-docs-core to 1.8.5
* Bump rocm-docs-core to 1.9.0
* Fix packaging for upgrading (#486)
Specify that "rocprofiler-compute" replaces / obsoletes the "omniperf" package.
* Renamed extension path from omniperf to rocprofiler_compute (#487)
Signed-off-by: Tim Gu <Tim.Gu@amd.com>
* MI300 rhel and sles roofline binaries (#480)
* Roofline bins for MI300 on rhel and sles distributions
Built from rocm-amdgpu-bench, tested on respective distro systems with MI300 hardware.
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Minor modifications removing hardcoded variables in roofline files.
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
---------
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Modify test_profile_general.py ctest to include MI300 enablement (#498)
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* part 1 to support rocprofv3 (#492)
* rocprofv3 support initial commit
-Can run rocprofv3 but ultimately fails. rocprofv3 says the counter capacity
is exceeded and the output CSV file format is different from v1/v2.
* Add rocprofv3 detection so v2 can still be used
It's hacky but it'll do for now.
* Add code path to convert rocprofv3 JSON output into CSV
* Grab correct value for Queue ID
* Use _sum suffix to sum TCC counters
Previously we were specifying each channel for TCC counters. rocprofv3 does
not support specifing each TCC channel, and instead will auto sum given
the TCC counter name. The counter name with the _sum suffix is also
supported and is also supported in v1 and v2. So we will use the TCC
counter name with the _sum suffix.
* Fix incorrect counter outputs when using rocprofv3
In the JSON output some counters appear multime times and must be
summed to get the correct value. These summed values match the
rocprofv3 output in CSV mode and also match the rocprofv2
output.
* Remove duplicate Correlation_ID and Wave_Size in output
* Handle json output that does not contain any dispatches
Omniperf was assuming each JSON output from rocprofv3 would always contain
dispatches. This is not the case. For example, in a multi-process
workload where one of the processes does not dispatch any kernels. A JSON
file will still be output for this process but it will not contain any dispatches.
* Code cleanup
* Update search path for rocprofv3 results
Rocprofv3 was updated to include the hostname in the path where
it outputs results.
* Handle accumulate counters
In v1/v2 rocprof uses the SQ_ACCUM_PREV_HIRES counter for the accumualte
counters. v3 does not have this. So we need to define our own counters
in counter_defs.yaml. For this we use the counter name + _ACCUM, for
example SQ_INSTR_LEVEL_SMEM_ACCUM.
To use rocprofv3 you will need to update counter_defs.yaml to include
these new counter definitions.
* Use correct GPU ID
When converting JSON -> CSV we were assigning node_id to GPU_ID. Since
the JSON contains non-GPU devices, the node_id for GPUs might not
start at 0 as expected.
This commit maps the agent ID to the appropriate GPU ID.
* Parse scratch memory per work item from JSON
* Support rocprofv3 CSV parsing
JSON decoding is very slow for large files. Include support for parsing
rocprofv3 CSV output and make that the default.
CSV/JSON can be toggled via the ROCPROF_OUTPUT_FORMAT environment
variable e.g. ROCPROF_OUTPUT_FORMAT=csv or ROCPROF_OUTPUT_FORMAT=json
* black format after merge
* format isort
* change return of rocprof_cmd to try to resolve test's error
* hack to pick last part of rocminfo's name
* debug log of hacks
* Modify test_profile_general.py ctest to include MI300 enablement. Currently failing because of explicitly excluded roofline files for the soc and autofailed asserts for roof-only tests- originally in place because roofline was not enabled on mi300 yet.
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* black and isort formated
* corrected line of copyright
---------
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
Co-authored-by: benrichard-amd <ben.richard@amd.com>
Co-authored-by: YANG WANG <ywang@ywang-ubuntu.amd.com>
Co-authored-by: Carrie Fallows <Carrie.Fallows@amd.com>
* fix for crash of timestamp of part 1 for rocprofv3 (#499)
* fix the error caused by ignoring the lack of counter csv file from rocprofv3 for timestamp
* isort and black formated
* quick fix for gfx906 roofline (#505)
* Multi node support (#503)
* [CTest] Pipeline failures for MI300 (#483)
* Propagate new chip_id logic to testing workflow
Signed-off-by: coleramos425 <colramos@amd.com>
* Add a debug line to tests
Signed-off-by: coleramos425 <colramos@amd.com>
* Trying to set rocprofv2 generally in CTest module
Signed-off-by: coleramos425 <colramos@amd.com>
* Remove temp debugging lines from CI
Signed-off-by: coleramos425 <colramos@amd.com>
* Add roofline entry for MI300 expected files in CI tests
Signed-off-by: coleramos425 <colramos@amd.com>
* Make num_devices modifier global in scope
Signed-off-by: coleramos425 <colramos@amd.com>
* Change kernel name in PyTest to confirm rocprofv2 bug
Related to https://ontrack-internal.amd.com/browse/SWDEV-503453
Signed-off-by: coleramos425 <colramos@amd.com>
---------
Signed-off-by: coleramos425 <colramos@amd.com>
* Spatial-multiplexing: part 1 profiling stage (#465)
* rocprofv3 support initial commit
-Can run rocprofv3 but ultimately fails. rocprofv3 says the counter capacity
is exceeded and the output CSV file format is different from v1/v2.
* Add rocprofv3 detection so v2 can still be used
It's hacky but it'll do for now.
* Add code path to convert rocprofv3 JSON output into CSV
* Grab correct value for Queue ID
* Use _sum suffix to sum TCC counters
Previously we were specifying each channel for TCC counters. rocprofv3 does
not support specifing each TCC channel, and instead will auto sum given
the TCC counter name. The counter name with the _sum suffix is also
supported and is also supported in v1 and v2. So we will use the TCC
counter name with the _sum suffix.
* Fix incorrect counter outputs when using rocprofv3
In the JSON output some counters appear multime times and must be
summed to get the correct value. These summed values match the
rocprofv3 output in CSV mode and also match the rocprofv2
output.
* Remove duplicate Correlation_ID and Wave_Size in output
* Handle json output that does not contain any dispatches
Omniperf was assuming each JSON output from rocprofv3 would always contain
dispatches. This is not the case. For example, in a multi-process
workload where one of the processes does not dispatch any kernels. A JSON
file will still be output for this process but it will not contain any dispatches.
* Code cleanup
* Update search path for rocprofv3 results
Rocprofv3 was updated to include the hostname in the path where
it outputs results.
* Handle accumulate counters
In v1/v2 rocprof uses the SQ_ACCUM_PREV_HIRES counter for the accumualte
counters. v3 does not have this. So we need to define our own counters
in counter_defs.yaml. For this we use the counter name + _ACCUM, for
example SQ_INSTR_LEVEL_SMEM_ACCUM.
To use rocprofv3 you will need to update counter_defs.yaml to include
these new counter definitions.
* debug code
* add logic code for multiplexing
* minor fix
* more fixes
* rocprofv3 support initial commit
-Can run rocprofv3 but ultimately fails. rocprofv3 says the counter capacity
is exceeded and the output CSV file format is different from v1/v2.
* Add rocprofv3 detection so v2 can still be used
It's hacky but it'll do for now.
* Add code path to convert rocprofv3 JSON output into CSV
* Grab correct value for Queue ID
* Use _sum suffix to sum TCC counters
Previously we were specifying each channel for TCC counters. rocprofv3 does
not support specifing each TCC channel, and instead will auto sum given
the TCC counter name. The counter name with the _sum suffix is also
supported and is also supported in v1 and v2. So we will use the TCC
counter name with the _sum suffix.
* Fix incorrect counter outputs when using rocprofv3
In the JSON output some counters appear multime times and must be
summed to get the correct value. These summed values match the
rocprofv3 output in CSV mode and also match the rocprofv2
output.
* Remove duplicate Correlation_ID and Wave_Size in output
* Handle json output that does not contain any dispatches
Omniperf was assuming each JSON output from rocprofv3 would always contain
dispatches. This is not the case. For example, in a multi-process
workload where one of the processes does not dispatch any kernels. A JSON
file will still be output for this process but it will not contain any dispatches.
* Code cleanup
* Update search path for rocprofv3 results
Rocprofv3 was updated to include the hostname in the path where
it outputs results.
* Handle accumulate counters
In v1/v2 rocprof uses the SQ_ACCUM_PREV_HIRES counter for the accumualte
counters. v3 does not have this. So we need to define our own counters
in counter_defs.yaml. For this we use the counter name + _ACCUM, for
example SQ_INSTR_LEVEL_SMEM_ACCUM.
To use rocprofv3 you will need to update counter_defs.yaml to include
these new counter definitions.
* count accu files as well
* Use correct GPU ID
When converting JSON -> CSV we were assigning node_id to GPU_ID. Since
the JSON contains non-GPU devices, the node_id for GPUs might not
start at 0 as expected.
This commit maps the agent ID to the appropriate GPU ID.
* fix error with csv file parse from json and merge during post-processing
* implemented parsing of csv files from v3 output for optimization
* Parse scratch memory per work item from JSON
* Support rocprofv3 CSV parsing
JSON decoding is very slow for large files. Include support for parsing
rocprofv3 CSV output and make that the default.
CSV/JSON can be toggled via the ROCPROF_OUTPUT_FORMAT environment
variable e.g. ROCPROF_OUTPUT_FORMAT=csv or ROCPROF_OUTPUT_FORMAT=json
* black format after merge
* format isort
* change return of rocprof_cmd to try to resolve test's error
* hack to pick last part of rocminfo's name
* debug log of hacks
* Modify test_profile_general.py ctest to include MI300 enablement. Currently failing because of explicitly excluded roofline files for the soc and autofailed asserts for roof-only tests- originally in place because roofline was not enabled on mi300 yet.
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* black and isort formated
* formated by isort and black
* change default rocprof's output to csv
* repaired crash caused by missing csv counter file when running for timestamp
* change name to spatial-multiplexing from multiplexing
* make necessary modification for review
* set the value of spatial_multiplexing in argument defautly to None
* repair the part that blocks regular pmc files' generation
---------
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
Co-authored-by: benrichard-amd <ben.richard@amd.com>
Co-authored-by: fei.zheng <fei.zheng@amd.com>
Co-authored-by: YANG WANG <ywang@ywang-ubuntu.amd.com>
Co-authored-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Simple fix for gpu model value. (#508)
Signed-off-by: xuchen-amd <xuchen@amd.com>
* Add FP64 to plot adhering to pdf name (#507)
* Replacing FP32-only plot with an FP32&FP64 combo plot. Results will likely be negligible but the plot name indicates both should be graphed.
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Remove duplicate AI plot to clean up fp32 fp64 graph
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
---------
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Add gpu series for roofline (#510)
* Add gpu_series for roofline.
* Use gpu_series in path names for roofline.
* Fix TCC on MI200 when introduce rocprofv3 (#509)
* quick fix for v2
* one more fix
* revert a bit
---------
Co-authored-by: ywang103-amd <ywang103@amd.com>
* Bump rocm-docs-core from 1.9.0 to 1.12.0 in /docs/sphinx (#511)
Bumps [rocm-docs-core](https://github.com/ROCm/rocm-docs-core) from 1.9.0 to 1.12.0.
- [Release notes](https://github.com/ROCm/rocm-docs-core/releases)
- [Changelog](https://github.com/ROCm/rocm-docs-core/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/ROCm/rocm-docs-core/compare/v1.9.0...v1.12.0)
---
updated-dependencies:
- dependency-name: rocm-docs-core
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Update sample roofline plot img (#516)
* Modify path to use gpu_model instead of gpu_series to match other workload directory path creation/search points. Affects manual testing, does not seem to affect ctests. (#513)
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
* Improve formatting when displaying rocprof command. (#476)
* Improve formatting when displaying rocprof command.
Signed-off-by: xuchen-amd <xuchen@amd.com>
* Fix python formatting.
Signed-off-by: xuchen-amd <xuchen@amd.com>
* Strip unwanted characters (rocprofv1 specific) from rocprof commands.
Signed-off-by: xuchen-amd <xuchen@amd.com>
* Strip unwanted characters (rocprofv1 specific) from rocprof commands.
Signed-off-by: xuchen-amd <xuchen@amd.com>
* Save the unmodified arguments for rocprof for debug message display.
Signed-off-by: xuchen-amd <xuchen@amd.com>
---------
Signed-off-by: xuchen-amd <xuchen@amd.com>
* quick fix for mpi_support (#518)
* Pass accumulate counters to rocprofv3 using -E option (#522)
rocprofv3 has a new -E option where extra counters can be passed (see accum_counters.yaml) instead
of defining them in counter_defs.yaml.
* Unify all file handling with pathlib (#512)
* Replace occurences of os.path functions with equivalent functions from
pathlib library
* Remove unwanted imports of os.path and os
* Add coding guidelines for using pathlib instead of os.path
* Auto sync staging and mainline on a weekly cadence (#517)
Signed-off-by: coleramos425 <colramos@amd.com>
---------
Signed-off-by: Daniel Su <danielsu@amd.com>
Signed-off-by: xuchen-amd <xuchen@amd.com>
Signed-off-by: Peter Park <peter.park@amd.com>
Signed-off-by: Carrie Fallows <carrie.fallows@amd.com>
Signed-off-by: coleramos425 <colramos@amd.com>
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com>
Signed-off-by: benrichard-amd <ben.richard@amd.com>
Signed-off-by: Tim Gu <Tim.Gu@amd.com>
Co-authored-by: Daniel Su <danielsu@amd.com>
Co-authored-by: xuchen-amd <xuchen@amd.com>
Co-authored-by: Peter Park <peter.park@amd.com>
Co-authored-by: cfallows-amd <Carrie.Fallows@amd.com>
Co-authored-by: David Galiffi <David.Galiffi@amd.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ben Richard <143630488+benrichard-amd@users.noreply.github.com>
Co-authored-by: Tim Gu <Tim.Gu@amd.com>
Co-authored-by: ywang103-amd <ywang103@amd.com>
Co-authored-by: benrichard-amd <ben.richard@amd.com>
Co-authored-by: YANG WANG <ywang@ywang-ubuntu.amd.com>
Co-authored-by: Fei Zheng <44449748+feizheng10@users.noreply.github.com>
Co-authored-by: fei.zheng <fei.zheng@amd.com>
Co-authored-by: vedithal-amd <Vignesh.Edithal@amd.com>
[ROCm/rocprofiler-compute commit: 272e5b6e32]
* Install LICENSE to ${CMAKE_INSTALL_DOCDIR}
* Update tarball.yml to test for LICENSE file after the install
---------
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
[ROCm/rocprofiler-compute commit: 50755de867]
- Update filenames.
- Update executable to `rocprof-compute`
- Update update package to `rocprofiler-compute`
- Update name in application output and logs
- Update name in README files
- Update testing and workflows
---------
Signed-off-by: Xuan Chen <xuchen@amd.com>
[ROCm/rocprofiler-compute commit: 31b4de1a38]
* Check that the minimum required Python (3.8) version is used.
Prints a descriptive error message, rather than a cryptic import
failure, if minimum Python version is not met.
Internal ticket SWDEV-477233.
* Disable the RPM mangling of shebangs.
The are changing the `#!/usr/bin/python3` to `#!/usr/libexec/platform-python`.
With this set, omniperf is always using the platform installed version
of python, which is python 3.6 on RHEL 8. Using virtual environments,
like conda, did not work.
* Fix pylint issues
---------
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
[ROCm/rocprofiler-compute commit: 75a4b51d0d]
* pip-compile docs/requirements.txt
Signed-off-by: Peter Jun Park <peter.park@amd.com>
Add Sphinx docs config
Signed-off-by: Peter Jun Park <peter.park@amd.com>
Add Sphinx config
Signed-off-by: Peter Jun Park <peter.park@amd.com>
Update docs build config
Signed-off-by: Peter Jun Park <peter.park@amd.com>
* style(conf.py): Apply black formatting to docs/conf.py
Signed-off-by: Sam Wu <22262939+samjwu@users.noreply.github.com>
* Update docs requirements
Signed-off-by: Peter Jun Park <peter.park@amd.com>
Update to rocm-docs-core 1.3.0
Signed-off-by: Peter Jun Park <peter.park@amd.com>
Update docs requirements
Signed-off-by: Peter Jun Park <peter.park@amd.com>
pip-compile requirements
Signed-off-by: Peter Jun Park <peter.park@amd.com>
bump rocm-docs-core to 1.5.0
bump rocm-docs-core to 1.4.1
Signed-off-by: Peter Jun Park <peter.park@amd.com>
* Add dependabot.yml and update CODEOWNERS
Signed-off-by: Peter Jun Park <peter.park@amd.com>
Update toc and conf
Signed-off-by: Peter Jun Park <peter.park@amd.com>
update dependabot
* Port docs to rocm-docs standard
Signed-off-by: Peter Jun Park <peter.park@amd.com>
Add toc and Diataxis cards
Signed-off-by: Peter Jun Park <peter.park@amd.com>
Add basic file structure
Signed-off-by: Peter Jun Park <peter.park@amd.com>
add glossary
Signed-off-by: Peter Jun Park <peter.park@amd.com>
add includes
Signed-off-by: Peter Jun Park <peter.park@amd.com>
Add license.rst
Signed-off-by: Peter Jun Park <peter.park@amd.com>
add compatible hw
Signed-off-by: Peter Jun Park <peter.park@amd.com>
fix spelling and license
Signed-off-by: Peter Jun Park <peter.park@amd.com>
clean up index
Signed-off-by: Peter Jun Park <peter.park@amd.com>
clean up installation guides
Signed-off-by: Peter Jun Park <peter.park@amd.com>
add basic usage (quickstart)
Signed-off-by: Peter Jun Park <peter.park@amd.com>
add ref to global options
update toc
Signed-off-by: Peter Jun Park <peter.park@amd.com>
modularize modes and global options
Signed-off-by: Peter Jun Park <peter.park@amd.com>
add profile mode
Signed-off-by: Peter Jun Park <peter.park@amd.com>
fixes
Signed-off-by: Peter Jun Park <peter.park@amd.com>
reorg and clean up
Signed-off-by: Peter Jun Park <peter.park@amd.com>
add dynamic omniperf version number in installation guide
Signed-off-by: Peter Jun Park <peter.park@amd.com>
add datatemplate
more reorg
Signed-off-by: Peter Jun Park <peter.park@amd.com>
clean up
Signed-off-by: Peter Jun Park <peter.park@amd.com>
reorg images
move profile mode
reorg
reorg
reorg more
fix formatting
fix headings
ref anchor mi2xx note
add extlinks
add extlinks
Signed-off-by: Peter Jun Park <peter.park@amd.com>
black format
fix formatting, anchors
Signed-off-by: Peter Jun Park <peter.park@amd.com>
reorg
fix words and formatting
Signed-off-by: Peter Jun Park <peter.park@amd.com>
formatting
Signed-off-by: Peter Jun Park <peter.park@amd.com>
same
reorg
format
fix formatting
fix toc
Signed-off-by: Peter Jun Park <peter.park@amd.com>
format
* impr internal linking and fix sphinx warnings
Signed-off-by: Peter Jun Park <peter.park@amd.com>
* add spellcheck/linting from rocm-docs-core
Signed-off-by: Peter Jun Park <peter.park@amd.com>
fix rst directives
satisfy spellcheck
fix more spelling
rm unused files
fix spelling and update wordlist
* bump rocm-docs-core to 1.6.0
Signed-off-by: Peter Jun Park <peter.park@amd.com>
* add fixes from @skyreflectedinmirrors and @lpaoletti
Signed-off-by: Peter Jun Park <peter.park@amd.com>
add references to toc
Signed-off-by: Peter Jun Park <peter.park@amd.com>
add more fixes
Signed-off-by: Peter Jun Park <peter.park@amd.com>
* add package manager install section
Signed-off-by: Peter Jun Park <peter.park@amd.com>
* add fixes
Signed-off-by: Peter Jun Park <peter.park@amd.com>
add metadata and fixes
Signed-off-by: Peter Jun Park <peter.park@amd.com>
add fixes
bump to 1.6.1
more fixes
fix fmt in profiling examples
Signed-off-by: Peter Jun Park <peter.park@amd.com>
add missing mem type table
Signed-off-by: Peter Jun Park <peter.park@amd.com>
fix formatting
fmt
* add custom css
Signed-off-by: Peter Jun Park <peter.park@amd.com>
fix css fs
* make images/figs click-to-expand
Signed-off-by: Peter Jun Park <peter.park@amd.com>
add missed image
update
fix link
* update documentation link in README
Signed-off-by: Peter Jun Park <peter.park@amd.com>
* formatting fixes
Signed-off-by: Peter Jun Park <peter.park@amd.com>
more formatting
* fix heading
Signed-off-by: Peter Jun Park <peter.park@amd.com>
* move archived docs
Signed-off-by: Peter Jun Park <peter.park@amd.com>
* exclude archived docs from docs build
Signed-off-by: Peter Jun Park <peter.park@amd.com>
* update archived docs workflow
Signed-off-by: Peter Jun Park <peter.park@amd.com>
move files
update archived docs workflow
Signed-off-by: Peter Jun Park <peter.park@amd.com>
fix version number
clean up workflow
workflow test
workflow test
another workflow test
* rm docs linting
Signed-off-by: Peter Jun Park <peter.park@amd.com>
* Apply cmake-format suggested changes
Signed-off-by: Sam Wu <22262939+samjwu@users.noreply.github.com>
* Apply cmake-format
Signed-off-by: Sam Wu <22262939+samjwu@users.noreply.github.com>
---------
Signed-off-by: Peter Jun Park <peter.park@amd.com>
Signed-off-by: Sam Wu <22262939+samjwu@users.noreply.github.com>
Co-authored-by: Sam Wu <22262939+samjwu@users.noreply.github.com>
[ROCm/rocprofiler-compute commit: a0dc485ceb]
Should have been checking environment variables for package name
overrides.
Disabled RPM Build's AUTOREQ and AUTOPROV
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
[ROCm/rocprofiler-compute commit: 98e7d0cfb3]
abchoudh-amd
vedithal-amd
jamessiddeley-amd
xuchen-amd
Fei Zheng
Young Hui - AMD
systems-assistant[bot]
ywang103-amd
Jason Bonnell
Ammar ELWazir
cfallows-amd
David Galiffi
Kunal Malviya
Cole Ramos
Peter Park
Karl W Schulz