AI/rocm-systems
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,007 Commits 3 Branches 0 Tags
c5ea1d0ff0e0eb7b03eec41f1b10202a9746a00e
Commit Graph

1007 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
benrichard-amd c5ea1d0ff0 Set correct number of TCC channels for gfx942 
Ran into rocprof error:
ROCProfiler: fatal error: input metric'TCC_EA0_RDREQ[16]' not supported on this hardware: gfx942

gfx942 has 16 channels, not 32.

Signed-off-by: benrichard-amd <ben.richard@amd.com>
 2024-08-09 09:46:42 -04:00
benrichard-amd 28900f88cb Remove unused method 
Signed-off-by: benrichard-amd <ben.richard@amd.com>
 2024-08-09 09:46:42 -04:00
benrichard-amd 4723ecb6c9 Update to work with rocprof v1 
Signed-off-by: benrichard-amd <ben.richard@amd.com>
 2024-08-09 09:46:42 -04:00
benrichard-amd d77d9f1719 Save accumulate counters to SQ_ files 
Omniperf analyze expects the accumulate files to be in SQ_*.csv files.

Since these files also contain PMC counters (we are trying to
fit as many counters into each file as possible to minimize runs),
we need to include these SQ_*.csv files in pmc_perf.csv.

Signed-off-by: benrichard-amd <ben.richard@amd.com>
 2024-08-09 09:46:42 -04:00
benrichard-amd 241a0949e1 Remove duplicate normal counters 
Interleve TCC channel counters in putput file  e.g.  TCC_HIT[0] TCC_ATOMIC[0] ... TCC_HIT[1] TCC_ATOMIC[1]

Signed-off-by: benrichard-amd <ben.richard@amd.com>
 2024-08-09 09:46:42 -04:00
benrichard-amd a4fdee488b Interleve TCC channel counters 
Signed-off-by: benrichard-amd <ben.richard@amd.com>
 2024-08-09 09:46:42 -04:00
benrichard-amd c93fead779 Improve perfmon coalescing 
Signed-off-by: benrichard-amd <ben.richard@amd.com>
 2024-08-09 09:46:42 -04:00
coleramos425 4532d7df74 Explicitly define Python path in CTest test declaration 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-08-09 09:46:42 -04:00
coleramos425 013bf218d0 Comply to Python formatting 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-08-09 09:46:42 -04:00
coleramos425 3d5493a3ff Add detail to Omniperf logs 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-08-09 09:46:42 -04:00
coleramos425 19a229a9f2 Split rocprofv2 cmd args prior to subprocess call (#347) 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-08-09 09:46:42 -04:00
coleramos425 fa90752d0c Fix bug in abs diff calculation for analysis output 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-08-09 09:46:42 -04:00
Ben Richard 69e5c32d52 Work around crash when profiling multi-process/multi-GPU application (#376) 
* Fix crash in multi-GPU scenario

Exclude -o option when invoking rocprof so that each rocprof process
writes to a different .csv file. Concatenate into a single .csv file
when finished.

Signed-off-by: benrichard-amd <ben.richard@amd.com>

* Only combine csv files when using rocprofv2

rocprofv1 does not have separate csv files

Signed-off-by: benrichard-amd <ben.richard@amd.com>

* Fix indices in combined CSV file

Use ignore_index flag to ensure there are no duplicate indices.

Signed-off-by: benrichard-amd <ben.richard@amd.com>

* Fix Dispatch_ID column and remove unnamed column

-Pandas was inserting an unnamed column (index column)
-Overwrite the Dispatch_ID column so that every row is unique, starting at 0
-Remove fixup_rocprofv2_dispatch_ids as no longer needed

Signed-off-by: benrichard-amd <ben.richard@amd.com>

* Fix code formatting

Signed-off-by: benrichard-amd <ben.richard@amd.com>

* Fix code formatting (for real this time)

Signed-off-by: benrichard-amd <ben.richard@amd.com>

---------

Signed-off-by: benrichard-amd <ben.richard@amd.com>
 2024-08-09 09:46:42 -04:00
David Galiffi 98e7d0cfb3 Fixing for packaging (#386) 
Should have been checking environment variables for package name
overrides.
Disabled RPM Build's AUTOREQ and AUTOPROV

Signed-off-by: David Galiffi <David.Galiffi@amd.com>
 2024-07-19 22:16:59 -04:00
coleramos425 6d24a074b4 Upgrade docs package 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-06-04 00:33:18 +00:00
coleramos425 d100173739 Update VERSION, CHANGELOG, and citation for next release 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-06-04 00:03:43 +00:00
Karl W Schulz 18f96b6b23 set CMAKE_HIP_ARCHITECTURES at configure time so that build matches 
desired execution environment

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
 2024-06-03 17:41:19 -05:00
Karl W Schulz c31ecf2a90 include ENABLE_TESTS=ON for profile tests 
Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
 2024-06-03 17:08:48 -05:00
Karl W Schulz 90ff4bf575 wrap up test compilation with an ENABLE_TESTS option 
Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
 2024-06-03 16:20:14 -05:00
Karl W Schulz 5077f9e24f update CI to build necessary vcopy binary via new additions to build 
system (a la, just run make)

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
 2024-06-03 16:20:14 -05:00
Karl W Schulz c54d3b34ea change project language to HIP; identify tests as cmake subdir 
Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
 2024-06-03 16:20:14 -05:00
Karl W Schulz 382a83f48b adding build-collateral for test subdir 
Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
 2024-06-03 16:20:14 -05:00
coleramos425 7046ea15bd Comply to formatting 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-06-03 13:47:45 -05:00
coleramos425 1d19ae9483 Detection of MI308X and hardcode mclk to address bug in rocm-smi 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-06-03 13:47:45 -05:00
coleramos425 295b344646 Re-implementing HBM stack / XCD info for incoming product sku 
Co-authored-by: Nicholas Curtis <nicholas.curtis@amd.com>
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-06-03 13:47:45 -05:00
coleramos425 7c06b4d2ae Missing test case 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-05-31 22:02:18 +00:00
coleramos425 55a31ef880 Fixing some typos from debug 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-05-31 21:46:18 +00:00
coleramos425 9ff90edc10 Comply to Python formatting 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-05-31 16:09:58 -05:00
coleramos425 fef13818f9 Remove outdated CTest case from CMakeLists 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-05-31 16:09:58 -05:00
coleramos425 0315ed4458 Update gpu-id on MI300 CI workloads and update test case 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-05-31 16:09:58 -05:00
coleramos425 73f18d20f5 Remove old workloads 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-05-31 16:09:58 -05:00
JoseSantosAMD 91f635e3e6 Updated test_analyze_commands to include mi300 directories 
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>
 2024-05-31 16:09:58 -05:00
coleramos425 c079f1330b Add vcopy workloads for MI300A and MI300X 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-05-31 16:09:58 -05:00
JoseSantosAMD db7d77ca3f updating expected exit codes for pytest 
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>
 2024-05-31 16:09:58 -05:00
JoseSantosAMD 6d98a1eb5e update test analyze scripts 
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>
 2024-05-31 16:09:58 -05:00
coleramos425 618c064571 Remove debug print statements from profile_general 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-05-31 16:09:58 -05:00
coleramos425 cdccd93afa Remove invalid test from workload generation script 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-05-31 16:09:58 -05:00
coleramos425 b8af814467 Add CI workloads for MI300A_A1 and MI300X_A1 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-05-31 16:09:58 -05:00
coleramos425 7d79fcc276 Remove redundant code in PyTest 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-05-31 16:09:58 -05:00
coleramos425 09264591f3 Support MI300 in PyTest profile tests 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-05-31 16:09:58 -05:00
coleramos425 49371cacec Create dedicated subdirectory in perfmon configs for archs supporting roofline 
Separate subdirs allows us to target different roofline counters for different archs (i.e. MI300 vs MI200)

Signed-off-by: coleramos425 <colramos@amd.com>
 2024-05-31 16:09:58 -05:00
Karl W. Schulz 7a01f499d7 remove use of distutils package entirely to avoid future deprecation 
issues

Signed-off-by: Karl W. Schulz <karl.schulz@amd.com>
 2024-05-28 15:34:00 -05:00
Karl W. Schulz aca084f41c updated approach for runtime dependency check that does use "pkg_resources" 
which will reportedly be deprecated at some point in the future.

Signed-off-by: Karl W. Schulz <karl.schulz@amd.com>
 2024-05-28 15:34:00 -05:00
coleramos425 c6cfa9cc26 Wrap text displayed in 'Top Dispatch' table for neatness 
Signed-off-by: coleramos425 <colramos@amd.com>
 2024-05-28 15:34:00 -05:00
Nicholas Curtis 047d7771f3 Add fix for case where we pass a single 'nan' value to to_avg 
This is triggered by doing e.g., analyze -p <whatever> -k <kernel> -n per_kernel -b 17 18
Manifests as e.g.:

```
  ERROR [analysis] 'float' object has no attribute 'empty'
```

because of:

https://github.com/ROCm/omniperf/blob/d1ee2ec8709b21f2e72536cc14dba8ac2f8621ab/src/utils/parser.py#L135

Instead, we first check whether numpy thinks the whole array is nan's, and bail early if so

Signed-off-by: Nicholas Curtis <nicurtis@amd.com>
 2024-05-28 15:34:00 -05:00
Nicholas Curtis 1f584c1612 handle unspecified case 
Signed-off-by: Nicholas Curtis <nicurtis@amd.com>
 2024-05-28 15:34:00 -05:00
Nick Curtis 5579beeed5 fix formatting 
Signed-off-by: Nick Curtis <nicholas.curtis@amd.com>
 2024-05-28 15:34:00 -05:00
Nick Curtis 989dd3b7ae Add ability to overide arch when name missing in rocminfo 
Signed-off-by: Nick Curtis <nicholas.curtis@amd.com>
 2024-05-28 15:34:00 -05:00
Cole Ramos 57c4f2047e Update packaging.yml checkout library 
Signed-off-by: Cole Ramos <colramos@amd.com>
 2024-05-17 15:00:18 -05:00
Cole Ramos 5f0978c278 Update AUTHORS 
Signed-off-by: Cole Ramos <colramos@amd.com>
 2024-05-17 13:42:36 -05:00