AI/rocm-systems
2
0
Форкнуть 0
Код Задачи Запросы на слияние Действия Пакеты Проекты Релизы Вики Активность
Files
25e5e1073ce1d6df6131a6d8c54ab097987bd2fe
rocm-systems/src/docs/performance_analysis.md
T
Karl W. Schulz caae8ec211 docs: heading change 
Signed-off-by: Karl W. Schulz <karl.schulz@amd.com>
2022-11-11 15:05:27 -06:00

16 KiB
Исходник Ответственный История

Performance Profiling

.. toctree::
   :glob:
   :maxdepth: 4

The Omniperf repository includes source code for a sample GPU compute workload, vcopy.cpp. A copy of this file is available in the share/sample subdirectory after a normal Omniperf installation, or via the $OMNIPERF_SHARE/sample directory when using the supplied modulefile.

A compiled version of this workload is used throughout the following sections to demonstrate the use of Omniperf in MI GPU performance analysis. Unless otherwise noted, the performance analysis is done on the MI200 platform.

Workload Compilation

vcopy compilation:

$ hipcc vcopy.cpp -o vcopy
$ ls
vcopy   vcopy.cpp
$ ./vcopy 1048576 256
Finished allocating vectors on the CPU
Finished allocating vectors on the GPU
Finished copying vectors to the GPU
sw thinks it moved 1.000000 KB per wave
Total threads: 1048576, Grid Size: 4096 block Size:256, Wavefronts:16384:
Launching the  kernel on the GPU
Finished executing kernel
Finished copying the output vector from the GPU to the CPU
Releasing GPU memory
Releasing CPU memory

Omniperf Profiling

The omniperf script, availible through the Omniperf repository, is used to aquire all necessary perfmon data through analysis of compute workloads.

omniperf help:

$ omniperf profile --help
ROC Profiler:  /usr/bin/rocprof

usage:

omniperf profile --name <workload_name> [profile options] [roofline options] -- <profile_cmd>



-------------------------------------------------------------------------------

Examples:

        omniperf profile -n vcopy_all -- ./vcopy 1048576 256

        omniperf profile -n vcopy_SPI_TD -b SQ TCC -- ./vcopy 1048576 256

        omniperf profile -n vcopy_kernel -k vecCopy -- ./vcopy 1048576 256

        omniperf profile -n vcopy_disp -d 0 -- ./vcopy 1048576 256

        omniperf profile -n vcopy_roof --roof-only -- ./vcopy 1048576 256

-------------------------------------------------------------------------------



Help:
  -h, --help                      show this help message and exit

General Options:
  -v, --version                   show program's version number and exit
  -V, --verbose                   Increase output verbosity

Profile Options:
  -n , --name                                           Assign a name to workload.
  -p , --path                                           Specify path to save workload.
                                                        (DEFAULT: /home/colramos/GitHub/omniperf/workloads/<name>)
  -k  [ ...], --kernel  [ ...]                          Kernel filtering.
  -b  [ ...], --ipblocks  [ ...]                        IP block filtering:
                                                           SQ
                                                           SQC
                                                           TA
                                                           TD
                                                           TCP
                                                           TCC
                                                           SPI
                                                           CPC
                                                           CPF
  -d  [ ...], --dispatch  [ ...]                        Dispatch ID filtering.
  --no-roof                                             Profile without collecting roofline data.
  -- [ ...]                                             Provide command for profiling after double dash.

Standalone Roofline Options:
  --roof-only                                           Profile roofline data only.
  --sort                                                Overlay top kernels or top dispatches: (DEFAULT: kernels)
                                                           kernels
                                                           dispatches
  -m , --mem-level                                      Filter by memory level: (DEFAULT: ALL)
                                                           HBM
                                                           L2
                                                           vL1D
                                                           LDS
  --axes  [ ...]                                        Desired axis values for graph. As follows:
                                                           xmin xmax ymin ymax
  --device                                              GPU device ID. (DEFAULT: ALL)

  • The -k <kernel> flag allows for kernel filtering, which is compatible with the current rocprof utility.

  • The -d <dispatch> flag allows for dispatch ID filtering, which is compatible with the current rocprof utility.

  • The -b <ipblocks> allows system profiling on one or more selected IP blocks to speed up the profiling process. One can gradually incorporate more IP blocks, without overwriting performance data acquired on other IP blocks.

The following sample command profiles the vcopy workload.

vcopy profiling:

$ omniperf profile --name vcopy -- ./vcopy 1048576 256
ROC Profiler:  /usr/bin/rocprof
 
--------
Profile only
--------
 
omniperf ver:  v1.0.3
Path:  workloads
Target:  mi200
Command:  ./vcopy 1048576 256
Kernel Selection:  None
Dispatch Selection:  None
IP Blocks: All
RPL: on '220527_130247' from '/opt/rocm-5.2.0-9768/rocprofiler' in '/home/amd/xlu/test'
RPL: profiling '""./vcopy 1048576 256""'
RPL: input file 'workloads/vcopy/mi200/perfmon/SQ_IFETCH_LEVEL.txt'
RPL: output dir '/tmp/rpl_data_220527_130247_1781699'
RPL: result dir '/tmp/rpl_data_220527_130247_1781699/input0_results_220527_130247'
Finished allocating vectors on the CPU
ROCProfiler: input from "/tmp/rpl_data_220527_130247_1781699/input0.xml"
  gpu_index =
  kernel =
  range =
  6 metrics
    GRBM_COUNT, GRBM_GUI_ACTIVE, SQ_WAVES, SQ_IFETCH, SQ_IFETCH_LEVEL, SQ_ACCUM_PREV_HIRES
Finished allocating vectors on the GPU
Finished copying vectors to the GPU
sw thinks it moved 1.000000 KB per wave
Total threads: 1048576, Grid Size: 4096 block Size:256, Wavefronts:16384:
Launching the  kernel on the GPU
Finished executing kernel
Finished copying the output vector from the GPU to the CPU
Releasing GPU memory
Releasing CPU memory
 
... ...
ROCPRofiler: 1 contexts collected, output directory /tmp/rpl_data_220527_130317_1787038/input_results_220527_130317
File 'workloads/vcopy/mi200/timestamps.csv' is generating
Total detected GPU devices: 2
GPU Device 0: Profiling...
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
HBM BW, GPU ID: 0, workgroupSize:256, workgroups:2097152, experiments:100, traffic:8589934592 bytes, duration:6.2 ms, mean:1382.7 GB/sec, stdev=2.4 GB/sec
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
L2 BW, GPU ID: 0, workgroupSize:256, workgroups:8192, experiments:100, traffic:687194767360 bytes, duration:157.9 ms, mean:4358.7 GB/sec, stdev=4.7 GB/sec
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
L1 BW, GPU ID: 0, workgroupSize:256, workgroups:16384, experiments:100, traffic:26843545600 bytes, duration:3.3 ms, mean:8247.1 GB/sec, stdev=5.1 GB/sec
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
LDS BW, GPU ID: 0, workgroupSize:256, workgroups:16384, experiments:100, traffic:33554432000 bytes, duration:2.4 ms, mean:14246.3 GB/sec, stdev=29.5 GB/sec
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
Peak FLOPs (FP32), GPU ID: 0, workgroupSize:256, workgroups:16384, experiments:100, FLOP:274877906944, duration:14.507 ms, mean:18949.6 GFLOPS, stdev=4.5 GFLOPS
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
Peak FLOPs (FP64), GPU ID: 0, workgroupSize:256, workgroups:16384, experiments:100, FLOP:137438953472, duration:7.5 ms, mean:18308.197266.1 GFLOPS, stdev=3.6 GFLOPS
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
Peak MFMA FLOPs (BF16), GPU ID: 0, workgroupSize:256, workgroups:16384, experiments:100, FLOP:2147483648000, duration:14.0 ms, mean:153574.8 GFLOPS, stdev=79.9 GFLOPS
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
Peak MFMA FLOPs (F16), GPU ID: 0, workgroupSize:256, workgroups:16384, experiments:100, FLOP:2147483648000, duration:14.5 ms, mean:147680.1 GFLOPS, stdev=34.7 GFLOPS
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
Peak MFMA FLOPs (F32), GPU ID: 0, workgroupSize:256, workgroups:16384, experiments:100, FLOP:536870912000, duration:14.5 ms, mean:37142.1 GFLOPS, stdev=8.4 GFLOPS
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
Peak MFMA FLOPs (F64), GPU ID: 0, workgroupSize:256, workgroups:16384, experiments:100, FLOP:268435456000, duration:7.3 ms, mean:36919.5 GFLOPS, stdev=14.1 GFLOPS
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
Peak MFMA IOPs (I8), GPU ID: 0, workgroupSize:256, workgroups:16384, experiments:100, IOP:2147483648000, duration:14.4 ms, mean:149570.6 GOPS, stdev=41.7 GOPS
GPU Device 1: Profiling...
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
HBM BW, GPU ID: 1, workgroupSize:256, workgroups:2097152, experiments:100, traffic:8589934592 bytes, duration:6.2 ms, mean:1382.7 GB/sec, stdev=2.9 GB/sec
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
L2 BW, GPU ID: 1, workgroupSize:256, workgroups:8192, experiments:100, traffic:687194767360 bytes, duration:157.6 ms, mean:4371.0 GB/sec, stdev=4.1 GB/sec
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
L1 BW, GPU ID: 1, workgroupSize:256, workgroups:16384, experiments:100, traffic:26843545600 bytes, duration:3.2 ms, mean:8297.4 GB/sec, stdev=11.6 GB/sec
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
LDS BW, GPU ID: 1, workgroupSize:256, workgroups:16384, experiments:100, traffic:33554432000 bytes, duration:1.8 ms, mean:18839.2 GB/sec, stdev=44.5 GB/sec
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
Peak FLOPs (FP32), GPU ID: 1, workgroupSize:256, workgroups:16384, experiments:100, FLOP:274877906944, duration:14.441 ms, mean:19037.6 GFLOPS, stdev=2.7 GFLOPS
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
Peak FLOPs (FP64), GPU ID: 1, workgroupSize:256, workgroups:16384, experiments:100, FLOP:137438953472, duration:7.5 ms, mean:18402.255859.1 GFLOPS, stdev=20.1 GFLOPS
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
Peak MFMA FLOPs (BF16), GPU ID: 1, workgroupSize:256, workgroups:16384, experiments:100, FLOP:2147483648000, duration:13.9 ms, mean:154240.3 GFLOPS, stdev=119.3 GFLOPS
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
Peak MFMA FLOPs (F16), GPU ID: 1, workgroupSize:256, workgroups:16384, experiments:100, FLOP:2147483648000, duration:14.5 ms, mean:148450.1 GFLOPS, stdev=112.6 GFLOPS
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
Peak MFMA FLOPs (F32), GPU ID: 1, workgroupSize:256, workgroups:16384, experiments:100, FLOP:536870912000, duration:14.4 ms, mean:37335.2 GFLOPS, stdev=43.1 GFLOPS
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
Peak MFMA FLOPs (F64), GPU ID: 1, workgroupSize:256, workgroups:16384, experiments:100, FLOP:268435456000, duration:7.2 ms, mean:37105.3 GFLOPS, stdev=39.5 GFLOPS
 99% [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| ]
Peak MFMA IOPs (I8), GPU ID: 1, workgroupSize:256, workgroups:16384, experiments:100, IOP:2147483648000, duration:14.3 ms, mean:150317.8 GOPS, stdev=203.5 GOPS

At the end of the profiling, all resulting csv files should be located in the SOC specific target directory, e.g., mi200.

Note: An SoC parameters file, sysinfo.csv, is also created to reflect the target device settings.

$ ls workloads/vcopy/mi200/
total 116
-rw-rw-r-- 1 amd amd   400 May 27 13:03 SQC_DCACHE_INFLIGHT_LEVEL.csv
-rw-rw-r-- 1 amd amd   452 May 27 13:03 SQC_DCACHE_TC_INFLIGHT_LEVEL.csv
-rw-rw-r-- 1 amd amd   451 May 27 13:03 SQC_DCACHE_UTCL1_INFLIGHT_LEVEL.csv
-rw-rw-r-- 1 amd amd   445 May 27 13:03 SQC_DCACHE_UTCL2_INFLIGHT_LEVEL.csv
-rw-rw-r-- 1 amd amd   396 May 27 13:03 SQC_ICACHE_INFLIGHT_LEVEL.csv
-rw-rw-r-- 1 amd amd   396 May 27 13:03 SQC_ICACHE_TC_INFLIGHT_LEVEL.csv
-rw-rw-r-- 1 amd amd   445 May 27 13:03 SQC_ICACHE_UTCL1_INFLIGHT_LEVEL.csv
-rw-rw-r-- 1 amd amd   442 May 27 13:03 SQC_ICACHE_UTCL2_INFLIGHT_LEVEL.csv
-rw-rw-r-- 1 amd amd   423 May 27 13:03 SQC_TC_INFLIGHT_LEVEL.csv
-rw-rw-r-- 1 amd amd   437 May 27 13:02 SQ_IFETCH_LEVEL.csv
-rw-rw-r-- 1 amd amd   374 May 27 13:03 SQ_INST_LEVEL_EXP.csv
-rw-rw-r-- 1 amd amd   374 May 27 13:03 SQ_INST_LEVEL_GDS.csv
-rw-rw-r-- 1 amd amd   374 May 27 13:02 SQ_INST_LEVEL_LDS.csv
-rw-rw-r-- 1 amd amd   392 May 27 13:03 SQ_INST_LEVEL_SMEM.csv
-rw-rw-r-- 1 amd amd   392 May 27 13:03 SQ_INST_LEVEL_VMEM.csv
-rw-rw-r-- 1 amd amd   516 May 27 13:03 SQ_LEVEL_WAVES.csv
drwxrwxr-x 2 amd amd  4096 May 27 13:02 perfmon
-rw-rw-r-- 1 amd amd 32797 May 27 13:03 pmc_perf.csv
-rw-rw-r-- 1 amd amd   958 May 27 13:04 roofline.csv
-rw-rw-r-- 1 amd amd   469 May 27 13:03 sysinfo.csv
-rw-rw-r-- 1 amd amd   317 May 27 13:03 timestamps.csv

IP Block Profiling

One can profile a selected IP Block to speed up the profiling process. All profiling results are accumulated in the same target directory, without overwriting those for other IP blocks, hence enabling the incremental profiling and analysis.

The following example only profiles SQ and TCC, skipping all other IP Blocks.

$ omniperf profile --name vcopy -b SQ TCC -- ./sample/vcopy 1048576 256
ROC Profiler:  /usr/bin/rocprof

--------
Profile only
--------

omniperf ver:  v1.0.3
Path:  workloads
Target:  mi200
Command:  ./vcopy 1048576 256
Kernel Selection:  None
Dispatch Selection:  None
IP Blocks:  ['SQ', 'TCC']
fname: pmc_ta_perf: Skipped
fname: pmc_sq_perf3: Added
fname: pmc_sqc_icache_perf2: Skipped
fname: pmc_sqc_dcache_perf2: Skipped
fname: pmc_sqc_dcache_perf3: Skipped
fname: pmc_sqc_icache_perf4: Skipped
fname: pmc_sqc_dcache_perf5: Skipped
fname: pmc_sqc_dcache_perf4: Skipped
fname: pmc_cpc_perf: Skipped
fname: pmc_sqc_icache_perf1: Skipped
fname: pmc_sq_perf4: Added
fname: pmc_sqc_icache_perf5: Skipped
fname: pmc_sq_perf5: Added
fname: pmc_grbm_perf: Skipped
fname: pmc_sq_perf8: Added
fname: pmc_sq_perf2: Added
fname: pmc_sq_perf6: Added
fname: pmc_sqc_icache_perf3: Skipped
fname: pmc_sqc_dcache_perf1: Skipped
fname: pmc_sq_perf7: Added
fname: pmc_cpf_perf: Skipped
fname: pmc_sqc_dcache_perf6: Skipped
fname: pmc_tcp_perf: Skipped
fname: pmc_spi_perf: Skipped
fname: pmc_td_perf: Skipped
fname: pmc_tcc_perf: Added
fname: pmc_tcc2_perf: Skipped
fname: pmc_sq_perf1: Added
RPL: on '220527_130730' from '/opt/rocm-5.2.0-9768/rocprofiler' in '/home/amd/xlu/test'
RPL: profiling '""./vcopy 1048576 256""'
RPL: input file 'workloads/vcopy/mi200/perfmon/SQ_IFETCH_LEVEL.txt'
RPL: output dir '/tmp/rpl_data_220527_130730_1788165'
RPL: result dir '/tmp/rpl_data_220527_130730_1788165/input0_results_220527_130730'
Finished allocating vectors on the CPU
ROCProfiler: input from "/tmp/rpl_data_220527_130730_1788165/input0.xml"
 
... ...
ROCPRofiler: 1 contexts collected, output directory /tmp/rpl_data_220527_130751_1791421/input_results_220527_130751
File 'workloads/vcopy/mi200/timestamps.csv' is generating
Total detected GPU devices: 2
GPU Device 0: Profiling...
... ...

Kernel Filtering

The following example demonstrates profiling on selected kernels:

$ omniperf profile --name vcopy -k vecCopy -- ./vcopy 1048576 256
ROC Profiler:  /usr/bin/rocprof
 
--------
Profile only
--------
 
omniperf ver:  v1.0.3
Path:  workloads
Target:  mi200
Command:  ./vcopy 1048576 256
Kernel Selection:  ['vecCopy']
Dispatch Selection:  None
IP Blocks: All
RPL: on '220527_164748' from '/opt/rocm-5.2.0-9768/rocprofiler' in '/home/amd/xlu/test'
RPL: profiling '""./vcopy 1048576 256""'
RPL: input file 'workloads/vcopy/mi200/perfmon/SQ_IFETCH_LEVEL.txt'
RPL: output dir '/tmp/rpl_data_220527_164748_1795414'
RPL: result dir '/tmp/rpl_data_220527_164748_1795414/input0_results_220527_164748'
Finished allocating vectors on the CPU
ROCProfiler: input from "/tmp/rpl_data_220527_164748_1795414/input0.xml"
  gpu_index =
  kernel = vecCopy
 
... ...

Dispatch Filtering

The following example demonstrates profiling on selected dispatches:

$ omniperf profile --name vcopy -d 0 -- ./vcopy 1048576 256
ROC Profiler:  /usr/bin/rocprof
 
--------
Profile only
--------
 
omniperf ver:  v1.0
Path:  workloads
Target:  mi200
Command:  ./vcopy 1048576 256
Kernel Selection:  None
Dispatch Selection:  ['0']
IP Blocks: All
... ...
Ссылка в новой задаче Показать git blame Копировать постоянную ссылку