Memory Allocation Tracking (#1142)
* Initial commit: Need to implement wrapper function to collect data and test that wrapper function is correctly replacing core HSA functions * Attempted to implement wrapper implementation for hsa memory allocation functions. Need to modify generate record files and test if implementation is working as expected * Debugging and implementing generateCSV function * Memory allocation size and starting address outputted to csv and json file formats * Formatting * Initial setup for OTF2 and Perfetto generation * Collecting agent id for memory_allocation and formatting * Modified memory_allocation.cpp to set up code for AMD_EXT commands * Support for memory_pool_allocate added * Removed accidently added file * Made flag optional and added more OTF2 and Perfetto code. Needs testing to ensure perfetto and OTF2 works * Formatting * Fixed perfetto and otf2 output * Fixed flag issue due to incorrect buffer use * Updated documentation * Small cleaning and comments * Added test for HSA memory allocation tracing * Fixed summary test validation errors due to allocation tracing. Added type to location_base to create unique event ids for allocation due to OTF2 trace error * Decreased lower limit of hip calls for test * Modified summary tests to vary number of allocate requests * Minor fixes to address comments. Still need to address OTF2 comments * Fix docs and changed OTF2 to use enum for type specified in location_base construction * Fixed schema error * Added vmem command tracking. Need to add test * Updated test to work with vmem command and updated generateCSV to output int instead of hex string. * OTF2 enum update and mispelling fix * CI does not support Virtual Memory API. Removed vmem test. Will add back if CI is modifed to suport vmem API * Update CMakeLists.txt for memory allocation test * Updated summary test * Minor fixes to address comments * Moved domain_type.hpp enum to before LAST * Fixed compile errors and formatting * Fixed stats summary domain name error * Added rocprofv3 test * Page migration test fix * Undo page migration test changes. Failures do not appear to have to do with memory allocation
This commit is contained in:
+10
-2
@@ -141,13 +141,13 @@ For MPI applications (or other job launchers such as SLURM), place rocprofv3 ins
|
||||
aggregate_tracing_options,
|
||||
"-r",
|
||||
"--runtime-trace",
|
||||
help="Collect tracing data for HIP runtime API, Marker (ROCTx) API, RCCL API, Memory operations (copies and scratch), and Kernel dispatches. Similar to --sys-trace but without tracing HIP compiler API and the underlying HSA API.",
|
||||
help="Collect tracing data for HIP runtime API, Marker (ROCTx) API, RCCL API, Memory operations (copies, scratch, and allocation), and Kernel dispatches. Similar to --sys-trace but without tracing HIP compiler API and the underlying HSA API.",
|
||||
)
|
||||
add_parser_bool_argument(
|
||||
aggregate_tracing_options,
|
||||
"-s",
|
||||
"--sys-trace",
|
||||
help="Collect tracing data for HIP API, HSA API, Marker (ROCTx) API, RCCL API, Memory operations (copies and scratch), and Kernel dispatches.",
|
||||
help="Collect tracing data for HIP API, HSA API, Marker (ROCTx) API, RCCL API, Memory operations (copies, scratch, and allocations), and Kernel dispatches.",
|
||||
)
|
||||
|
||||
basic_tracing_options = parser.add_argument_group("Basic tracing options")
|
||||
@@ -173,6 +173,11 @@ For MPI applications (or other job launchers such as SLURM), place rocprofv3 ins
|
||||
"--memory-copy-trace",
|
||||
help="For collecting Memory Copy Traces. This was part of HIP and HSA traces in previous rocprof versions but is now a separate option",
|
||||
)
|
||||
add_parser_bool_argument(
|
||||
basic_tracing_options,
|
||||
"--memory-allocation-trace",
|
||||
help="For collecting Memory Allocation Traces. Displays starting address, allocation size, and agent where allocation occurred.",
|
||||
)
|
||||
add_parser_bool_argument(
|
||||
basic_tracing_options,
|
||||
"--scratch-memory-trace",
|
||||
@@ -686,6 +691,7 @@ def run(app_args, args, **kwargs):
|
||||
"marker_trace",
|
||||
"kernel_trace",
|
||||
"memory_copy_trace",
|
||||
"memory_allocation_trace",
|
||||
"scratch_memory_trace",
|
||||
"rccl_trace",
|
||||
):
|
||||
@@ -697,6 +703,7 @@ def run(app_args, args, **kwargs):
|
||||
"marker_trace",
|
||||
"kernel_trace",
|
||||
"memory_copy_trace",
|
||||
"memory_allocation_trace",
|
||||
"scratch_memory_trace",
|
||||
"rccl_trace",
|
||||
):
|
||||
@@ -724,6 +731,7 @@ def run(app_args, args, **kwargs):
|
||||
["rccl_trace", "RCCL_API_TRACE"],
|
||||
["kernel_trace", "KERNEL_TRACE"],
|
||||
["memory_copy_trace", "MEMORY_COPY_TRACE"],
|
||||
["memory_allocation_trace", "MEMORY_ALLOCATION_TRACE"],
|
||||
["scratch_memory_trace", "SCRATCH_MEMORY_TRACE"],
|
||||
]
|
||||
).items():
|
||||
|
||||
Referens i nytt ärende
Block a user