- improved error handling in dyninst
- improved error handling in omnitrace exe
- new logging facility for omnitrace exe
- improved backtraces
- disable concurrent kernels in rocprofiler
- updates `setup-env.sh` and modulefile
- set `omnitrace_ROOT`
- set `HSA_TOOLS_LIB` if roctracer or rocprofiler enabled
- set `ROCP_TOOL_LIB` if rocprofiler enabled
- closes#163
- No longer make setting `HSA_ENABLE_INTERRUPT=0` the default
- this has performance implications
- this was set to workaround a bug in ROCR which caused an ioctl call in
ROCm to hang when interrupted. But it was only interrupted when realtime
sampling was enabled since the CPU-clock doesn't increment when waiting
- This bug should be fixed in ROCm 5.3
- omnitrace no longer activates a realtime sampler by default when
sampling, thus this bug is no longer encountered unless the user
explicitly triggers realtime sampling
[ROCm/rocprofiler-systems commit: 90ff7188f8]
- improved error handling in dyninst
- improved error handling in omnitrace exe
- new logging facility for omnitrace exe
- improved backtraces
- disable concurrent kernels in rocprofiler
- updates `setup-env.sh` and modulefile
- set `omnitrace_ROOT`
- set `HSA_TOOLS_LIB` if roctracer or rocprofiler enabled
- set `ROCP_TOOL_LIB` if rocprofiler enabled
- closes#163
- No longer make setting `HSA_ENABLE_INTERRUPT=0` the default
- this has performance implications
- this was set to workaround a bug in ROCR which caused an ioctl call in
ROCm to hang when interrupted. But it was only interrupted when realtime
sampling was enabled since the CPU-clock doesn't increment when waiting
- This bug should be fixed in ROCm 5.3
- omnitrace no longer activates a realtime sampler by default when
sampling, thus this bug is no longer encountered unless the user
explicitly triggers realtime sampling
Prior to launch some ASICs may re-use PCI DIDs from older generations.
This can cause issues during topology initialization as hsa_gfxip_table
lookups will override sysfs-provided gfx versions, causing incorrect
gfxip selection. Since no new entries will be added to hsa_gfxip_table,
limit its search only to pre-GFX11 ASICs.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I53eaefac5db2650a36a6ce9f21daf750f50cfd26
Prior to launch some ASICs may re-use PCI DIDs from older generations.
This can cause issues during topology initialization as hsa_gfxip_table
lookups will override sysfs-provided gfx versions, causing incorrect
gfxip selection. Since no new entries will be added to hsa_gfxip_table,
limit its search only to pre-GFX11 ASICs.
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I53eaefac5db2650a36a6ce9f21daf750f50cfd26
[ROCm/ROCR-Runtime commit: 79279e860f]
Remove the activity_prof::CallbacksTable. The table was redundant with
the information already stored in the roctracer library. Instead use a
single callback into the roctracer library to query whether the activity
is enabled, and to report it.
Change-Id: I2e05b0881bb4a1953c14361d00ea310d02eb6e0c
[ROCm/clr commit: 52eb28930a]
Remove the activity_prof::CallbacksTable. The table was redundant with
the information already stored in the roctracer library. Instead use a
single callback into the roctracer library to query whether the activity
is enabled, and to report it.
Change-Id: I2e05b0881bb4a1953c14361d00ea310d02eb6e0c
Profiling should be enabled for any command reporting activities as the
activity record captures the profilingInfo's start and end timestamps.
Since IS_PROFILER_ON is only used to determine whether API tracing is
enabled, there is no need to expose it globally, it should be a property
of the activity_prof::CallbacksTable.
Change-Id: I44a0d19ed2862606cfbc9a98c1a07a336ab7e26c
[ROCm/clr commit: e713b5c7d0]
Profiling should be enabled for any command reporting activities as the
activity record captures the profilingInfo's start and end timestamps.
Since IS_PROFILER_ON is only used to determine whether API tracing is
enabled, there is no need to expose it globally, it should be a property
of the activity_prof::CallbacksTable.
Change-Id: I44a0d19ed2862606cfbc9a98c1a07a336ab7e26c
The activity_ is only instantiated if profiling is enabled.
Remove the HIP private global record ID. Instead, use the correlation ID
stored in the hip_api_data_t by the profiler while the last HIP function
is in scope.
For NDRange and Copy commands, store the kernel name and byte size
(respectively) in the record.
General cleanups to improve the code's readability.
Change-Id: I01907484b0d9611eb9440c3a7c4865479dc42289
[ROCm/clr commit: 4fbae91468]
The activity_ is only instantiated if profiling is enabled.
Remove the HIP private global record ID. Instead, use the correlation ID
stored in the hip_api_data_t by the profiler while the last HIP function
is in scope.
For NDRange and Copy commands, store the kernel name and byte size
(respectively) in the record.
General cleanups to improve the code's readability.
Change-Id: I01907484b0d9611eb9440c3a7c4865479dc42289
- Add hipDeviceSynchronize after hipMemset with device memory, as hipMemset is asynchronous in this case
Change-Id: Ia55d106931984fed450be29c1b745c8069151f76
[ROCm/hip commit: 6e021f1adc]
- Add hipDeviceSynchronize after hipMemset with device memory, as hipMemset is asynchronous in this case
Change-Id: Ia55d106931984fed450be29c1b745c8069151f76
Remove the api_callbacks_table_t that was holding the API activities and
user callbacks. Instead use a single roctracer callback (TracerCallback)
used to report both API activities and callbacks.
Remove the hipInitActivityCallback that was setting the ROCtracer
callback and memory pool for asynchronous activities as it did not
allow disctinct pools to be used for each activity. Instead, use
hipRegisterTracerCallback to set the single roctracer callback.
Change-Id: I4c10f04f29a6e4cce8caf15db3016c3f72c86b04
[ROCm/clr commit: 82bce811ee]
Remove the api_callbacks_table_t that was holding the API activities and
user callbacks. Instead use a single roctracer callback (TracerCallback)
used to report both API activities and callbacks.
Remove the hipInitActivityCallback that was setting the ROCtracer
callback and memory pool for asynchronous activities as it did not
allow disctinct pools to be used for each activity. Instead, use
hipRegisterTracerCallback to set the single roctracer callback.
Change-Id: I4c10f04f29a6e4cce8caf15db3016c3f72c86b04
The CallbacksTable::is_enabled() can simply be implemented by checking
if enabled_api_count is > 0. The ROCclr does not use IS_PROFILER_ON
to report asynchronous activities.
Change-Id: Iab3d034357e51282bf2c453b2ac5c9726786b9eb
[ROCm/clr commit: eeddffc1b1]
The CallbacksTable::is_enabled() can simply be implemented by checking
if enabled_api_count is > 0. The ROCclr does not use IS_PROFILER_ON
to report asynchronous activities.
Change-Id: Iab3d034357e51282bf2c453b2ac5c9726786b9eb
Since the hip_api_data and record are only needed at the HIP function's
scope, there is no need to allocate/free them in the ROCtracer activity
callback, they can reside on the HIP function's stack frame.
This solves an issue with the thread local stacks of records the tracer
maintains that are destroyed first (before any global destructor) on
process exit, making it impossible to use HIP functions in global
destructors when the profiler is enabled.
Change-Id: Ib1d70124d009a44dc1f08d41edff95e5f9f84369
[ROCm/clr commit: 85fd969253]
Since the hip_api_data and record are only needed at the HIP function's
scope, there is no need to allocate/free them in the ROCtracer activity
callback, they can reside on the HIP function's stack frame.
This solves an issue with the thread local stacks of records the tracer
maintains that are destroyed first (before any global destructor) on
process exit, making it impossible to use HIP functions in global
destructors when the profiler is enabled.
Change-Id: Ib1d70124d009a44dc1f08d41edff95e5f9f84369
Jonathan R. Madsen
Graham Sider
Daniel Phillips
Maneesh Gupta
ROCm CI Service AccountиLaurent Morichetti
kjayapra-amd
Satyanvesh Dittakavi
Chauncey Hui
Wenkai Du
pghafari