AI/rocm-systems
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update HSA async copy active signals handling (#732)

Browse Source 
* Enable INFO logging on retried CI jobs

* Update lib/rocprofiler-sdk/async_copy.cpp

- rework active_signals
  - make hsa_signal_t member variable
  - remove sync from destructor
  - replace _is_set with atomic counter
  - timeout of 30 seconds hsa_signal_wait
  - switch from relaxed to scacquire/screlease memory ordering
- improve logging and error handling
- destroy hsa signal in active_signals in async_fini

* Update lib/rocprofiler-sdk/async_copy.cpp

- active_signals::create
- change initial value of signal to 1 instead of value of completion signal
- change condition trigger of signal callback

* Update tests/counter-collection/validate.py

* Update lib/rocprofiler-sdk/async_copy.cpp

- improved logging
- fix hsa_signal_wait_scacquire_fn check

* Cleanup tests/lib/transpose/transpose.cpp

- remove huge comment block

* Appears to be working on MI200

Dependency Versions:

clr: f7b1398361  - compile mode: release

hsa-runtime: 4cd6c62f25dbbdbaa8580dd4ad8f388c98c508da - compile mode: RelWithDebug

* Update source/lib/rocprofiler-sdk/hsa/async_copy.cpp

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Format fix

---------

Co-authored-by: Benjamin Welton <bewelton@amd.com>
Co-authored-by: Ammar ELWazir <ammar.elwazir@amd.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ammar ELWazir <aelwazir@hpe6u-21.amd.com>

[ROCm/rocprofiler-sdk commit: 8c5399a68a]
This commit is contained in:
Jonathan R. Madsen
2024-04-09 08:31:08 -05:00
committed by GitHub
parent e2c30bd438
commit 73ff4f2502
5 changed files with 167 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files
projects/rocprofiler-sdk/tests/counter-collection/validate.py
+10 -3
View File
@@ -23,16 +23,23 @@ def test_data_structure(input_data):
def test_counter_values(input_data):
data = input_data
agent_data = data["rocprofiler-sdk-json-tool"]["agents"]
counter_data = data["rocprofiler-sdk-json-tool"]["buffer_records"][
"counter_collection"
]
scaling_factor = 1
for itr in data["rocprofiler-sdk-json-tool"]["agents"]:
for itr in agent_data:
if itr["type"] == 2 and itr["wave_front_size"] > 0:
scaling_factor = 64 / itr["wave_front_size"]
break
for itr in data["rocprofiler-sdk-json-tool"]["buffer_records"]["counter_collection"]:
for itr in counter_data:
value = itr["counter_value"]
assert int(round(value, 0)) == int(round(1 * scaling_factor, 0)), str(data)
if int(round(value, 0)) > 0:
assert int(round(value, 0)) == int(
round(1 * scaling_factor, 0)
), f"agent_data:\n{agent_data}\n\ncounter_data:\n{counter_data}"
if __name__ == "__main__":
projects/rocprofiler-sdk/tests/lib/transpose/transpose.cpp
-23
View File
@@ -233,28 +233,5 @@ run_transpose(size_t nthreads, size_t nitr, size_t nsync)
MPI_Barrier(MPI_COMM_WORLD);
#endif
// for(int i = 0; i < ndevice; ++i)
// {
// HIP_API_CALL(hipSetDevice(i));
// HIP_API_CALL(hipDeviceSynchronize());
// }
// #if defined(USE_MPI)
// MPI_Barrier(MPI_COMM_WORLD);
// #endif
// if(rank == 0)
// {
// for(int i = 0; i < ndevice; ++i)
// {
// HIP_API_CALL(hipSetDevice(i));
// HIP_API_CALL(hipDeviceReset());
// }
// }
// #if defined(USE_MPI)
// MPI_Barrier(MPI_COMM_WORLD);
// #endif
roctxRangeStop(range_id);
}