Updated: rocm_smi.py
- Remove all else: clauses from functions where rsmi_ret_ok is part of the if clause, as requested.
- rsmi_ret_ok() function already handles unsucessful return codes and gracefully handles them.
- Updated check_runtime_status() function to sweep through /sys/class/drm to find active runtime_status.
- Updated the message to' AMD GPU device(s) is/are in a low-power state. Check power control/runtime_status'
- This clarifies the status of the GPU and tells them where to check for more info.
Signed-off-by: Juan Castillo <juan.castillo@amd.com>
Co-authored-by: Juan Castillo <juan.castillo@amd.com>
Co-authored-by: Maisam Arif <Maisam.Arif@amd.com>
Co-authored-by: gabrpham <Gabriel.Pham@amd.com>
* Enable GDA+IPC
Fix ROCSHMEM_DISABLE_IPC for both RO and GDA
* add more functionality to bootstrap class
we need a few more functions in the boostrap class to be able to fully
handle the rocshmem requirements:
- add a function to return the list of local ranks
- provide a groupAllgather operation which takes a vector of ranks
participating
- provide a groupAlltoall operation which takes a vector of ranks
participating
Also, update the functionality of the gda-Alltoall and gda-Allreduce
operations to take advantage of these functions.
* ipc_policy adapted to use bootstrap groupallgather
* bugfix: there was a mistake in computing sendto in groupallgather
* bugfix: shm_size and shm_rank were set in a local variable rather than
the class member
* mpi-bootstrap: remove an unecessary allgather
---------
Co-authored-by: Edgar Gabriel <Edgar.Gabriel@amd.com>
[ROCm/rocshmem commit: 801d2c5012]
* Enable GDA+IPC
Fix ROCSHMEM_DISABLE_IPC for both RO and GDA
* add more functionality to bootstrap class
we need a few more functions in the boostrap class to be able to fully
handle the rocshmem requirements:
- add a function to return the list of local ranks
- provide a groupAllgather operation which takes a vector of ranks
participating
- provide a groupAlltoall operation which takes a vector of ranks
participating
Also, update the functionality of the gda-Alltoall and gda-Allreduce
operations to take advantage of these functions.
* ipc_policy adapted to use bootstrap groupallgather
* bugfix: there was a mistake in computing sendto in groupallgather
* bugfix: shm_size and shm_rank were set in a local variable rather than
the class member
* mpi-bootstrap: remove an unecessary allgather
---------
Co-authored-by: Edgar Gabriel <Edgar.Gabriel@amd.com>
* Added check in Queue::sync to verify that there is a callback for every dispatch
* Removed new atomic, using get_balanced_signal_slots() atomic with initial value of NUM_SIGNALS to verify dispatches complete
* rocr: Fix Incorrect Assertion Check
The wrong variable is used in the assertion statement, should be error
checking for the value of paramEndLoc after it is modified by the call
to find().
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
* rocr: Fix Potential Undefined Behaviour
In the event that the SvmProfileControl destructor is called and
event == -1 is true then the call to close(event) is effectively
close(-1) which is undefined behaviour. This has been changed to only
call close() on valid file descriptors.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
* rocr: Add Error Check on Bytes Read
In the case that there is an incomplete read the call to copyTo() will
now return an error.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
* rocr: Fix Exception Error
Destructors are implicitly marked with noexcept being true by default
so if its not explicitly marked false in the destructor or the
functions it calls, any thrown exceptions will cause the program to
crash.
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
---------
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
Co-authored-by: Sunday Clement <Sunday.Clement@amd.com>
* Use queries instead of views in summary.py
* Export queries when created
* Remove HIP and HSA from output
* Fix domain query
* Export summary queries in the main function
* Fix comments and variable names
* Change syntax for old python versions
---------
Co-authored-by: Young Hui <young.hui@amd.com>
Legal Requirements:
For AMD software being released as open source, add copyright at the top of each new file.
Signed-off-by: Alysa Liu <Alysa.Liu@amd.com>
* Fix grid_group::group_dim to return grid_dim and not block_dim
* Add unit test for grid_group.group_dim()
* Fix unit test errors
* Skip group_dim() assertions for base_type test
* Added Fortran (amdflang) openmp tests using the openmp-vv project
---------
Signed-off-by: David Galiffi <David.Galiffi@amd.com>
Co-authored-by: David Galiffi <David.Galiffi@amd.com>
* add more derived metrics for navi4.
* addr comments
* addr comments, and add more derived counters.
* EOF.
* misc.
* remove duplicate counter.
* misc.
* Remove gfx12 architecture definition for ldslatency
* remove extra architectures for gfx12.
* use wgp for normalization
* move these changes to another PR.
---------
Co-authored-by: Venkateshwar Reddy Kandula <venkateshwar.kandula1306@gmail.com>
If a compressed changelog exists from a previous build, reconfiguring
the project fails with
```
[rocm-core configure] CMake Error at utils.cmake:213 (message):
[rocm-core configure] Failed to compress: gzip:
[rocm-core configure] /home/ben/src/TheRock/build/base/rocm-core/build/DEBIAN/changelog.Debian.gz
[rocm-core configure] already exists; not overwritten
```
Add `-f` to force overwriting.
* The QueuePair object was out of scope at the end of the for loop.
So the deconstructor was called.
* Although correct for C++ to do this, it ignores that we copied the QueuePair object into
device memory and have an instance there.
* Early deconstruction resulted in calling ibv_dereg_mr on the atomics memory region.
So when the GPU kernel tried to use the memory region it wasn't
registered which resulted in a protection domain error.
* The solution was to allocate our QueuePair obj with the new operator which leaves memory
management to us, then we can manually call the deconstructor.
[ROCm/rocshmem commit: e856fbb0eb]
* The QueuePair object was out of scope at the end of the for loop.
So the deconstructor was called.
* Although correct for C++ to do this, it ignores that we copied the QueuePair object into
device memory and have an instance there.
* Early deconstruction resulted in calling ibv_dereg_mr on the atomics memory region.
So when the GPU kernel tried to use the memory region it wasn't
registered which resulted in a protection domain error.
* The solution was to allocate our QueuePair obj with the new operator which leaves memory
management to us, then we can manually call the deconstructor.
systems-assistant[bot]
Aurelien Bouteiller
Yiltan
itrowbri
Sunday Clement
ywang103-amd
Matt Williams
Karthik Ganesan
Kapil S. Pawar
xuchen-amd
Aleksei Tumakaev
harkgill-amd
arvindcheru
Bertan Dogancay
Alysa Liu
Danylo Lytovchenko
vstojilj
Saeed, Oosman
Dmitrii
Venkateshwar Reddy Kandula
ycui1984
Julian Jose
ammallya
amd-hsivasun
marandje
Kian Cossettini
habajpai-amd
abchoudh-amd
Marius Brehler