Commit Graph

106 Commits

Author SHA1 Message Date
Chris Freehill e202097fb0 Don't use hwmon# as indicator of gpu
Previously, during the rsmi_init discovery process, the existence
of an hwmon# directory was used to distinguish between gpus nodes
and non-gpu nodes. This isn't reliable in some scenarios. Instead,
the existence of the vbios_version file is used as an
indicator that the node is indeed a gpu.

Change-Id: Icfbe5c42ed0970077b05f25c3d209308a31bec85


[ROCm/rocm_smi_lib commit: ff9546aa62]
2021-01-29 13:05:10 -05:00
Ori Messinger 12fd0f8c40 ROCm SMI Python CLI & LIB: Add GPU Reset Functionality
The purpose of this patch is to implement GPU reset functionality
in the LIB, and to call it from the rocm_smi python CLI.

Signed-off-by: Ori Messinger <Ori.Messinger@amd.com>
Change-Id: Iaf525f7016f8354a7fd93af0209ca2e97ef4fd56


[ROCm/rocm_smi_lib commit: 80f629b9be]
2021-01-26 17:52:24 -05:00
Chris Freehill d71ba4c666 Remove adding of bogus hwmon label entries
If we fail to find an expected temperature or voltage label
file, previously we were attempting to re-add a mapping of file
index to sensor types. Attempting to insert a map item that is already
present has no effect, so there should be no functional change.

This was a remnant of old code that should have been deleted.

Change-Id: Ie6f8a62f619a1ae58756e0fd891532434518cf78


[ROCm/rocm_smi_lib commit: bb5132a66c]
2021-01-06 11:01:07 -05:00
Chris Freehill 185ebc2f07 Introduce RSMI_DEBUG_INFINITE_LOOP
The environment variable RSMI_DEBUG_INFINITE_LOOP is introduced
to facilitate debugging RSMI in user applications. When this
env. variable is non-zero, an infinite loop will be entered in
rsmi_init(). At this point, a debugger can be attached and RSMI
can be debugger. This only applies to debug builds.

Change-Id: I23f6dd730fc965764295070de053314a1cc5b6aa


[ROCm/rocm_smi_lib commit: 68095b50e7]
2021-01-06 10:30:24 -05:00
Divya Shikre c022ddd343 Fix for error while reading gpu_metrics sysfs file
Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: If69b7eeb3573ebece9ed0cb539f5ddffbe3c2f09


[ROCm/rocm_smi_lib commit: efd234c9e3]
2020-12-18 15:31:16 -05:00
Divya Shikre 9f37923767 Fix for inconsistent GPU indexing between rocm-smi and rbt/hip.
Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: I0d966c91bfe0f0d51859ff098d15011a3e4e8b29


[ROCm/rocm_smi_lib commit: 47ca37aef7]
2020-12-11 15:11:21 -05:00
Chris Freehill df789612b6 Show more info in stderr when rsmi_init() fails
Some rsmi apps fail without much explanation when
rsmi_init() fails. This patch hopes to provide some clues to
the reason for the failure.

Change-Id: Id51308dc327b9871d537dd3e709b677db4ef10bc


[ROCm/rocm_smi_lib commit: 6377e0258d]
2020-12-10 07:32:03 -05:00
Chris Freehill aef625bfd3 Fix process killed while holding mutex
Previously, when a process holding a shared mutex was killed,
the next time an RSMI application was started, it would not be
able to obtain the mutex--the application would have to exit.
This fix uses pthread_mutexattr_setrobust() to detect this
situation and act accordingingly.

Also, add some missing, needed mutexes and move mutexes
closer to where the protect resource is used.

Change-Id: Icfdc3a246f4cfa3fd008e3f13472199abd76fd35


[ROCm/rocm_smi_lib commit: f4938b0ac9]
2020-12-04 12:59:55 -05:00
Divya Shikre 6d4fb11c6e Adding Performance Determinism Mode to rocm_smi lib, CLI & gtest.
A special mode of operation to achieve minimal performance variation by letting
the user have the ability to provide the desired frequency to be set as the soft limit.
The user can control the entry and exit to the mode via rocm-smi a mechanism to
enter / exit performance determinism mode as below.

Enter performance determinism mode:
- hold a lock
- write performance_determinism to power_dpm_force_performance_level
- write input clk_freq to pp_dpm_sclk
- release lock

Exit performance determinism_mode:
- hold a lock
- write auto to power_dpm_force_performance_level
- release lock

Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: Ia24e27954cdf1c4337ffc83d8948fbdfaf4552d2


[ROCm/rocm_smi_lib commit: 60d0f3052f]
2020-12-02 11:11:00 -05:00
Chris Freehill 6fc9f802ae Quiet address sanitizer warnings
Also,
* Fix some doxygen issues
* Fix address sanitizer issues in rsmitst

Change-Id: Ie6c6fd9af5c418210b7064e79650fb92cd4a5e2b


[ROCm/rocm_smi_lib commit: 63064b0000]
2020-11-10 14:16:39 -06:00
Ramesh Errabolu 8b53e7812f Update ROCm SMI library with ability to read CU occupancy
Change-Id: Ib9882fa2d81c13604af282279bfa116bc2fd05a4


[ROCm/rocm_smi_lib commit: 328878343c]
2020-10-14 09:33:37 -04:00
Chris Freehill c2acc451af Revert "Revert "Support for RSMI_EVNT_GRP_XGMI_DATA_OUT counters""
This reverts commit c009809fcd.



Change-Id: Ic412a64d35aab74caf12bf4c791f0a66ac15b061


[ROCm/rocm_smi_lib commit: 5465d872aa]
2020-10-08 10:36:30 -04:00
Kent Russell f5015e6cb4 Remove extraneous mutexes
We already grab the mutex before getting the device name, so we don't
need to grab it again

Change-Id: Ib627ba3a39c485f6069af052cfd3e6c522873d43


[ROCm/rocm_smi_lib commit: e350278b68]
2020-10-08 07:55:07 -04:00
Chris Freehill c009809fcd Revert "Support for RSMI_EVNT_GRP_XGMI_DATA_OUT counters"
This reverts commit 8acd845e5b.

Temporarily reverting until the driver side of this is upstream

Change-Id: I2d8243208c1271ebad90bc2ee0fda2dfefb0831b


[ROCm/rocm_smi_lib commit: ae6d3fbdd0]
2020-10-07 18:42:56 -04:00
Kent Russell e432b4e9e3 Check FRU-based product information if available
WKS and server cards have an FRU with product information, so try to use
that for product name and product SKU if it exists.

Signed-off-by: Kent Russell <kent.russell@amd.com>
Change-Id: I40bbd3bf62f4cb02e96015ed1630112691cacbc3


[ROCm/rocm_smi_lib commit: df7c3434cd]
2020-10-07 14:09:23 -04:00
Chris Freehill 624f906f07 Fail gracefully if drm directory is not found
Change-Id: I0f3ab2721108355752caf0280124469b98af4967


[ROCm/rocm_smi_lib commit: c6f02b4d62]
2020-10-05 21:12:11 -04:00
Chris Freehill 8acd845e5b Support for RSMI_EVNT_GRP_XGMI_DATA_OUT counters
Also some format fixes

Change-Id: Id3c0f6b3cf5b327bb9ca6acb6091dc67764c8032


[ROCm/rocm_smi_lib commit: 946bf93dfb]
2020-10-05 17:22:19 -05:00
Divya Shikre 94fc1524c3 Adding functionality that will parse gpu_metrics sysfs file
Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: I3a84870b83eb4cd0ed46f10bb19169c91f99fd8e


[ROCm/rocm_smi_lib commit: 8b48564ce3]
2020-10-02 10:25:41 -04:00
Mukul Joshi 602f182344 Use correct string conversion function for VRAM and SDMA usage
VRAM and SDMA usage can be 64-bit long numbers. Use stoull()
instead of stoi() to convert the VRAM and SDMA usage strings to
numbers.

Change-Id: Ifadbada9f33320fc67666036ce8439823c1d1fb7


[ROCm/rocm_smi_lib commit: fb2ed24372]
2020-09-21 12:28:22 -04:00
Mukul Joshi 8082416569 Update KFD SMI event notification handling
Event bitmask in KFD SMI event is now replaced with event index in
the SMI event message. Sending a event bitmask, which was a 64-bit
field with only 1 bit set, was quite wasteful of memory and also
potentially limiting to 64 events. Instead the kernel would send
event index in the SMI event message. As a result, update the
KFD SMI event handling to expect the event index in the message.

Change-Id: I3e74620788d3c1f7c0bdaa69e9d9ab3d1aba2c92


[ROCm/rocm_smi_lib commit: 406859ca8a]
2020-09-16 13:24:50 -04:00
Chris Freehill fb7952f401 Make sure all sensor labels have valid mappings
There may not be label files for some sensors on older
devices. We need to make sure there is a valid dummy
mapping in these cases.

Change-Id: Id6a8b71e554552be84a0e42a477070b504151e7f


[ROCm/rocm_smi_lib commit: b015052a07]
2020-09-11 17:32:54 -05:00
Divya Shikre 31f3b6d33d Adding setsrange, setmrange, setvc, setslevel and setmlevel functionality to rocm lib and cli
Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: I5fd65ea7bcd5403aaf2e42d2aa28d837929da253


[ROCm/rocm_smi_lib commit: 54d4b9d500]
2020-09-08 18:42:39 -04:00
Chris Freehill b662e7ce51 Correct usage of bitwise &
Also, fix warning related to catch() and cpplint error.

Change-Id: I4292170538d0f700fccb605814c5058543abe74a


[ROCm/rocm_smi_lib commit: c2439d28e8]
2020-07-26 20:08:24 -05:00
Chris Freehill 5c2ac56166 Update xgmi event counter documentation
Also:
* fix doxygen manual generation that was altered during
  OAM refactor
* quiet some compile warnings.

Change-Id: I548a3cf00eb887bea3dbf58e362ca6dfe90bde28


[ROCm/rocm_smi_lib commit: 52514835f0]
2020-07-16 17:42:56 -05:00
Mukul Joshi fdda24038f Add support to retrieve process SDMA usage information.
Also, print SDMA usage information in TestProcInfoRead.

Change-Id: I8d19be3b8653e298c81237e5067eca75a1743e70


[ROCm/rocm_smi_lib commit: eea1ed8c3d]
2020-07-13 17:32:08 -04:00
Chris Freehill 202434abad Handle un-readable kfd properties files
Some systems have kfd sysfs properties entries that
are unreadable--for example, when a multi-gpu system is
dividing the gpus among containers, each container may
only be able to access certain gpus.

Previously, all kfd topology node properties entries were
assumed to be valid. Now, we check for readability before
declaring them "valid".

Fixes SWDEV-240169

Also:
* remove an assertion that would happen when no pcie
device identifier files are found on the system.
* fix cpplint issues

Change-Id: I74321b685159dd2628c890b33c39ad82988cb9dd


[ROCm/rocm_smi_lib commit: 68155baed5]
2020-07-10 12:35:31 -04:00
Chris Freehill 001aa0b825 Fix docs + cmake_utils path issues
This corrects issues that arose after OAM reorganization.
It should address SWDEV-243294.

Also, fix some compile warnings that show up on RHEL.

Change-Id: Id14d444905da35cd7346bcfbcd82b6d0572708c4


[ROCm/rocm_smi_lib commit: c2ef9a6879]
2020-07-08 09:47:25 -05:00
Chris Freehill 77083980a8 Quiet spurious pthread_unlock warnings
A message is output in debug builds when pthread_unlock
returns an error. However, in most cases, it should return
EPERM. In fact, if it doesn't return EPERM, it is an
indication of a problem. This commit adjusts accordingly.

Change-Id: Ia5cad89aa6e68e79c1291ea21adffb0fa68f2300


[ROCm/rocm_smi_lib commit: 866438966d]
2020-06-30 15:12:58 -05:00
Chris Freehill 98b976ef3e Refactor rsmi to support oam
Change-Id: Idc524e01ba06eb5c8d1682becaf5bf8ced5bffcf


[ROCm/rocm_smi_lib commit: 6594f8f58b]
2020-06-22 18:51:46 -05:00
Chris Freehill 4c94842508 Ensure no device mutexes are left held on shut_down
Also, fix TestMutualExclusion and TestEvtNofifReadWrite.
Previously, some of the normal SetUp function was not
being done for this test. In some cases, no DRM
devices are being found on the test machine. Skip
those.

Change-Id: Iaa5a257841eb459aa57491ae9680c34a60d5ac2b


[ROCm/rocm_smi_lib commit: 59394f3354]
2020-06-19 13:59:20 -05:00
Mike Li b3bb190b8d Add support to retrieve XGMI hive id
Change-Id: I1eee05dd85ecb856889d1cfe0565454d2f538856
Signed-off-by: Mike Li <Tianxinmike.Li@amd.com>


[ROCm/rocm_smi_lib commit: 488bbb668a]
2020-06-19 07:35:23 -07:00
Divya Shikre 82825b40d3 Adding current voltage feature & gtest.
Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: Ic555a3af265e603419e2875d1989a366abc82596


[ROCm/rocm_smi_lib commit: 2805ed16a4]
2020-06-16 11:48:56 -04:00
Chris Freehill b07fd8fca7 Update XGMI perf counter test to show utilization
Also:
* When destroying a counter, make sure to stop the counter first
* In the test, do not stop (disable) the counter before
  reading it.
* Clean up some whitespace in other tests
* Re-add manual pdf file

Change-Id: I0786ef3a994ca568299c77e44f092af8943ac33d


[ROCm/rocm_smi_lib commit: f946ea37ef]
2020-06-10 12:49:49 -04:00
Chris Freehill 430e52c394 Check only the minor ioctl version for event support
Change-Id: I70ddab4298a62178b2509a0365ee4cd6937302c1


[ROCm/rocm_smi_lib commit: 42c10633b6]
2020-05-27 09:27:01 -04:00
Mukul Joshi 506f71a01c Add support to retrieve process VRAM usage information.
Change-Id: I60843a99207a658022a26aa346b79f91863833cf


[ROCm/rocm_smi_lib commit: e30ebbc787]
2020-05-26 15:19:24 -04:00
Chris Freehill 11b90a242a Return an error instead of assert when reading bad data
Assert doesn't help with release builds.

Change-Id: Ib076791fd442e96c7544914cdf08774fc7a40a94


[ROCm/rocm_smi_lib commit: 754a993d32]
2020-05-19 15:58:40 -04:00
Mike Li f7885be06e Add functions that are used to query Hardware topology.
Change-Id: I0f4cd02b237bde4d6dccfb0e83e65376ecb1cfaa
Signed-off-by: Mike Li <Tianxinmike.Li@amd.com>


[ROCm/rocm_smi_lib commit: c7d349183a]
2020-05-18 12:37:27 -04:00
Chris Freehill d4dda0017c Don't use static variable for monitors
Change-Id: I24b5ccfa94b2d722b070a6c6385af9201d21d9c5


[ROCm/rocm_smi_lib commit: 02e4a9c14f]
2020-05-15 08:05:06 -04:00
Chris Freehill 2862d311da Catch and handle regex exception
When pattern matching file names to determine API support, in
some environments std::regex will throw. This change is meant
to handle this more gracefully.

Change-Id: If1ccfe5bdd71ec4d08663c80692024488072e11b


[ROCm/rocm_smi_lib commit: 27148a02cb]
2020-05-14 15:39:40 -04:00
Chris Freehill 0ab5e76b33 Add ref counting for rsmi init and shutdown
Also, clean lint from kfd_ioctl.h file.

Change-Id: I5a2ae127ab6ab6676a1b075ed10858d0ebfe13c1


[ROCm/rocm_smi_lib commit: 8e03d10035]
2020-05-11 15:57:42 -05:00
Chris Freehill 1959ccb265 Add event notification API
Change-Id: Ib6e8efbe6cdefaa7de1f74bd26993e9b4b011649


[ROCm/rocm_smi_lib commit: 2235ede34c]
2020-05-06 14:07:25 -05:00
Chris Freehill fee5adb228 Handle rsmi app running on machine with no AMD gpus
This fixes a seg fault that would happen in release builds when
there are no KFD nodes on a system, which occurs when there are
not AMD gpus present in the system. This use case occurs
for higher application code that is meant to be gpu agnostic.

Change-Id: If374930bc2e62f9898f337349cde3ebb16091ff0


[ROCm/rocm_smi_lib commit: 806f665a85]
2020-04-28 00:35:16 -04:00
Chris Freehill 01401b0caa Add device mutual exclusion tests and related fixes
* Added a new test to verify mutual exclusion of access to device
  resources
* Added some missing acquiring of mutexes to some RSMI calls, as
  well as try-catch blocks.

Change-Id: I87aac009878a0b2d1f975e1d5b794d887bb23ff9


[ROCm/rocm_smi_lib commit: f8b57c3b16]
2020-04-08 15:05:11 -05:00
Chris Freehill 49b562b209 Shared mutex fixes and improvements
* Don't make different shared memory mutexes for different users
* Don't delete (unlink) the shared mutex file if the mutex
  initialization fails. This may mess up other processes that
  are using it. Instead, print a message on how to resolve the
  situation, and then throw an error.

  Note, this situation comes up when debug builds (usually)
  either assert() or otherwise end execution without a proper
  clean up.
* Remove cpplint from shared_mutex code

Change-Id: I5f8ca6150cac5c2405fb97007516da345093f966


[ROCm/rocm_smi_lib commit: 52196caaee]
2020-04-06 17:08:33 -05:00
Mukul Joshi 7137023637 Add rsmi_topo_get_numa_affinity()
Given a device index, return the corresponding NUMA node for the
device.
Also, add NUMA node tests to Sys Info Read test.

Change-Id: I0df4937470e6362e6737ccea568d4b3e5890c91a


[ROCm/rocm_smi_lib commit: fd79e5c161]
2020-04-01 11:38:08 -04:00
Chris Freehill 17871ecb14 More general solution to api support hwmon mapping
This solution takes into account that some hwmons use
label files to map sensor types. The previous solution
did not take this into account.

Change-Id: I1d6204573cefa8197b2cfe0ffb412b545df3d80a


[ROCm/rocm_smi_lib commit: 324c0ca0e5]
2020-03-16 11:37:47 -05:00
Chris Freehill 4e2d769dcc Fix indexing problem with api support function
Also fix potential issue with evaluating functionality of
functions with multiple sub-variants.

Change-Id: I9a09e52f3d3f3181e72578ed1f3bfd0d85516aa3


[ROCm/rocm_smi_lib commit: 1d8e16bff2]
2020-03-12 11:43:01 -05:00
Chris Freehill a7ca81d161 Don't assert or re-throw exception caught at top level
Instead, return error and let caller deal with it.

Change-Id: I1a55337134b00aa4259af27281b2450fc2252be9


[ROCm/rocm_smi_lib commit: d54a9484be]
2020-03-11 12:11:29 -05:00
Chris Freehill 95d3da04b9 Add rsmi_compute_process_gpus_get()
Given a process ID, give the device indices that process is
currently using.

Also:
* made corrections to how RSMI, amdgpu (ie, "card#") and
  KFD indicies translate from one another
* add a few missing error codes to rsmi_status_string()
* fix some formatting

Change-Id: Icd2cae66bb4fec768da96af7cf9cf8b8b66ec7f9


[ROCm/rocm_smi_lib commit: 2d6e15190c]
2020-02-22 10:47:58 -06:00
Chris Freehill 61db4c7e15 Ensure string is non-empty before calling stoul or stoi
Change-Id: I2c6314fb86d3bba8fd6aab932dbb989263fa8542


[ROCm/rocm_smi_lib commit: f748868818]
2020-01-28 17:05:14 -06:00