Commit Graph

8920 Commits

Author SHA1 Message Date
Jonathan R. Madsen 39cf760a4e Rename hosttrace to omnitrace (#18) 2021-11-24 04:59:59 -06:00
Divya Shikre f61cb1b41d Add fix for out of range temperature value for HBM.
Driver mem fills in 0xFF for all for the metrices not supported for that ASIC.
So if 0xFF is detected, return RSMI_STATUS_NOT_SUPPORTED

Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: Iacb6474486e3732f2aa824ff447c17f8243b65cd
2021-11-23 15:37:41 -05:00
Jonathan R. Madsen 752424efc2 Reorganization and critical trace support (#17)
* Roctracer wall clock integration (#16)

* Integrates roctracer values into wall-clock

* Fixed scoping + timemory roctracer

* Fixed data race in roctracer

* Synchronized HIP API on main thread

- Cache hip activity callbacks and execute on main thread
- Minor updates to transpose

* Debugging + MPI + transpose updates

* PTL + HSA and timemory + kernel timing

- PTL usage fixed HSA + timemory issues bc we could control the thread destruction
- Fixed laps counting in roctracer callbacks

* Ignore select HIP API types

- The ignored API types are ignored because there appears to be a bug
  which causes the "end" callback to be labeled as begin
- hipDeviceEnablePeerAccess
- hipImportExternalMemory
- hipDestroyExternalMemory

* Tweaks to PTL config

* Timemory update + pid-prefix w/ mpi headers

- %pid%- prefix with mpi headers
- timemory submodule update

* CMake + critical trace + reorganize library source

- clang-tidy tweaks
- cmake function updates to use hosttrace_ prefix
- update gitignore
- cmake HOSTTRACE_MAX_THREADS option
- Formatting.cmake
- cleaned up MacroUtilities.cmake
- PTL submodule + usage
- tweak to Findroctracer.cmake
- MT transpose
- Updated PTL submodule
- Updated timemory submodule
- fix to hosttrace return value type if type not found
- reorganized library source code
- support for critical trace

* Remove bits/stdint-uintn.h headers

* Rename + config + depth + critical path

- rename hosttrace_timemory_data to instrumentation_bundles
- rename hosttrace_bundle_t to main_bundle_t
- rename bundle_t to instrumentation_bundle_t
- rework of configuration setup
- critical_trace write directly to file option
- tweaked depth calculation
- updated timemory submodule
- improved parallel support in roctracer callbacks
- working critical_trace
- perfetto device-critical-trace and host-critical-trace categories
- made transpose example parallel
- made parallel-overhead example a bit uneven
- relocated LTO activation

* Fixed duplicates in perfetto critical-trace

* reworked critical trace support

- substantial perf improvement (30-45 min -> 30 sec)
- changes to configuration (new and removed options)

* Removed "%pid%-" output prefix in mpi_gotcha

* Update timemory submodule
2021-11-23 02:53:14 -06:00
Jeremy Newton 7649cd862e Fix packaging of license file
CPACK doesn't have proper logic for installing the license as described
by CPACK_RESOURCE_FILE_LICENSE.

For Debian packaging, the license is expected to be installed as:

/usr/share/doc/PACKAGENAME/copyright

To do this, I've added a bit of logic for CPACK to copy this into the
package using CPACK_INSTALL_COMMANDS to prep the directory, and
CPACK_INSTALLED_DIRECTORIES to add it to the package. This applies to
both RPM and DEB, so I've added some logic to the spec file to exclude
this file (note that CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION does
not work for files installed with CPACK_INSTALLED_DIRECTORIES).

For RPM install, I've just added a small bit of logic to the spec file
to handle it. The file needs to be copied into the spec working
directory, then a macro is used to handle the rest. Note the license
macro does not work on EL6, but I don't think we want to support this.

Change-Id: I06ce63d300419893cb8274bc504a15633e304d91
Signed-off-by: Jeremy Newton <Jeremy.Newton@amd.com>
2021-11-18 16:41:48 -05:00
Jeremy Newton 529c96c08b Fix to previous commit
I used the binary directory instead of the source directory to specify
the spec.in path, which passed local testing since these directories
are in the same location. This is not guarenteed to be true.

Change-Id: I1b49ca8453b9c074a947104c26fb39667d728a8e
Signed-off-by: Jeremy Newton <Jeremy.Newton@amd.com>
2021-11-17 17:47:53 -05:00
Sreekant Somasekharan 3f27dcc1ac Modify bool variable to true in if condition of src=dst
Change-Id: Ie2024b3a6ad68e48384bb3472fe8785bcd643665
2021-11-17 12:53:40 -05:00
Jeremy Newton 3c0e4fee0f Implement RPM recommends for libdrm-amdgpu
CPack does not support recommends for RPM generation, so I've generated
a template RPM SPEC files in order to make modifications to allow for
support of recommends.

The spec.in file was generated using the cpack option
"CPACK_RPM_GENERATE_USER_BINARY_SPECFILE_TEMPLATE" and was modified very
sparingly to avoid any maintanance burden, e.g. can be easily
regenerated. The CPACK_RPM_USER_BINARY_SPECFILE is then used to specify
the customized template file, instead of using the cmake's template.

From what I understand, the point of these two options is to allow
developers to tailor the specfile to their desire, since rpm spec files
are much more advanced then the equilivent debian file.

Change-Id: I80c69be58a3c57729ed997fd2ce01f5d16b9e9b9
Signed-off-by: Jeremy Newton <Jeremy.Newton@amd.com>
2021-11-17 11:57:55 -05:00
Jeremy Newton 86c27a7af8 Use recommends for libdrm-amdgpu-amdgpu1
For the use of libdrm-amdgpu-amdgpu1 and libdrm-amdgpu, we should use
recommends, as we want these packages installed with a strong dependency
but avoid a strict dependency, since this is enhancement feature.

Using the newer libdrm, which is build for amdgpu-dkms, is ideal since
it will produce more correct marketing names, but should not be mandated
due to two reasons:
- A user may not want to install both libdrms on their system
- The system might not have the newer libdrm available

This patch only fixes the Ubuntu/debian package since recommends is not
properly implemented for the RPM generator for CPACK. For now,
"suggests" will have to do, since it's the closest option we have. I
will investigate if we can get around this issue.

Change-Id: I33a90c3ead235bbbe265238c026933688ea63fe3
Signed-off-by: Jeremy Newton <Jeremy.Newton@amd.com>
2021-11-16 11:08:40 -05:00
Icarus Sparry 39a11d7dc4 Merge "Allow creation of just deb or rpm" into amd-staging 2021-11-15 23:03:57 -05:00
Sean Keely fc75731034 Add comments to GetPcieBlit.
Comments call out the specific operation being selected since the
ternary nest is a bit hard to read.

Change-Id: If033dbaa6cba132e96196ad3fc6d5572042041f4
2021-11-15 19:34:03 -06:00
Sean Keely b198016949 Fix leak in hsa_amd_interop_map_buffer.
Agent temp array could have leaked if one of the given agent
handles was invalid.

Change-Id: I9e638b3a4f6bb917a4e3209ad81a1253bb603365
2021-11-15 19:22:20 -06:00
Sean Keely f48a786662 Correct order of argument check and default assignment in lock APIs.
Argument must be checked for nullptr before being dereferenced and
filled with the default return value.

Change-Id: I9ff366f066a5e18c78129bf59cc3ba00fca3ef18
2021-11-15 19:22:02 -06:00
Sean Keely 322588a60e Add missing return in ScopeGuard::operator=.
This omission did not cause problems earlier due to having not been
instanced.

Change-Id: I7a54f82e06c299902f3bf6b4d3737cc5e30961ad
2021-11-15 18:50:46 -06:00
Icarus Sparry f4c7309592 Allow creation of just deb or rpm
Remove hard code of generating both deb and rpm, allow the user to
specify what is desired and cache that choice.

Create executable with owner write permission to work around binutils
bug.

Change-Id: I67655e5d351b227d1a8db4645228300d2bb83f9a
Signed-off-by: Icarus Sparry <icarus.sparry@amd.com>
2021-11-16 00:14:14 +00:00
Kent Russell e842d7f480 Revert "CMakeLists: Fix libdrm-amdgpu dependencies"
This reverts commit 9529bf7063.

Reason for revert: Infra still not ready for it yet

Change-Id: I03e043c1ca7924264e3e70e3e82c73b4efc2ae75
2021-11-12 14:30:04 -05:00
kiumars 7a8f5b5e90 SWDEV-303820: [LLNLA-63]rocprof does not check input arguments at start
Change-Id: I820a2b6df10b799f776518a98519e9c8ca40ec53
2021-11-11 17:31:56 -05:00
Sean Keely 19454fcf26 Correct node id assertion in pointer info.
Size of the node map was used as the max node id previously.  This
is wrong when RVD is used.

Change-Id: Ic632ec96891b92186e5b68cd53f81414db34f59f
2021-11-10 22:09:24 -06:00
Sean Keely c9eb85e205 Correct size of SVM node array.
Was size of the map.  Needs to be size of the node id range.

Change-Id: I92501ea7adca5c30dbb0fdabd2c421dea58f8d6f
2021-11-10 21:23:42 -06:00
Sean Keely d65e00bcc5 Include event_id in SDMA interrupt payload.
The event id assists KFD in locating the proper event associated
with the interrupt.

Change-Id: I75d58b6be74dd5b1edb0c5fe2b9d01538a649ba1
2021-11-10 20:57:11 -06:00
Kent Russell 9529bf7063 CMakeLists: Fix libdrm-amdgpu dependencies
SUGGESTS doesn't get installed by default by dpkg, it requires
--install-suggests. While the libdrm-amdgpu-amdgpu1 lib might not be
critical to core thunk usage, it's necessary enough to increase the
dependency from weak to strong, thus we use RECOMMENDS instead

Since CPack doesn't support RECOMMENDS for RPMs, and SUGGESTS isn't
strong enough, we just set it as REQUIRED. This should handle any
standard installation situations instead of trying to be fancy to try to
make things exactly the way we want them to be.

Signed-off-by: Kent Russell <kent.russell@amd.com>
Change-Id: I5a676fb4a3fbf5c7b35f503acc30cf01c056266d
2021-11-10 17:23:31 -05:00
Jeremy Newton 48e4e2c5ff Set License field for RPM package
This really should be set to conform to distro standards.

Signed-off-by: Jeremy Newton <Jeremy.Newton@amd.com>
Change-Id: I8c3bdcc7eb103cec9db6aa9f9cfec25754784be8
2021-11-10 14:06:17 -05:00
Jeremy Newton fa2309fcdd Set License field for RPM package
This really should be set to conform to distro standards.

Change-Id: I63442b9ad58f70c4691c499b666f7c359dc9609d
Signed-off-by: Jeremy Newton <Jeremy.Newton@amd.com>
2021-11-10 13:56:33 -05:00
Ori Messinger 40eed25a3b ROCm SMI CLI: Fix printErrLog Arguments
This patch removes every erroneous occurance of a third argument
when calling printErrLog(device, err), since it takes two arguments.

Signed-off-by: Ori Messinger <Ori.Messinger@amd.com>
Change-Id: I5971cc68b69c86f37c69f44e4785dabfc82c7955
2021-11-08 12:54:00 -05:00
Aaron Liu f2a50c34f9 Fix compiling error with gcc-10.3.0
On gcc-10.3.0 environment, hsa-runtime building is failed as below log:
compute/hsa/runtime/rocrtst/suites/negative/queue_validation.cc:470:18: error: conversion from ‘unsigned int’ to ‘uint16_t’ {aka ‘short unsigned int’} changes value from ‘4294967295’ to ‘65535’ [-Werror=overflow]
  470 |     aql().header |=  0xFFFFFFFF << HSA_PACKET_HEADER_TYPE;
      |     ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus: all warnings being treated as errors
make[2]: *** [CMakeFiles/rocrtst64.dir/build.make:339: CMakeFiles/rocrtst64.dir/home/aaliu/work/compute/hsa/runtime/rocrtst/suites/negative/queue_validation.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Change-Id: I95fe72030368abc211b4b97b5a7ba00b5e094730
2021-11-04 10:55:11 +08:00
Kent Russell 7e90eef254 CMakeLists: Refer to libdrm-amdgpu1 for RPM distros
The infra is in place to make sure that we can get around the issue
where older versions of rpm don't know what "Suggests" is. Using the
HSAKMT_REQUIRES_LIBDRM variable will force the package to require
libdrm-amdgpu1 instead of just suggesting it in newer distros.

Change-Id: I5d595c4e4be5ad8e8c411dcec3a58365e1984f05
2021-11-03 13:44:48 -04:00
Kent Russell 98baeca615 Merge amd-staging into amd-master
Conflicts:
	python_smi_tools/rocm_smi.py

Change-Id: Iad29365d4dd0ac30d19013dae31105343b126733
2021-11-01 11:47:00 -04:00
Icarus Sparry 8ea268b3f4 Forcibly disable parallelism
The manually written Makefiles in the test directory are not safe to
use by more than one job. For example we see things like
	all: clean $(EXECUTABLE)
which says that the 'all' target depends on the 'clean' and
'$(EXECUTABLE)' targets. If make is invoked with -j2 then the clean
and '${EXECUTABLE}' can be built in parallel, so the clean can delete
things whilst they are being built!

Change-Id: I9c56db4c629081b8d812dad45dfd4afde10e481f
Signed-off-by: Icarus Sparry <icarus.sparry@amd.com>
2021-10-31 15:19:36 -07:00
Sean Keely b83efe00a4 Update README.
Document use of sysfs for gfx arch lookup.

Change-Id: Ic326769be3e23d627e16fb1aae56c0d16ce99d78
2021-10-29 20:24:11 -05:00
Sean Keely 7766360dd6 Close /dev/kfd after access check.
Cleanup dangling file handle.

Change-Id: I37bc42a2ffb7122f441cbf8a09c141d4396db928
2021-10-29 20:23:32 -05:00
Sean Keely b57c02d131 Handle high levels of concurrency in rocm_agent_enumerator.
rocm_agent_enumerator may invoke rocminfo.  Rocminfo opens the
GPU device which allocates limited resource.  Beyond 254
concurrent processes this resource will be exhausted and rocminfo
will return an error.

This patch loops rocm_agent_enumerator when recieving a failure
message from rocminfo indicating KFD is out of memory.

Change-Id: I8637e214f5fa012642975c28578ae6bf9200eda8
2021-10-29 20:22:19 -05:00
Joseph Greathouse f419b81bdf Add capability to pull gfx architecture from KFD topology
New versions of amdkfd include the gfx architecture version number
for all GPUs surfaced in the HSA topology. This patch adds this as
the preferred way for rocm_agent_enumerator to check for supported
gfx architecture numbers.

Kernels that are missing this feature will not have the value in
the topology. rocm_agent_enumerator will fall back to checking
against the PCI IDs in this case. If PCI IDs fail, we fall back
to the heavyweight rocminfo method.

Change-Id: I5cf22e1069114675092e97ae52331b829cfafb04
2021-10-29 20:20:04 -05:00
Joseph Greathouse e9b7de43be Switch order of lspci and rocminfo for gfx arch query
rocminfo is a very heavyweight mechanism for learning a lot of
information about the GPUs that are attached to the system.
It opens up the limited /dev/kfd resource to gather lots of
information about each device, while rocm_agent_enumerator really
only wants the gfx number of AMD devices attached to the system.

To avoid this heavyweight lookup in most cases, this patch switches
the order of tests. Rather than starting with rocminfo and then
falling back to a poorly-maintained PCI ID list, this patch changes
the agent enumerator to start by checking in the PCI ID list (fast
case) and then falling back to rocminfo (slow case) if the PCI ID
list is out of date.

Change-Id: If24b8bc3baeeb6adad362abbb288ef3728383bce
2021-10-29 20:16:22 -05:00
Joseph Greathouse 0e1807c72d Update cmake to include pciutils dependencies
When building packages, add in pciutils as a dependency because
rocm_agent_enumerator uses this as a mechanism for looking up
what GPUs exist on the system.

Change-Id: I10ac088c461c6d0bca435b61fbc90b685556fdf4
2021-10-29 18:45:32 -05:00
Joseph Greathouse ea5ce46fb4 Add better PCI ID backup in rocm_agent_enumerator
The PCI ID backup method in rocm_agent_enumerator, where the
tool uses lspci to find all AMD GPU devices in the system and
manaully match them to gfx version, is extremely outdated. The
PCI ID list did not include anything after Vega 10, and the
actual call to lspci no longer returned anything due to some
missing conversions.

The patch adds all GPUs that might be needed by ROCr up through
Navy Flounder. The PCI ID to gfx matching pulls from the amdgpu
driver and libhsakmt.

Change-Id: I58b77bb6aa631f575352fc444d2542f265909706
2021-10-29 18:45:23 -05:00
Sean Keely 2b4b0c8862 Add dependency on kmod.
rocminfo uses lsmod to check for loaded drivers.  Use of sysfs is
possible but sysfs' stable interface does not allow for easy parsing.
Use of lsmod avoids needing to walk the sysfs tree and avoids issues
of permissions to examine sysfs.

Both Debian and Fedora list lsmod under kmod.  Presumption is that
CentOS and RHEL also follow this.

Change-Id: Ic5033e0b780100c54d2fe0b4f501c40acbc237fb
2021-10-29 16:23:01 -05:00
Sean Keely 86560046de Correct and simplify package dependencies.
Package should depend on hsa-rocr, not hsa-rocr-dev.  Also
Remove negative path for dependency rocm-core (unconditional add +
conditional remove simplified to conditional add).

Change-Id: I6ead202f4f3f2f77b1da2ffed77feee956caf2e9
2021-10-29 16:23:01 -05:00
Sean Keely e32a4d9bed Don't touch CMAKE_CXX_FLAGS.
Cmake manages CMAKE_CXX_FLAGS in standard ways from user inputs,
env vars, and target properties.  Use target properties for our
options and let cmake handle the rest.

Change-Id: I20ef738e4df7880258d770f4ec13c09d8b323fba
2021-10-29 16:23:01 -05:00
Yi Qian e1df81cb8f Add a new rocm_agent_enumerator option '-name' which lists target names obtained from the output of rocminfo.
Change-Id: Iea366c744e2f03a46fd5c902ae4faa94b96f6942
2021-10-29 16:23:01 -05:00
Ashutosh Misra 2bd7657503 Adding package dependency
SWDEV-260294:Rocminfo shall explicitly depend on hsa-rocr for package installation

Signed-off-by: Ashutosh Mishra <ashutosh.mishra@amd.com>
Change-Id: I9a6d1691cd5a821f135a1d1cff599e9a2c40d06f
2021-10-29 16:23:01 -05:00
Chris Freehill 102f22f974 Append CXXFLAGS env. variable to CMAKE_CXX_FLAGS
This facilitates address sanitizer builds via env. variable.

Change-Id: I69fc9a5782ca80d7bdbf2b1ec0c89cccc4a5ca11
2021-10-29 16:23:01 -05:00
Elena Sakhnovitch 13cde8429d [ROCm-SMI] add --showNodesBw
Display min and max bandwidth between gpu nodes

Signed-off-by: Elena Sakhnovitch
Change-Id: I7289fb83f80e2f899996b7d7560ece670cc5f31f
2021-10-29 12:49:35 -04:00
Elena Sakhnovitch 15e4fe80e1 [rocm_smi.py] remove repetitive footnote
Printing "Primary die (usually one above or below the secondary) shows
total (primary + secondary) socket power information" footnote only one time, not
for every secondary die.

Signed-off-by: Elena Sakhnovitch
Change-Id: Iae9c5c94945ec38ecdb128a576a4eacafc30a044
2021-10-29 08:32:06 -04:00
Sreekant Somasekharan ce46fd237a Add test case for rsmi_is_P2P_accessible API.
Change-Id: Iccfede42925c98d96454b5f25cc0ed6fc9258911
2021-10-28 17:06:07 -04:00
Elena Sakhnovitch 50ea68e694 [ROCm SMI LIB]: Add rsmi_minmax_bandwidth_get()
API provides min/max bandwidth values between nodes.
(Current implementation only supports directly (1 hop)
connected XGMI devices.

Signed-off-by: Elena Sakhnovitch
Change-Id: Ifc95da13845fbe7903c5386d320183ffd58c5b53
2021-10-28 17:00:41 -04:00
Divya Shikre e96d6ab77e Add failing rsmi tests to exclude file to enable blacklisting
Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: Ibdad4d54ffe87391b13379c63e005fd04c6abaf5
2021-10-26 17:57:05 -04:00
Bill(Shuzhou) Liu adfa89631d Fix the compile error for gcc-11
Fix the error: 'sleep_for' is not a member of 'std::this_thread'

Change-Id: If25ef03023df17081878f9b44c3a68195f07c653
2021-10-26 15:36:52 -04:00
Philip Cox 489db9fac6 libhsakmt: Add more debug information
We need to add some more information about the debug features supported
by the platform.  We are adding the following:
        - debug supported
        - dispatch info always valid
        - precise memop supported
        - watchpoints shared

Change-Id: I68deed98619396d17e28c6e18bad424b58297485
Signed-off-by: Philip Cox <Philip.Cox@amd.com>
2021-10-22 13:49:34 -04:00
Bill(Shuzhou) Liu 78e2f2486b Support GPU memory test and compute queue test using Rocr
A new diagnostic module librdc_rocr.so is created. The
module uses Rocr to test the memory allocation, memory access
and compute queue ready status.

Change-Id: I9098f4fc3209bf381b7cb3658a4e94c2e22f2fe9
2021-10-21 11:01:12 -04:00
Bill(Shuzhou) Liu 6ab71e1a4a Correct the install path of grpc
Correct the grpc install path which miss $.

Change-Id: I17736a81ee24d2abc680a3646b1536efafcb3d69
2021-10-19 17:11:35 -04:00
Kent Russell 2801342048 kfdtest.exclude: Remove KFDEvictTest.QueueTest on GFX10
Since it's shader-based, the GFX9 shader won't work for GFX10. Blacklist
it for now until we move to a more ASIC-agnostic shader

Signed-off-by: Kent Russell <kent.russell@amd.com>
Change-Id: I353e9a3fe8c7ba6f33556e39c7e4f6b36e010480
2021-10-19 08:08:15 -04:00