Commit Graph

943 Commits

Author SHA1 Message Date
Sean Keely deee152909 Add support for exporting portable handles to GPU allocations.
Adds hsa_amd_portable_export_dmabuf and hsa_amd_portable_close_dmabuf
which allow obtaining dmabuf handles to rocr allocations.  These handles
may be shared with other APIs to support cross vendor & cross device
memory sharing.
Adds query to return whether dmabuf export is supported

Signed-off-by: Jonathan Kim <Jonathan.Kim@amd.com>
Signed-off-by: David Yat Sin <David.YatSin@amd.com>

Change-Id: I7f98501087d9563d07fc2cb428cc886b1e518b1e


[ROCm/ROCR-Runtime commit: 42243c1e8f]
2023-03-06 12:39:01 -05:00
Jonathan Kim 57064af98d Fix Engine Offsetting for Copy on Engine
Forgot SDMA blit engine indices are offset by DevToDev 0-position in
a couple of places.

Change-Id: Ie811d8281bc812738ed0107694f3dffde5e93685


[ROCm/ROCR-Runtime commit: 7364a93b98]
2023-03-03 20:45:35 -05:00
David Yat Sin 908a2b1eba Revert "Add flag for external memory allocations"
This reverts commit 000f4c0547.

Change-Id: I32a92672553c4c38ffae53a085f83c0403c160ae


[ROCm/ROCR-Runtime commit: 7ed6d73b6d]
2023-02-23 11:31:15 -05:00
David Yat Sin a2e4ba149c Revert "Enforce uncached memory on AllocatePCIeRW request"
This reverts commit 36da397f96.

Change-Id: I5a7fe9e99685f589f95dd89eacf04d44e5587f2f


[ROCm/ROCR-Runtime commit: 37b5b421b3]
2023-02-22 21:55:48 -05:00
David Yat Sin d022746cb7 Use mwaitx when busy-waiting signals
Use mwaitx instructions when busy waiting for signals to reduce CPU
energy usage.
This can be disabled by setting HSA_ENABLE_MWAITX=0

Change-Id: Ic207895a491b2bf6dacba47ef0921df3faad5b5a


[ROCm/ROCR-Runtime commit: cc48dfdbff]
2023-02-22 16:55:43 +00:00
David Yat Sin 72e7fe7aec Add function for parse CPUID information
Used to detect whether mwaitx instruction is supported

Change-Id: I66fe906325aa523c8815133cf782df3a17a7edab


[ROCm/ROCR-Runtime commit: 0ed1568afc]
2023-02-22 16:55:42 +00:00
Yifan Zhang 90e271e19b Fix MemoryConcurrentTest failure for APUs w/ small VRAM
Signed-off-by: Yifan Zhang <yifan1.zhang@amd.com>
Change-Id: I85b8f9f1ff0fbb5a063b310aa6f72b9b5cdc13b4


[ROCm/ROCR-Runtime commit: d0330d7958]
2023-02-16 20:23:38 +08:00
Yifan Zhang 565665f141 Fix rocrtstPerf.Memory_Async_Copy failure for APUs w/ small VRAM
Signed-off-by: Yifan Zhang <yifan1.zhang@amd.com>
Change-Id: Ieec5b76f0e058d5655145b51fdea48e3d87560b4


[ROCm/ROCR-Runtime commit: 83cb79510e]
2023-02-16 20:18:04 +08:00
Yifan Zhang e93868b503 Fix rocrtstFunc.Memory_Available failure for APUs w/ small VRAM.
Signed-off-by: Yifan Zhang <yifan1.zhang@amd.com>
Change-Id: I0e9d5f1880c0e484e88ed424888d94d1bcac4d53


[ROCm/ROCR-Runtime commit: 9bab46130a]
2023-02-16 20:16:28 +08:00
Yifan Zhang 94d5ab8c9e Avoid memory leak when rocrtstFunc.Memory_Available fails
Assert abort the test thread w/ memPtr1 allocated. Free memPtr1
to avoid memory leak.

Signed-off-by: Yifan Zhang <yifan1.zhang@amd.com>
Change-Id: I4e1a202c1acb9ba71a23e112254f875bf5a0abcf


[ROCm/ROCR-Runtime commit: afae35b0fd]
2023-02-16 20:13:15 +08:00
Yifan Zhang 04ea6db7e6 Fix rocrtstFunc.Memory_Max_Mem failure for APUs w/ small VRAM
Signed-off-by: Yifan Zhang <yifan1.zhang@amd.com>
Change-Id: I1c0f481af8b1d2a0939d28fb184ff6887747ab03


[ROCm/ROCR-Runtime commit: 4ebb9857ee]
2023-02-16 20:12:19 +08:00
Lang Yu d44cbfd4ad Fix memory async copy test performance issue
Copying memory from device to host with a CPU agent
would cause a poor performance due to the reading of
uncahced device memory by CPU.

Fix it by using a GPU agent.

Signed-off-by: Lang Yu <Lang.Yu@amd.com>
Change-Id: Ia3b562758fe73ef9efaa284f47e67bf569cc7b7b


[ROCm/ROCR-Runtime commit: 8501c0bcb1]
2023-02-15 22:24:55 -05:00
Ranjith Ramakrishnan 1ff8eae7be File reorg backward compatibility message changed to #error
Change-Id: I699dee834865ee573a516d58b8b8faa1da4f288a


[ROCm/ROCR-Runtime commit: 3636d487c9]
2023-02-14 21:46:43 -08:00
Jonathan Kim ff620e9fdc Add interface to DMA copy directly to a target engine.
Change-Id: Ic87cfeabb11c1a465f98f3f444d39955f5300525


[ROCm/ROCR-Runtime commit: 30920fc94d]
2023-02-13 13:50:49 -05:00
Jonathan Kim f161963c09 Make SDMA engine availability status queryable.
Report the availability of SDMA engines for memory copies.

Change-Id: Ie31b02d6b65355122bb8c98bc73700a59bee166e


[ROCm/ROCR-Runtime commit: 8f27f495c6]
2023-02-13 13:50:49 -05:00
Jonathan Kim 9021f5970d Make the number of per agent SDMA engines queryable.
Change-Id: Iae1cc9b7ec783fdda05f9384f0ad0327ea1a8cc3


[ROCm/ROCR-Runtime commit: 4f283d9bb3]
2023-02-13 13:50:49 -05:00
Cordell Bloor 921ccf5f60 Fix static initialization order
Change-Id: I1d51e150b526d050b988fe5a422644667a561cd7


[ROCm/ROCR-Runtime commit: 5873a78d58]
2023-02-09 13:51:08 -05:00
David Yat Sin 000f4c0547 Add flag for external memory allocations
ROCr internally uses the same allocation_map_ list to track memory
allocations that are both for internal allocations and allocations by
users of ROCr library. In some edge cases, the library user would call
hsa_amd_pointer_info on an invalid pointer, but ROCR would return the
pointer as valid because this pointer belongs to a memory range that
was allocated internally within ROCr. Adding a flag to differentiate
between internal and external allocations.

Change-Id: I98c52bd85f3985d1ba1b0e3101d2254b003412cf


[ROCm/ROCR-Runtime commit: 59685f4492]
2023-02-09 13:21:43 -05:00
Sean Keely c6d7c62307 Track size of pending operations in blits.
Track and report the size, in bytes, of pending unexecuted blit
commands.  To be used in copy ganging.

Change-Id: Ia7453ff88571e927df771c6c819b73c17e67708e


[ROCm/ROCR-Runtime commit: 27596aef0c]
2023-02-06 12:38:40 -05:00
Konstantin Zhuravlyov e51d58a646 Compile image blit kernels with code object v4
Change-Id: I4b1923fe8f22dda1277409794d0856419228eceb


[ROCm/ROCR-Runtime commit: f115a3505c]
2023-02-02 17:33:15 -05:00
Ranjith Ramakrishnan f4ccd8979e rocrtst package name updated as per ROCm standards
Change-Id: I6d7096a5c5c27648bf0cbfb4b1e83e72b7949421


[ROCm/ROCR-Runtime commit: a649357dec]
2023-01-26 13:04:41 -05:00
Shweta Khatri 4f670d6df7 Fixes hang due to change in order of initialization of libraries
Fixes hang due to change in order of initialization of libraries
that have cyclical dependencies and they call hsa_init() during their
initialization phase.
This implementation looks for a symbol called "HSA_AMD_TOOL_PRIORITY"
across all loaded shared libraries using dynamic section entries of the
loaded lib instead of using dlopen and dlsym for the same purpose.

Change-Id: I4865f2fd18dd186ec311a432ec38fbb5583805d2


[ROCm/ROCR-Runtime commit: 8aac885318]
2023-01-26 01:17:22 -05:00
David Yat Sin 8a86cddd3a Add query for IOMMU support
Reporting whether IOMMU V2 is supported.
IOMMU V1 support is not relevant to user, so not reporting it.

Change-Id: I77389484a87a352da9c2f7b2a5d9de264f90ee53


[ROCm/ROCR-Runtime commit: e30be76f37]
2023-01-19 11:33:21 -05:00
David Yat Sin 580ce4fd25 Add memory pool query to return location
Change-Id: I240b77119d7b8ccfc5ff6a3190d6669d69f243e8


[ROCm/ROCR-Runtime commit: 722794e258]
2023-01-19 08:45:05 -05:00
David Yat Sin 523bdde26f Add env variable to print image SRD contents
Add environment variable HSA_IMAGE_PRINT_SRD to print contents of SRD
registers for image functions

Change-Id: Ifb47a73dcfad8745ee7445e20de96e1021b80bd6


[ROCm/ROCR-Runtime commit: a4f898ad15]
2023-01-13 11:01:04 -05:00
Alexander Turek e907b85904 isa: Add fix for hsa_isa_iterate_wavefronts always returns 64
Currently, Wavefront::GetInfo(HSA_WAVEFRONT_INFO_SIZE.. always returns
64. Instead, return the proper wavefront size based on the ISA.

Temporarily, we only return 1 wavefront size for each ISA. As we do not
have mechanism from upper layers to determine correct wavefront when
there are multiple wavefronts supported. We are temporarily
returning 32 for all gfx1xxx cards even though they support 64 as the
kernels for gfx1xxx are compiled for wavefront-32 by default.

Change-Id: Ic6c2917b7e6d3704daf742d243f5ec7f49430de9


[ROCm/ROCR-Runtime commit: f7e3782b42]
2023-01-12 08:40:07 -05:00
Shweta Khatri 36da397f96 Enforce uncached memory on AllocatePCIeRW request
Change-Id: Ib5a624ab979220d50205448ef37b4550672fb97d


[ROCm/ROCR-Runtime commit: ed0a1be2c3]
2023-01-11 16:52:15 -05:00
Ranjith Ramakrishnan 829d6536f8 Revert "Remove RPATH/RUNPATH from ROCm libraries"
This reverts commit 993b1dee7e.

Reason for revert:  is blocked due to new proposal. so reverting the changes 

Change-Id: Id9b8cc1560ba3eea6e484e67df3fdc647da9f37d


[ROCm/ROCR-Runtime commit: dbf8905dd1]
2023-01-10 13:52:02 -05:00
David Yat Sin 474d087407 Force rocrtsts to use Code Object V4
Temporarily force rocrtsts to use Code Object V4 while compiler team is
about to switch the default Code Object to V5. Will switch back to using
default compiler setting once everything is tested/fixed.

Change-Id: I18e5c6771fffd8c60792fc197501d373c7ec22f3


[ROCm/ROCR-Runtime commit: 0f2fa3ba72]
2023-01-06 12:01:03 -05:00
Shweta Khatri 811c411e78 Fixed GFX11 Texture, Buffer and Sampler Resource Descriptor definitions
Change-Id: I101806f9f91ec2ad78339dabc98375bd09946dd0


[ROCm/ROCR-Runtime commit: e72329ab76]
2023-01-05 15:40:47 -05:00
Ranjith Ramakrishnan 2d2992ea65 Corrected libelf package name in depends list
libelf1 package contains libelf.so.1. Updated the package name
Improvement: Removed the initialization of cmake_install_libdir in  source code
Build scripts is initializing  the variable to "lib" and passed as build argument

Change-Id: I16a8cdc4c231487410c1114b818e9d01df4854de


[ROCm/ROCR-Runtime commit: 5c90c762f9]
2022-12-15 23:30:22 -08:00
David Yat Sin 93c4ffe473 Add Stream Performance Monitor(SPM) APIs
Change-Id: I0d48782887814ef245b7e0182e2d5570aa8c3f50


[ROCm/ROCR-Runtime commit: 6bfe57aeb2]
2022-12-08 13:56:29 -05:00
David Yat Sin 652a617846 Add agent info for fw and sdma ucode
Add two new agent info fields:
HSA_AMD_AGENT_INFO_UCODE_VERSION
HSA_AMD_AGENT_INFO_SDMA_UCODE_VERSION

Change-Id: I51cb853724b23a26e945e5c1ac32c16d0cb3bc31


[ROCm/ROCR-Runtime commit: ecdebef0b9]
2022-12-07 19:07:31 -05:00
raghavmedicherla 2b666f57fa [hsa-runtime] Modify elfsection checks in amd_elf_image class
Modified If condition checks in GElfImage::pullElf() of amd_elf_image.cpp to
 check using section types instead of a string check.

Change-Id: I1ab92f0a9118fb2382652a1cc900a3150cbee2da


[ROCm/ROCR-Runtime commit: 5727a10a1b]
2022-12-05 14:42:02 -05:00
David Yat Sin 736e5ae731 Check for debug support after parsing topology
Thunk keeps an internal cache of system topology that can be used to
speed up subsequent calls to hsaKmtAcquireSystemProperties(). This cache
is cleared by calling hsaKmtReleaseSystemProperties() at the beginning
of BuildTopology().
hsaKmtRuntimeEnable() also calls hsaKmtAcquireSystemProperties() inside
Thunk. Move call to hsaKmtRuntimeEnable() after BuildTopology() so that
we can re-use Thunks internal cache.
Parsing of of topology can take ~150 ms on systems for large number of
nodes.

Change-Id: I741709d49d67d244f5fbd707fe8f01ab923bb153


[ROCm/ROCR-Runtime commit: e39ad34d9c]
2022-12-02 11:26:00 -05:00
Shweta Khatri c236e10be6 Fixed callback method for dl_iterate_phdr api which is called for each loaded shared object
Simplified the callback method. Also fixed the way, loaded shared object were getting appended into a string vector,
which was not being passed to this callback method.

Change-Id: I68661dd73f61a11c42fa92f670e8e7b6ffcb5711


[ROCm/ROCR-Runtime commit: 8751e65b79]
2022-11-21 19:00:34 -05:00
Ranjith Ramakrishnan 3268fe2a56 Change pragma message to warning
File reorganization feature was implemented with backward compatibility
The backward compatibility support will be deprecated in future release.
Changed the #pragma message to #warning for a smooth transition

Change-Id: Ibaedc1873bc764d25f74d9ca9416077d084e332d


[ROCm/ROCR-Runtime commit: a34804ed3e]
2022-11-17 09:38:24 -08:00
David Yat Sin 406115eaac Revert "Correct limit query return type to match spec ABI."
This reverts commit 689e9ce6a4.

Changing the parameter sizes breaks backward ABI.

Change-Id: Iff14b7c11294f0931f36fcfd42fff11a492d4205


[ROCm/ROCR-Runtime commit: b9d1ad8604]
2022-11-14 19:13:58 -05:00
David Yat Sin 3e6721df9f Allow page-aligned len for ipc_memory_create
Previous versions of HIP will call hsa_amd_ipc_memory_create with then
len aligned to granularity. Temporarily allow this so that we go not
break backward compability. Will remove this after 2 releaes

Change-Id: I6b5ac2cad5d32d62c803637cf1a2c6deebc03169


[ROCm/ROCR-Runtime commit: cb71e2d715]
2022-11-09 15:01:47 +00:00
David Yat Sin f5e7534574 Use paged memory for queues on MEC devices
MES devices need GART mappings and therefore need non-paged memory. But
using non-paged memory introduces performance regression where it can
take over 80 ms to see the signal changes if the memory is in the wrong
NUMA node. Currently, we cannot control NUMA affinity when allocating
non-paged memory. Using non-paged memory allocation only on devices that
have MES scheduler

Change-Id: Ib27fb01d75247aa4f2bb2aa4503c6af5a98afda0


[ROCm/ROCR-Runtime commit: c1e836b6ab]
2022-11-04 13:23:21 +00:00
David Yat Sin d08d9a4a7b Use os::createThread to launch SVM profiler thread
Using previous method of std::thread for SVM profiler task was causing
segfaults on thread launch on RHEL 8 if libhsa-runtime library is loaded
using dlopen.

Change-Id: Ic010cd6ae9bc6e6ed0605de02b93f6aae8ed3e97


[ROCm/ROCR-Runtime commit: 0e4c7336ff]
2022-11-03 10:52:11 -04:00
Jonathan Kim b553bf1c24 Fix doorbell offset fetch for GFX11
Transient exec usage is not required for GFX11 and will result in a NULL
return of s_sendmsg_rtn if directly returned to exec_lo.

Directly fetch and mask the doorbell ID to ttmp3 for GFX11 instead.

Change-Id: Ie17ed69d68d84ab18869b1c7871a0ed0482cd661


[ROCm/ROCR-Runtime commit: f9edf73cd7]
2022-11-02 11:55:37 -04:00
Nirmal Unnikrishnan a0977f8108 Updating the Rocrtst packaging
Update rocrtst packaging to add dependency on rocm-core so that rocrtst
gets uninstalled when rocm-core package is removed

Depends-On : I1e7ed52d7eed2c190d0b5651e7ded7192d7634b5

Change-Id: I7243dd29950b93a2665720a0062816c574f0f640


[ROCm/ROCR-Runtime commit: 8225271e18]
2022-11-02 09:38:48 -04:00
Ranjith Ramakrishnan 161abc3316 Add libelf-dev to package depends list
In ubuntu, the package depends list was not showing libelf. Added the same

Change-Id: I713951bd7181f44d667561aaf437f85c6cd783b0


[ROCm/ROCR-Runtime commit: 76cf5d2edc]
2022-10-31 13:07:55 -07:00
David Yat Sin e80b937d93 No-Op for allow access on imported IPC
If hsa_amd_agents_allow_access is called for an imported IPC handle,
ignore the request as this pointer will already have these pointers
mapped to other GPUs during IPCAttach()

Change-Id: I4bf33ed57e93b5a3ead749d4f87ab6f2750bed58


[ROCm/ROCR-Runtime commit: b4f26534eb]
2022-10-25 22:38:47 +00:00
David Yat Sin 8b9275bb36 Early return for invalid pointer queries
If a user queries the pointer info on an invalid pointer,
hsaKmtQueryPointerInfo will return error or unknown pointer. The other
fields in HsaPointerInfo are invalid, so we do not return them to the
user.
Also removing the assert and returning unknown pointer instead. As the
assert will not trigger in release builds.
hsaKmtQueryPointerInfo may also return unknown pointer for userptrs as
they are not always tracked by thunk. Adjusting code to still treat
these pointers as valid in this case.

Change-Id: Idf5cd8b61cd532d31b072f449839d223369bb138


[ROCm/ROCR-Runtime commit: 18547173e9]
2022-10-21 15:28:48 -04:00
Freddy Paul 993b1dee7e Remove RPATH/RUNPATH from ROCm libraries
:Since all public interface libraries are present in
same folder RUNPATH/RPATH is not required in the library itself.
Application shall provide the required RPATH/RUNPATH to load all
libraries.

Change-Id: I1d1ba920bf291eb89bd1f4c0fd0cfd80c7d739bd


[ROCm/ROCR-Runtime commit: ac66865385]
2022-10-21 11:05:06 -04:00
David Belanger 9f2c88c46a Initial changes for gfx1101, based on gfx1100/gfx1102 implementation.
Change-Id: I949c1027ccabf38b4f924590e42e7327dc550f73
Signed-off-by: David Belanger <david.belanger@amd.com>
Reviewed-by: Jonathan Kim <jonathan.kim@amd.com>


[ROCm/ROCR-Runtime commit: a0d3db6e8d]
2022-10-13 09:28:39 -04:00
David Yat Sin f5dc99bf9e Use user requested size for memory fragments
Amount of memory requested by user may be aligned-up internally to
the memory pool granularity. The extra padded memory should not be
considered when validating pointers from the user. Also return the
user requested size when user queries pointer information.

Change-Id: I28b25448ea03c836b44fafdb34b7330cf6887424


[ROCm/ROCR-Runtime commit: 39632a713e]
2022-10-07 21:32:49 +00:00
David Yat Sin 2affd1a1f6 Fix compile warnings and remove unused variables
Change-Id: I7acaee5e9cf218b358ffaf0e3af6067faf6f3d2a


[ROCm/ROCR-Runtime commit: 9cb10a3dd8]
2022-10-06 10:11:17 -04:00