Fix data race by protecting events_page access with mutex in event create
Fix potential NULL dereference in hsaKmtWaitOnMultipleEvents_Ext
Fix unchecked return value in hsaKmtCreateEvent function
Change-Id: I434bef43666e5205a8b061259569c1d99a952752
[ROCm/ROCR-Runtime commit: 857200e28c]
We had skipped doing it for PAGE_SIZE, but it should be left as the
regular PAGE_SHIFT name, especially for users who are using different
headers. We want PAGE_SHIFT and PAGE_SIZE to be consistent with one
another, so set them both explicitly to the same value if either
of them is undefined
Change-Id: I121d81c48409dd77351b59a192d824e2419a2410
Signed-off-by: Kent Russell <kent.russell@amd.com>
[ROCm/ROCR-Runtime commit: daad183bf8]
Add check before close to prevent closing invalid file descriptors
Change-Id: Ie1d50e0d55159512a14a70c1e4be058218aae668
[ROCm/ROCR-Runtime commit: ff6e1b44bf]
The fmm_node_[added|removed] functions were added in the initial FMM
support, but weren't used. Remove them now since no one's referencing
them
Change-Id: I1e46e57294a72012227b38f46c7099de0b9263be
Signed-off-by: Kent Russell <kent.russell@amd.com>
[ROCm/ROCR-Runtime commit: 3b61f75f49]
To support fully-static library ROCm builds, ensure that all global
symbols are prefixed with something meaningful to avoid collisions with
other libraries
A script was made using" objdump -C -t" to get a list of symbols,
then checking if the global symbols have a meaningful prefix (for thunk:
hsakmt or kmt in various cases)
Change-Id: Ifd353f64a3344eb60d1f6c4e041aa20967b38a59
Signed-off-by: Kent Russell <kent.russell@amd.com>
[ROCm/ROCR-Runtime commit: 3da42a0847]
trace is calloc'd but never freed. Free it.
Change-Id: I5795cbe5738f25a9621d24be86abb35c263fa8b7
Signed-off-by: Kent Russell <kent.russell@amd.com>
[ROCm/ROCR-Runtime commit: 4dc9d49aa6]
Previous code would blindly set executable bit on all allocations.
Change-Id: Id1154f08f6ba21c633905fd46b06053994d6f3cc
[ROCm/ROCR-Runtime commit: 75143555fa]
Enum type for compute AQL is defined as larger then targeted SDMAs
enum types. We should only deny legacy calls for SDMA queues that
require targeted engines.
Change-Id: I6386a8700b3b18af825b6f0d2be27052cc8de0f5
[ROCm/ROCR-Runtime commit: ae99effb29]
Core dump support relies on debugger related KFD ioctl which have been
introduced in version 1.13 of the interface. However, the code checks
for KFD_IOCTL_MINOR_VERSION (currently 17), making it impossible to
produce core dumps when using some drivers that should support it.
Update the CHECK_KFD_MINOR_VERSION calls in the debugger related ioctl
wrappers and look for KFD 1.13 or above.
Change-Id: I10a7fd03bf8f678b6318d7c25d6a7ded804dac67
[ROCm/ROCR-Runtime commit: d5acab2b39]
Extend the current Thunk implementation of queue creation to target
specific SDMA engine IDs.
Also expose the new recommend SDMA engines per IO link from the KFD
sysfs.
Change-Id: I51f9a0d83c0f1fc4d5dc837f879a7ae332e7d7e9
[ROCm/ROCR-Runtime commit: 2f588a2406]
When HSA_OVERRIDE_GFX_VERSION is used, save the overrided GFX
version to OverrideEngineId instead of original EngineId. There
are places where real GFX properties still needed, e.g. CWSR size
calculation.
Change-Id: I9d9149bae465b7cfe55604fc19e7ca34e48b7b1c
Signed-off-by: Yifan Zhang <yifan1.zhang@amd.com>
[ROCm/ROCR-Runtime commit: 3f1f68c8cb]
Fallback to old userptr registration in case SVM method fails.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Change-Id: I70c3ec74a8b4f762713e6a0619453642f3fca8e5
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 626eb4bfaf]
This lets you run two unsupported-but-really-supported cards of different architecture together in the same program. Works great w/ llama.cpp on my 7900XT + 6600.
Example usage (device 0 is RDNA3, device 1 is RDNA2):
HSA_OVERRIDE_GFX_VERSION_1="11.0.0" HSA_OVERRIDE_GFX_VERSION_2="10.3.0" ollama serve
Change-Id: Ic63ef462f698dee722d360f7fc3ef72789c277b7
Signed-off-by: AdamNiederer
Signed-off-by: Kent Russell <kent.russell@amd.com>
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 84567b6416]
Since PC Sampling is still under experiment, we can't
bump KFD_IOCTL_MINOR_VERSION to enable pc sampling.
KFD_IOCTL_MINOR_VERSION 16 already includes all pc sampling
code, so use version 16 to enable pc sampling implicitly for
customer to try-out this new feature.
Need update the version accordingly when pc sampling upstream.
Change-Id: I65840128f94e8f347c0617971c0aa4b7e478691a
Signed-off-by: James Zhu <James.Zhu@amd.com>
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 338721c24a]
KFD ioctl version is 1.16 on upstream for contiguous memory support.
Remove pc_sampling version, should be added after pc_sample upstream.
Change-Id: I6e6c3340bc8e371d68dd7741b02578be2fdef801
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 6e6f445f75]
The application may use parent process KFD handle or invalid KFD handle,
add CHECK_KFD_OPEN in all APIs to catch this application bug earlier
without calling to KFD.
Change-Id: I0391e91eeca8e6752fc9c23f0742445b823ea9b0
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: c98a8dc179]
New API to support optional alignment parameter for memory allocations.
The alignment should be larger than or equal to page size and a power
of 2.
Change-Id: Ic3fec43b3c4281f74dd33a57ab4143dcf76e1186
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: a31e84eaef]
hsaKmtRegisterMemory* can only register OS allocated userptr.
v2: Apply changes to all hsaKmtRegisterMemory* stuff.(Philip)
v3: Unlock aperture->fmm_mutex to aviod deadlock.
Suggested-by: Felix Kuehling <Felix.Kuehling@amd.com>
Change-Id: I1045af7edb4da8206cb878f64c0176ba4fc59f60
Signed-off-by: Lang Yu <Lang.Yu@amd.com>
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 4844a70d94]
To differentiate discrete and integrated GPU more flexibly in runtime,
this will aid in querying HSA_AMD_MEMORY_PROPERTY_AGENT_IS_APU
and hipDeviceAttributeIntegrated.
Change-Id: Ic8a6c9aea3b4bd19c4d5f6729af7e64c328fc61d
Signed-off-by: Lang Yu <Lang.Yu@amd.com>
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: ae3ede062f]
Set max size needed for VGPR when doing a CWSR for GFX12 and GFX12.1.
Signed-off-by: David Belanger <david.belanger@amd.com>
Change-Id: Iddefc62f1ad419c6f5ab6a872048457a1dc24037
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 259a724e21]
Skip test when PC Sampling is not supported by ASIC.
Change-Id: I6f9be0bdaed66e51052723b6df6908079470cefb
Signed-off-by: James Zhu <James.Zhu@amd.com>
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 1087dea925]
C Error returns are positive in user space and should check against errno
instead.
Fix declaration of return to type HSAKMT_STATUS.
KFD IOCTL should handle size return when querying capabilities so return
size to caller unconditionally.
Clean up error translations per function so that it's stylistically
clear.
Signed-off-by: Jonathan Kim <jonathan.kim@amd.com>
Change-Id: Ic37390425f370c7ad88f9ed014444decf19383a3
Signed-off-by: Chris Freehill <cfreehil@amd.com>
[ROCm/ROCR-Runtime commit: 206db80a56]
Shweta Khatri
Kent Russell
David Yat Sin
Xuanteng Huang
Joseph Greathouse
Jonathan Kim
Lancelot SIX
Yifan Zhang
Alex Sierra
Adam Niederer
James Zhu
Philip Yang
Lang Yu
Yiyang Wu
David Belanger
Chris Freehill