In order for hipMemPrefetchAysnc_v2() api to work, we need rocr to
migrates the ranges of pages requested to the particular NUMA node in
question, via move_pages().
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
* SWDEV-561708 Initial shared queue pool apis
* Validate params; some fixes in callback function (but still needs to be checked)
* Dtor cleanup
* minor
* Enable profiling; remove callback since aql_queue takes care of it
* setPriority and setCuMask APIs updated for counted queues
* Increasing step and minor version for rocprofiler
* Tests for CountedQueueManager
* tests
* Code refactored to make pool manager part of GpuAgent only (incomplete); unique handles issue pending
* Refactored code to support CQM inside GpuAgent and unique handles; multithreaded test added
* Changed to ASSERT_SUCCESS macros for all tests
* RIng buffer overflow test added
* tests fixed; cleanup added at hsa_shutdown
* priority conversion table changes
* Compiler warnings fixed
* Rewrite 1 test; add desc and improve SetUp() code
* Improvement
* Unififed getinfo for both counted and non-counted queues
* Address PR feedback
* Addressing feedback: memleak, data type mismatch, documentation
* improve comment
* format
* Missing HSA_API macros for roctracer
* Revert "Addressing feedback: memleak, data type mismatch, documentation"
This reverts commit 5e498a55fb3640e00d06cec63dcec79293fb23de.
* Improving acquire api doc
* release api doc improved
* error codes for release api doc
* SWDEV-555889 - Support mipmap on rocr
Support mipmap in hip-rt on rocr backend.
Enable all mipmap tests in Windows.
Some other minor improvement.
Add some SRD logs that will be removed finally.
* Add sampler.mipFilter to fix sampler issues on mipmap in rocr.
Fix format issues of view of leveled image and mipmap image in blit kernel in rocr.
Enabled disabled mipmap tests.
* Rewrite view logic
* Set word4.f.PITCH = 0 for mipmap SRD on navi31 to fix unstable test issues.
Reset last error in nagative tests.
* Remove SRD dump log from hip-rt
Let Rocr mipmap log be in condition.
* minor format chang
* Exclude mipmap tests for mi200+ which don't support mipmap.
* Fix set/get access failure for VMM on windows
* seperate code paths for linux and windows to avoid using import/export calls in windows
---------
Co-authored-by: Rahul Manocha <rmanocha@amd.com>
* Add hipDeviceAttributeExpertSchedMode
---------
Co-authored-by: Stefan Sokolovic <stefan.sokolovic2@amd.com>
* Update hipDeviceAttributeExpertSchedMode unit test
* Move check to ROCr from thunk interface
* Revert unrelated whitespace changes
* Revert version bump
---------
Co-authored-by: Stefan Sokolovic <stefan.sokolovic2@amd.com>
* rocr: Add ProtectMemory API and use it in RemoveAccess
Replace munmap + mmap with mprotect when removing memory access.
This improves performance by 5-10x, ensures atomicity (no race
condition window), and prepares for WSL/DXG compatibility fixes.
Suggested-by: David Yat Sin <David.YatSin@amd.com>
Signed-off-by: Flora Cui <flora.cui@amd.com>
Signed-off-by: Horatio Zhang <Hongkun.Zhang@amd.com>
* rocr: Skip CPU mapping operations on WSL
On WSL, CPU cannot access GPU VRAM due to platform restrictions.
CPU access would fault-in system RAM instead, causing data corruption
and memory leaks. Return HSA_STATUS_ERROR to fail fast rather than
silently creating broken mappings. GPU-to-GPU mappings remain functional.
Signed-off-by: Flora Cui <flora.cui@amd.com>
Signed-off-by: Horatio Zhang <Hongkun.Zhang@amd.com>
* rocr: reduce ifdef linux
v2: Fix IsDXG check logic
Signed-off-by: David Yat Sin <David.YatSin@amd.com>
Signed-off-by: Horatio Zhang <Hongkun.Zhang@amd.com>
---------
Signed-off-by: Horatio Zhang <Hongkun.Zhang@amd.com>
Signed-off-by: David Yat Sin <David.YatSin@amd.com>
Signed-off-by: Flora Cui <flora.cui@amd.com>
SWDEV-539526 - Add support for Mipmapped Array in Rocr
Add support for Mipmapped Array functionality in Rocr Runtimeenabling GPU applications to work with multi-level texture mipmaps. The implementation introduces new public APIs for creating, querying, and managing mipmapped arrays across different GPU architectures.
Signed-off-by: Apurv Mishra <Apurv.Mishra@amd.com>
Co-authored-by: Shweta Khatri <shweta.khatri@amd.com>
Co-authored-by: taosang2 <tao.sang@amd.com>
* SWDEV-569319 Replace ScopedAcquire with stdcpp wrappers
* Remove KernelMutex and KernelSharedMutex abstractions with std::mutex and std::shared_mutex
* Replaced unique_locks with lock_guards
* More changes
* Replace new and deletes with smart pointers
* Replaced some more with shared ptrs
* Replacements with smart pointers - pt 2
* missed change
* Add HasExpertSchedMode device prop
* Add unit tests for HasExpertSchedMode
* Add gfx12 check for HasExpertSchedMode prop
* Update gfx major version check and test for ExpertSchedMode
* Minor fix and ROCr version bump
* Update projects/rocr-runtime/runtime/hsa-runtime/inc/hsa_ext_amd.h
* Update projects/rocr-runtime/runtime/hsa-runtime/inc/hsa_ext_amd.h
* Apply suggestion from @dayatsin-amd
* Apply suggestion from @dayatsin-amd
---------
Co-authored-by: Stefan Sokolovic <stefan.sokolovic2@amd.com>
Co-authored-by: David Yat Sin <77975354+dayatsin-amd@users.noreply.github.com>
* SWDEV-558848 - vmm api support for rocr on windows
* Fixes to VMM handle Map/Unmap Set/Get Access
* Fix GetShareableHandle to use pointer for shareable handle
* Update os specific map/unmap memory calls
* clang format update
* Minor syntax fixes from code review
Co-authored-by: Yiannis Papadopoulos <102817138+ypapadop-amd@users.noreply.github.com>
---------
Co-authored-by: Rahul Manocha <rmanocha@amd.com>
Co-authored-by: Yiannis Papadopoulos <102817138+ypapadop-amd@users.noreply.github.com>
* Run pre-commit's whitespace related hooks on projects/rocr-runtime
In order for pre-commit to be useful, everything needs to meet a common
baseline.
Signed-off-by: Mario Limonciello (AMD) <superm1@kernel.org>
* Add missing semicolon which would block compilation on big endian CPUs
Signed-off-by: Mario Limonciello (AMD) <superm1@kernel.org>
---------
Signed-off-by: Mario Limonciello (AMD) <superm1@kernel.org>
* Fixing the copy back to the original buffer malformed packets
* Addressing Copilot Comments
* Addressing Review comments
* Adjust staging buffer size allocation
Change staging buffer size to match the number of packets.
This patch enhances compatibility for DXG environments by introducing conditional
checks for DRM operations, particularly around buffer object metadata handling
in IPC scenarios. These changes improve robustness in DXG IPC memory management
without impacting existing functionality in standard Linux environments.
Signed-off-by: Horatio Zhang <Hongkun.Zhang@amd.com>
WSL uses the call just for the thread wake-up, however under Windows
KMD needs the actual value (SWDEV-568592). The interface is changed
to avoid programming of a modified write_ptr value, which somewhat
changes the client's logic.
Core dumps are not supporetd for gfx110x, but should be possible for
gfx115x. The current code disables core dumps completly for all gfx11xx
agents, relax this to allow gfx115x.
* rocr: Fix exception on AsyncEventControl init
Fix exception on init when compiling with in release mode.
* rocr: Fix crash when interrupts are disabled
Fix segfault due to assert for signal->EopEvent() being false when
HSA_ENABLE_INTERRUPT=0. Use Signal::WaitMultiple(..) when interrupt is
disabled.
---------
Co-authored-by: JeniferC99 <150404595+JeniferC99@users.noreply.github.com>
Fix race condition when SetAsyncSignalHandler for the first time because
async_events_thread_ could be null and launched twice.
Refactored async-events to use lazy_pointer.
* Support Windows HANDLE in interop_map_buffer
* Refactored Windows HANDLE in interop_map_buffer
* ROCr System Dependent Handle Type
* Fix for ROCr Handle Conversion Bug
* Remove Windows Header
Enable image build in Windows.
Remove some useless codes that fail building in Windows.
Some minor improvement.
Temporarily exclude mipmap test files.
Prevent negative tests affect some tests.
Move some catch info log codes into failed cases.
Workaround for rare issue on gfx90x asics when SDMA_OP_POLLREGMEM
returns before polled memory has value of 0.
Removing previous SW workaround to double-poll as it was not reliable.
* rocr: hsa_amd_pointer_info return err on shutdown
Decrement ref count before starting to unload to make sure API
calls during shutdown return error.
Delete blit objects during agent destructor.
* Add support for HSA_AMD_SYSTEM_SHUTDOWN_EVENT
Add support for new event to indicate shut down within the
hsa_amd_register_system_event_handler API.
* SWDEV-555347 - Remove lock contention in async events loop
* SWDEV-555347 - Introduce Pool of AsyncEventItems
* create generic mempool for AsyncEventItem
* Use BaseShared allocate and free for async event pool
---------
Co-authored-by: Rahul Manocha <rmanocha@amd.com>
* rocr: Add support for VMM and RDMA
Add extra CPU mapping so that kernel-mode drivers can look up the memory
mapping by virtual address.
* Update projects/rocr-runtime/runtime/hsa-runtime/core/runtime/runtime.cpp
Co-authored-by: Yiannis Papadopoulos <102817138+ypapadop-amd@users.noreply.github.com>
* Update projects/rocr-runtime/runtime/hsa-runtime/core/inc/runtime.h
Co-authored-by: Yiannis Papadopoulos <102817138+ypapadop-amd@users.noreply.github.com>
* rocr: Honor uncache flag in memory_lock_to_pool()
Also, combined several flag options used in apis into a
single integer.
Signed-off-by: Chris Freehill <cfreehil@amd.com>
* rocr: Fix hsa_amd_pointer_info on CPU agents
Fix hsa_amd_pointer_info query returning allowd on VMM pointers for CPU
agents when CPU mapping was mapped with PROT_NONE.
---------
Signed-off-by: Chris Freehill <cfreehil@amd.com>
Co-authored-by: Yiannis Papadopoulos <102817138+ypapadop-amd@users.noreply.github.com>
Co-authored-by: Chris Freehill <cfreehil@amd.com>
Co-authored-by: cfreeamd <166262151+cfreeamd@users.noreply.github.com>
Query IPC handles on shared memory export/import for any metadata as a
means to uniquely identify handles that happen to be backed by buffers
that point to the same memory.
Danylo Lytovchenko
David Yat Sin
Sunday Clement
pghoshamd
Tao Sang
Rahul Manocha
Filip Jankovic
hongkzha-amd
Jin Jung
Apurv Mishra
Yiannis Papadopoulos
Maneesh Gupta
Mario Limonciello
Ammar ELWazir
German Andryeyev
lancesix
Jaydeep
Alysa Liu
cfreeamd
JeniferC99
Tim Huang
Xie, AlexBin