Commit Graph

746 Commits

Author SHA1 Message Date
Andryeyev, German 2305f8ae56 SWDEV-465041 - Add support for user events with DD (#321)
* SWDEV-465041 - Add support for user events with DD

User events can be replaced with HSA signals. Add the interface
to allocate HSA signal for user events and update the status on
CL_COMPLETE.
Force pinned path with DD to avoid blocking calls. Pinned memory
can be released only when the command is complete.
Simplify device enqueue path to use generic kernel arg buffer and
signals

* Fix notifyCmdQueue() logic for OCL

* Avoid blocking calls in OCL with DD

* Add event  destruciton in a case of the failure.
2025-08-12 19:04:36 -04:00
Chaudhary, Jatin Jaikishan 926eedc230 SWDEV-508540 - adjust for compile time wavefront 64 (#747) 2025-08-11 10:17:27 +05:30
Kudchadker, Saleel 9b045922a8 SWDEV-538195 - Introduce threshold for handler submission (#723)
- When doing device/stream sync, we can submit a handler which may
  introduce some host side delays. Use DEBUG_CLR_BATCH_CPU_SYNC_SIZE to
  batch commands for host wait. Default for HIP is 8 commands.
- Investigation is underway in ROCr but need to address this for now in
  HIP runtime.
2025-08-06 20:34:42 -07:00
Xie, Pengda cd46294b31 SWDEV-520384 - Improve Fat Binary loading latency (#390)
Init and fini kernel needs to be launched when we load and unload code object. Avoid looping through all kernels within a code object just to run the init and fini kernels. Compiler currently only generates 1 init and fini kernel.
2025-08-05 14:02:05 -07:00
Xie, Pengda 4121a860bf SWDEV-505833 - Remove DEBUG_CLR_SKIP_RELEASE_SCOPE flag (#735)
Cleanup debug flag DEBUG_CLR_SKIP_RELEASE_SCOPE
2025-08-05 08:31:55 -07:00
Kudchadker, Saleel 1492328894 SWDEV-547614 - Demangle graph kernel names (#809) 2025-08-01 14:35:30 -07:00
Manocha, Rahul 3f6f9d6081 SWDEV-539710 - Defer allocation of managed variable (#652)
Co-authored-by: Rahul Manocha <rmanocha@amd.com>
2025-07-31 08:30:23 -07:00
Betigeri, Sourabh a1f056bd11 SWDEV-483895 - Use ROCr to reserve memory for HMM (#590) 2025-07-15 21:40:13 +05:30
Sang, Tao 1351cd7fa8 SWDEV-539145 - Simplify host memory pool management (#668)
* SWDEV-539145 - Simplify host memory pool management

Remove unnecessary variables and functions.
Make code simpler and clear.

* Change cpu_agent_info_ into pointer.

* Restore getPreferredNumaNode()
2025-07-11 10:38:40 -04:00
Sang, Tao 397f303d97 SWDEV-508776 - support VGPRs validation (#274)
Clarify some VGPRs terms description.
Fix some wrong query logics of availableVGPRs_ and
availableRegistersPerCU_ in device info.
Add hipDeviceAttributeMaxAvailableVgprsPerThread
attribute query.
Remove hardcoding of following
 info_.vgprAllocGranularity_
 info_.vgprsPerSimd_
2025-07-09 10:46:52 -04:00
Kudchadker, Saleel 46d766e4e2 SWDEV-539378 - Use agent of IPC memory owner (#570)
- Currently runtime just uses the local agent as it did not check for
  IPCShared()
- With this fix we query hsa_amd_pointer_info and get the right agent
  for the memory to pass it to the HSA copy api
2025-07-08 12:02:01 -07:00
Xie, Pengda 9c1bff0ae7 SWDEV-540576 - Abort if user request a core dump (#653)
* SWDEV-539414 - Return error status from runtime handler when HIP_SKIP_ABORT_ON_GPU_ERROR is false

* SWDEV-539414 - default handler when GPU core file is generated

* SWDEV-540576 - Abort if user request a core dump

Change-Id: I9e2c640acf559880bd13641de9103e660ef822a3

---------

Co-authored-by: Assiouras, Ioannis <Ioannis.Assiouras@amd.com>
Co-authored-by: agunashe <ajay.gunashekar@amd.com>
2025-07-03 00:29:14 +05:30
Sang, Tao a7d7687b8f SWDEV-539145 - Support extended fine grained system memory pool (#603)
* Add hipHostMalloc() new flag hipHostMallocUncached which will force to allocate pinned
host memory on extended fine grained system memory pool.
* Add hipHostAlloc() new flag hipHostAllocUncached which will force to allocate pinned
host memory on extended fine grained system memory pool.
* Add hipHostRegister() new flag hipHostRegisterUncached which will force to map
host memory onto extended fine grained system momory pool.
2025-07-01 10:10:10 +05:30
Sang, Tao f4d78d427c SWDEV-539399 - Support ROCCLR_MEM_HSA_CONTIGUOUS (#610)
Fix missing support of ROCCLR_MEM_HSA_CONTIGUOUS in
hipExtMallocWithFlags().
2025-06-26 14:05:23 -04:00
Kudchadker, Saleel 5d53c83806 SWDEV-535490 - Improve logging (#441)
- Include HIP version/githash in the logs
- Add a new method to print library path
2025-06-24 13:00:00 -07:00
Brzak, Branislav 494c0e3201 SWDEV-531996 - Add legacy fallback Mesa loading via dlsym (#512) 2025-06-11 08:02:51 +05:30
Brzak, Branislav 5fabfedbd1 SWDEV-531996 SWDEV-530154 - Use *getprocaddress to obtain Mesa functions (#504) 2025-06-10 13:31:32 +05:30
Andryeyev, German 5cc172c99c SWDEV-533074 - Expose hipDeviceAttributeNumberOfXccs attribute (#336) 2025-05-21 22:35:42 +05:30
Jayaprakash, Karthik 12131de4a9 SWDEV-529929 - hipMemGetHandleForAddressRange implementation. (#245) 2025-05-20 15:56:04 -04:00
Kudchadker, Saleel 1b0ea080e4 SWDEV-523279 - Use preferred engine mask for SDMA (#317)
- ROCr now reports preferred engine for copy status. We can leverage
this for max bandwidth for inter-GPU copies
- Cleanup logging
2025-05-19 16:04:51 -07:00
Kudchadker, Saleel 5712944c7c SWDEV-531518 - Fix offset accumulation (#333)
srcAddress/dstAddress accumulation was cumulative, which shouldnt be
done if we increment offset.
2025-05-19 18:03:06 +05:30
Jayaprakash, Karthik bb7750a946 Revert "SWDEV-522707 - Set phys_mem_handle type to sizeof(size_t) to avoid blocking address range. (#105)" (#348)
This reverts commit 6811fd90b8.
2025-05-19 15:19:16 +05:30
Jayaprakash, Karthik f5b8db33f1 SWDEV-531711 - Report correct error code based on device failure. (#286) 2025-05-17 06:33:13 -04:00
Andryeyev, German da198ac5b2 SWDEV-531678 - Remove split path from the dispatch (#283)
The split path for blit kernels are no longer necessary, since the new blit kernels
don't use the copy size as the global workload
2025-05-12 12:50:32 -04:00
Xie, Jiabao(Jimbo) a320a3f214 SWDEV-528913 - support gfx950 in rocsetting (#217)
* SWDEV-528913 - support gfx950 in rocsetting

---------

Co-authored-by: Jimbo Xie <jiabaxie@amd.com>
2025-05-07 15:44:49 -04:00
Andryeyev, German 65a0181a7c SWDEV-528808 - Release all HW queues even if only one is idle (#240)
Pytorch may not explicitly idle each queue. Thus, some queues can be considered as busy,
but have idle state in reality
2025-05-05 19:09:01 -04:00
Andryeyev, German 9b018165ce SWDEV-528808 - Disable dynamic queue by default (#256)
Dynamic queue management will be disabled by default and
the original sort logic is restored
2025-05-05 10:56:35 -04:00
Assiouras, Ioannis d3fb8eda8b SWDEV-525593, SWDEV-527293 - Acquire active queue after xferQueue is created (#165)
For xferQueue VirtualGPU::create is called after ProfilingBegin
so the active queue needs to be acquired.
2025-04-30 09:21:11 +01:00
Jiang, Julia eb62fe9f62 SWDEV-522634 - Fix device properties in hipInfo (#203) 2025-04-29 11:29:47 -04:00
Sang, Tao 96cadbc9e9 SWDEV-520352 - Remove HostThread and legacy monitor (#230)
* SWDEV-520352 - Remove HostThread and legacy monitor

Remove HostThread, semaphore and  legacy monitor.
Make original logics of thread and command queue stricker.
Add more comments to make logics clearer.
Some other minor improvement.

Also part of SWDEV-458943.
2025-04-29 09:55:24 -04:00
Jayaprakash, Karthik 6811fd90b8 SWDEV-522707 - Set phys_mem_handle type to sizeof(size_t) to avoid blocking address range. (#105) 2025-04-29 11:19:16 +05:30
Jayaprakash, Karthik b2388dfb88 SWDEV-506467 - Skip Abort in case of crash from the device. (#60)
Change-Id: I964b2f2647d068202e9c38fcddb1337da754df8d
2025-04-29 11:19:02 +05:30
Kudchadker, Saleel 072fb0804e SWDEV-521647 - Fix tracking of hw_event (#206)
- When a command may possibly have two packets(like device heap
  initializer), and if there is no signal on the main kernel packet the
tracking was broken as it marked HW event of the command as the first
packet signal.
- Make sure if no completion signal is attached to the second packet
  then clear the HW event for the command.
2025-04-25 08:46:44 -07:00
Kudchadker, Saleel ce24936970 SWDEV-510186 - Improve logging (#220)
- Print all arguments for logs, this is useful for debug
2025-04-25 08:40:31 -07:00
Sang, Tao 1113eff3f9 SWDEV-493275 - Support scratch limit (#20)
Support programmatic query and change of scratch limit on
AMD devices.

Change-Id: Id5da355a77366f97868e462847f3916e87fd2af6
2025-04-24 17:15:25 -04:00
Andryeyev, German 3fd7650fe3 SWDEV-459758 - Pass workgroup size explicitly (#185)
It's easier for compiler to move explicit kernel arguments into user SGPRs
2025-04-15 15:22:15 -04:00
Yao, Longlong 0de73eeaf8 SWDEV-518966 - Avoid creating Arena Memobj for VMM pointer (#39)
Change-Id: I69c6c0a1464d01e674ac929de34ab10047012f1a

Signed-off-by: Longlong Yao <Longlong.Yao@amd.com>
2025-04-11 16:55:53 +05:30
Andryeyev, German 4c363df3bf SWDEV-517481 - Add more restrictions to the queue management (#168) 2025-04-10 21:51:45 +05:30
Patel, Jaydeepkumar 9e7248aa36 SWDEV-521011 - Allow max stack size as per ISA. (#73) 2025-04-08 10:15:38 +05:30
Patel, Jaydeepkumar b5c9cbc236 SWDEV-508632 - Align address to 2 MBs for hidden heap allocation. (#29) 2025-04-02 16:33:29 +05:30
Arandjelovic, Marko e7ada4effe Revert SWDEV-512344 - Unmap all subbuffers (#26)
This reverts commit 0b69120cfcb5b4689d9f2037b1a01e274d85c20f.
2025-03-19 21:17:36 +05:30
Andryeyev, German 28967982b2 SWDEV-517481 - Add dynamic queue management (#37)
Enabled by defaulty. DEBUG_HIP_DYNAMIC_QUEUES controls the feature
2025-03-19 11:22:50 -04:00
Saleel Kudchadker 78d0ff2dbc SWDEV-519596 - Avoid passing dep signal to SDMA
- For D2H cases avoid passing dependent signals to SDMA, the signals
  take a while to resolve on SDMA engine

Change-Id: I569635228af977847f201c82ca897002f8f2f4a8
2025-03-07 17:37:21 -05:00
Branislav Brzak c2d1776ebd SWDEV-516564 - SWDEV-512817 - Remove mentions of gfx940 and gfx941
Change-Id: Ia069fcb9c6948c3fc9a00961593c9dcc59609375
2025-03-05 04:26:07 -05:00
Saleel Kudchadker e03e4f3b5d SWDEV-502365 - Track last used command
- This change tries to save extra synchronization packets we may insert
  as we didnt track the completion signals for every command. We track
the current enqueued command until it exits the enqueue stage. We also
record the exit scope to know if we flushed the caches
- Handle correct release scopes and store completion signal as HW events
- Use a new finishCommand implementation to only wait for the command
  passed as the argument

Change-Id: Ie4350c5dd24f5d48dfa6ccbabd892f0544caadcc
2025-03-04 16:05:02 -05:00
Marko Arandjelovic 3ec1d2d2f1 SWDEV-512344 - Unmap all subbuffers
Since hipMemMap can be called for multiple device handles on the same virtual memory, the same is true for hipMemUnmap, meaning that virtual memory can be "partially unmapped".

This means that the unmap function can be called for a specific part of the reserved address, meaning that only the designated subbuffer should be released. If unmap is called on the entire reserved memory, then all subbuffers should be released.

The main point is that for every hsa_amd_vmem_map, there should be a corresponding hsa_amd_vmem_unmap. Otherwise, if entire memory is unmapped by a single unmap call, then HSA will report the memory as "in use" if an attempt is made to delete it.

Change-Id: I039308eafb820decfb1c09f60347f26cdad1a362
2025-03-02 13:41:48 -05:00
Saleel Kudchadker 37d606d193 SWDEV-513197 - Unify getBuffer implementation
- Use getBuffer/releaseBuffer in BlitManager
- Cleanup XferBuffer as we use ManagedBuffer for both reads/writes

Change-Id: I2661b85dd012763b17a38a743fec1b1d79125f67
2025-02-28 12:47:51 -05:00
Saleel Kudchadker ca530c660b SWDEV-513197 - Improve launch perf for Device Heap kernels
- If any kernel uses device heap, the launch needs to be preceeded by an
  init kernel, Save on the extra barrier packet launch/flush between the
init heap kernel and user kernel

Change-Id: I8ebc6246188200e5f673dc464bc76a53bcb8b7c6
2025-02-27 19:17:51 -05:00
Rahul Manocha 4bf634dfca SWDEV-510849 - Restore pinned memory copy path
1) Create getBuffer method to return pinned host memory or staging buffer
2) for D2H path use managed buffer instead of static buffer
3) use staging buffer copy for 16KB < size < 1MB
4) use pinned memory copy for size > 1MB

Change-Id: I13d4d6ab60691bc6c7724239db1e11e23f0f3dc2
2025-02-26 11:25:02 -05:00
kjayapra-amd 1f583a6870 SWDEV-516303 - Remove SDMA retainer logic to select the engine.
Change-Id: I818129444131825cdb87e06cb495afa3e5cdb683
2025-02-20 11:34:38 -05:00