Commit Graph

585 Commits

Author SHA1 Message Date
kjayapra-amd 74ffc5f0d5 SWDEV-413997 - Cleanup fixes for Virtual Memory Management.
Change-Id: I9a4a4d9087b5daf15e3ba31e786d34db431212a1
2024-04-22 10:58:06 -04:00
German Andryeyev 7448113cfc SWDEV-440746 - Remove obsolete code
The "optimized" version of memcpy is outdated and
was used in win32 only.

Change-Id: I7f2e0e9051e37cec95438266824b5b0025c324c6
2024-04-22 09:56:42 -04:00
German Andryeyev 329ba271fa SWDEV-440746 - Wait for signal before release
Change-Id: I9e2aefdbcbba153c7f1080d80aab7a345eaf1eb4
2024-04-19 18:33:28 -04:00
Ioannis Assiouras bf74ef4025 SWDEV-451594 - Implement Readback and Avoid HDP Flush workaround for device kernel args
Change-Id: I6d41a089a17f55306e7ff402588a1e831b20a7a7
2024-04-19 09:29:20 -04:00
kjayapra-amd 56ebf5157a SWDEV-413997 - VMM IPC implementation for Linux.
Change-Id: Icfeb83ca51e96be35abb67a94d6e3e1a1ca5a934
2024-04-18 11:28:13 -04:00
German Andryeyev c95a75a2bf SWDEV-444670 - Enable teardown class
Force implicit runtime teardown with a global destructor.

Change-Id: Iabe63dedf5b94fefc98668585c45a61607120669
2024-04-16 12:00:06 -04:00
Rakesh Roy 52db98edd9 SWDEV-453180 - Add UUID support for HIP_VISIBLE_DEVICES on Linux
- UUID is Ascii string with a maximum of 21 chars which uniquely identifies a GPU
- Convert set UUID in HIP_VISIBLE_DEVICES to device index internally
- Then use existing device index logic for HIP_VISIBLE_DEVICES

Change-Id: I8cab4fe42459f8209b97f909300789e6e687b9ac
2024-04-13 22:07:19 -04:00
kjayapra-amd d52d16c8e6 SWDEV-413997 - Fixing multiple device cases.
Change-Id: I10ad3fbfca887e92cd81f68392fa1acf753cbd2b
2024-04-13 06:14:03 -04:00
German Andryeyev 7de7da4016 SWDEV-455254 - Reduce blit kernels signature
Remove offset from blit kernels, since it can be applied in setup.

Change-Id: I06b585068d68a0ee8e125ddf46a36fccb372f30d
2024-04-12 14:45:55 -04:00
taosang2 35c80dd482 SWDEV-424956 - Fix half vector printf issue
Refactor PrintfDbg::outputArgument() to remove potential risk.
Fix half vector printf issue on all devices.
Fix FEAT-56794 as well.

Change-Id: Iae39359d2128588def2e43d77fe58e868b8e71ff
2024-04-12 14:25:44 -04:00
German Andryeyev f0c7ecf617 SWDEV-455254 - Add kernel arg optimization
Add kernel arguments optimization into blit path.
Enabled by default on MI300.

Change-Id: I2694a81b90d48ad07d86dfe4c0c64fe187bada8e
2024-04-10 18:08:37 -04:00
kjayapra-amd 2b8634bada SWDEV-446298 - Adding error code to the logs on p2p hsa api failure.
Change-Id: Ic41b1ad1b64cca0e31986337a83a5146d52a7328
2024-04-10 06:00:00 -04:00
Saleel Kudchadker 3f0bcf7834 SWDEV-301667 - Fix SDMA mask reuse
If we are using the mask returned by getLastUsedSdmaEngine() then we
need to apply the SDMA Read/Write mask to it before using with HSA
copy_on_engine API.

Change-Id: I6e5dc6c187eeb3c61ee159e9d2a0fa7b4737c06e
2024-04-08 15:42:52 -04:00
Sourabh Betigeri dbac2976e4 SWDEV-451964 - Limit gpu single allocation percentage for gfx940 only
Change-Id: Iadcdadd734e7aeeb23742e426353defa972d3ad5
2024-04-05 09:43:42 -04:00
Ioannis Assiouras d7f352dbed SWDEV-453301 - Remove the option to write multiple packets in dispatchGenericAqlPacket
Dispatching multiple packets with ring the doorbell once is not supported by the lower layers

Change-Id: I7665a2dcdd4ef9e47dadfe410180fed64c5a4ee0
2024-04-05 05:28:10 -04:00
Saleel Kudchadker c157bfb202 SWDEV-301667 - Create TS for each node recorded in graph
- Create a vector to allow multiple TS to be stored in Command.
- This would mean we dont wait for entire batch in Accumulate command
to finish when we exhaust signals.
- Reduce the number of signals created at init to 64. This min value
may still need to be tuned but the KFD allows max of 4094 interrupt
signals per device.
- Store kernel names whenever they are available and not just when
profiling. If we dynamically enable profiling like for Torch, a crash
can happen if hipGraphInstantiate wasnt included in Torch profile scope
beacuse we previously entered kernel names only when profiler is
attached.

Change-Id: I34e7881a25bbc763f82fdeb3408a8ea58e1ec006
2024-03-26 14:47:24 -04:00
Rakesh Roy d1fff7cea2 SWDEV-445096 - Fix -O0 crash in OpenCL tests
- With https://gerrit-git.amd.com/c/lightning/ec/llvm-project/+/1002628 applied, at -O0 Kernel::dynamicParallelism() returns true but virtual queue isn't created
- This causes segfault inside VirtualGPU::submitKernelInternal() when getVQVirtualAddress() is called

Change-Id: Ia7af042adad2329e870c142caaac3e8fa886f8b8
2024-03-26 11:42:33 -04:00
kjayapra-amd 5cbd74b554 SWDEV-413997 - Save hsa_handle as ptr in hipMemCreate path.
Change-Id: Ica32017ef7b00326dfb6d1f604e126d40ad5b786
2024-03-26 10:24:29 -04:00
Anusha GodavarthySurya 7f84df9f74 SWDEV-301667 - Disable HostBlit copy for HIP correct if check
Change-Id: I33d1359d5e4c871f63350d8300f726e039664d86
2024-03-26 02:18:51 -04:00
German Andryeyev f296159f62 SWDEV-353281 - Change pool type for graphs
Under ROCr physical allocations don't have initial VA and require extra
flag in ROCclr. Add an option to have a mempool of physical allocations.

Change-Id: I4d062fe0dd8113d4eaf6e8b51749ed56d8701d1e
2024-03-25 10:21:05 -04:00
Ioannis Assiouras 51926b6b6b SWDEV-451736 - Revert "SWDEV-444670 - Register the Runtime::tearDown function to be called at exit"
This reverts commit 5e294f89a8.

Change-Id: Ib9cb1cc0c3903bfba56c9a5d05ae8afe96be583a
2024-03-23 07:05:02 -04:00
Ioannis Assiouras 96f5c44851 SWDEV-451166 - Disable kernel args for non-XGMI if HDP flush register is invalid
Change-Id: I227e046e2b9cb25476a50240f5d070adbd558f21
2024-03-15 05:27:52 -04:00
Saleel Kudchadker f1adecd186 SWDEV-301667 - Use right macros and level for logging
- Sometimes we want to mask out kernel names, use right level for kernel
logging

Change-Id: Ideae9647c57b86ae390ff2f4131f6d8c6df5c086
2024-03-12 19:00:03 -04:00
kjayapra-amd f5ca620baa SWDEV-423835 - Fixing kernel launch issues on Virtual Memory Management path.
Change-Id: I9f5e8a3d83af3809b2c50b21a10697e26113dd23
2024-03-12 17:22:07 -04:00
Satyanvesh Dittakavi 1b25484f0f SWDEV-447405 - Reset the last SDMA engine after every few copies
The copies can get blocked if the last SDMA engine is used by another
copy and this can lead to perf drop in some of the tests like Gromacs.
Resetting the last engine by checking the engine status and fetching the
new mask after few copies can avoid this.

Change-Id: I8fe8ea678db508d291c6242f3741fa9215e99921
2024-03-12 02:10:27 -04:00
Ioannis Assiouras 5e294f89a8 SWDEV-444670 - Register the Runtime::tearDown function to be called at exit
Change-Id: I2200b8154ce6806bf99150d0d68f6f20138fa3dc
2024-03-08 18:46:59 -05:00
Vikram 6f390f5af9 SWDEV-424956 - Fix OpenCL printf bug while printing vectors of half type
OpenCL printf handling did not process vector of half precision floats properly
 (mainly because compiler packs 2 halfs into a dword and runtime failed to extract the
 individual parts).

 This patch fixes the issue.

Change-Id: Ia1f15ccfb5db52b71c43cfd588dd38f551ee5277
2024-03-04 03:53:18 -05:00
Saleel Kudchadker 984c86f407 SWDEV-301667 - Better log
- Print SWq for AQL packets, this helps correlating a stream to the HWq
mapped

Change-Id: I610430c0872a1abc6636027c00163ec46983cd65
2024-03-01 16:43:06 -05:00
Saleel Kudchadker 68f40f78dd SWDEV-443760 - Enable device kernel args for MI300
- Enable Device kernel args for MI300* for now.
- Fix a perf issue which impacts graph instantiate when dev kernel args
are enabled.

Change-Id: I962e58fd9d8dd1a8db95e601cb03a8e9c7bac97f
2024-02-28 19:10:04 -05:00
Anusha GodavarthySurya e0e63eb04d SWDEV-447545 - Fix Enable/Disable node with hipGraph
Node can be enabled/disabled only for kernel, memcpy and memset nodes.
If the node is disabled it becomes empty node.
To maintain ordering just enqueue marker with respective node dependencies.

Change-Id: I710f3e88ab4e76c81f6f86a40a7dc61fd4c7e440
2024-02-28 17:34:03 -05:00
Ajay e643406caa SWDEV-347670 - StreamWait and StreamWrite on Windows
__amd_streamOpsWrite blitkernel in device-libs has only 3 args.
so getting rid of the 4th unused arg (sizeBytes)

Change-Id: I81cc1107f8b424bf58558c93a2495a1b878aef91
2024-02-26 22:45:10 -05:00
Jaydeep Patel 4177b4b675 SWDEV-359287 - uniform_work_group_size is interger type is 1 for true and 0 for false.
Change-Id: Iac5584c3cdd3bf78e3f1e89033b9ce7e07afefca
2024-02-22 22:37:31 -05:00
Ioannis Assiouras 1f6d416684 SWDEV-446399 - Fixed segfault in hipMemSetAccess
Change-Id: Ia1200d9bee03e8abade211287505f081e635ceec
2024-02-20 18:51:05 -05:00
Saleel Kudchadker f138e0d113 SWDEV-443760 - Enable device kern args
- Implement workaround to ensure HDP writes are done by writing and
reading the HDP MMIO register.
- Implement the same workaround for graphs, we no longer need sentinel
write/readback

Change-Id: I0d3027b46a1f61131ec62e3c8c669ff5184fa6b2
2024-02-20 02:03:14 -05:00
Saleel Kudchadker 81b8598af9 SWDEV-301667 - Cleanup code and better log
Change-Id: Ie2345264e84026156a9f81b421eed3cf4aeeeffc
2024-02-19 05:42:47 -05:00
kjayapra-amd 7d5b4a8f7a SWDEV-437832 - Changes to update host unified memory and iommuv2 flags.
Change-Id: I88998cf57c21fc446fa28e250f826c607923670b
2024-02-07 06:27:47 -05:00
Saleel Kudchadker 0567c3b720 SWDEV-301667 - Better log
Display queue base pointer in the log. This can be co-related with AQL
packets

Change-Id: I544f9b6db6ae01c85e57e4b3f0b3fffefcd7c2ed
2024-02-05 05:08:11 +00:00
Saleel Kudchadker 9a6ddae7b2 SWDEV-301667 - Reset profiler correlation_id_
- The correlation_id had random junk values which we were inserting in
the dispatch AQL packet even when no profiler was attached but if we had
a valid timestamp.
- Also make sure we dont even write the reserved2 field in the AQL
packet if no profiler attached.

Change-Id: Icdb7493198c1bb5e2d786a97e027288660854cd7
2024-02-05 05:08:11 +00:00
German Andryeyev 842eda5e7f SWDEV-440746 - Remove pending dispatch check for SDMA P2P
Change-Id: I7290345cfc60cd878fb39a06b03105441793c27b
2024-02-05 05:08:11 +00:00
cadolphe 243b86dd23 SWDEV-368741 - Check for uninstalled hsa-amd-aqlprofile to avoid Segmentation Fault
Change-Id: I0c0c7a2ea7c35237332ddd7c2461e4d9ff8e2f1c
2024-02-05 05:08:11 +00:00
Alex Xie 5e1de3cc26 SWDEV-433820 - Optimize queue initialization (ROCM)
Change-Id: I0619a9f66ae3d64a0f3fc36384f46adb302e725d
2024-02-05 05:08:11 +00:00
kjayapra-amd b366a7c992 SWDEV-437832 - Adding device property to check if the device is accelerator.
Change-Id: I8349e99c03422c268bbb60a8c143bd492d9cec09
2024-02-05 05:08:11 +00:00
Lang Yu cc29df873a SWDEV-439419 - Unlock host ptr when device ptr equals host ptr
hsa_amd_memory_lock_to_pool() and hsa_amd_memory_unlock()
should be called balanced.

Change-Id: I8b1549861bff752aabbb6399d717d1e346079a38
Signed-off-by: Lang Yu <Lang.Yu@amd.com>
2024-02-05 05:08:11 +00:00
Ioannis Assiouras 28743d8dbd SWDEV-432599 - Added null checks for rocMemory ptrs in copyMemory
Change-Id: I8ccda2c915dad22c4ea67556dfaab9ba47770069
2024-01-18 15:48:30 -05:00
Jaydeepkumar Patel 77be355fd9 Revert "SWDEV-359287 - uniform_work_group_size is interger type is 1 for true and 0 for false. Lighting kernel's init sets uniform work group flag and hence not need to set it from options."
This reverts commit ea25e65834.

Reason for revert: Waiting for staging results before finally merging it.

Change-Id: Iaabb510325f50147f368108e98531291217627c0
2024-01-18 01:29:31 -05:00
Jaydeep Patel ea25e65834 SWDEV-359287 - uniform_work_group_size is interger type is 1 for true and 0 for false. Lighting kernel's init sets uniform work group flag and hence not need to set it from options.
Change-Id: I8d3a9a6ed1c3bddaaa86f7cdc242a45e6a1d09f6
2024-01-17 04:42:52 -05:00
Saleel Kudchadker 0b0df605d4 SWDEV-301667 - Remove resetFenceDirty
Dont track the status of fence_dirty_ flag on the host, instead clear it
when we submit a barrier on the respective stream.

Change-Id: I4d98dbf20c81379c9c5da9f5b67629a8f9f6dfcd
2024-01-15 15:43:14 -05:00
German 31101c6219 SWDEV-440746 - Limit WG for compute P2P
Use only 16 workgroups for compute P2P copies.
That should be enough to utilize XGMI bandwidth.

Change-Id: I60dfe019279bb95f93c8874244c1738aad1896d8
2024-01-12 14:56:29 -05:00
Saleel Kudchadker d6b4892600 SWDEV-439242 - Fix perf regression
This issue is due to the fact that we set fence_dirty_ flag for every
kernel launch. Whenever any next HIP API is caused, the stream logic
assumes fence is dirty and queues a marker, when its not needed.

Change-Id: I10a49ee951daa92221aa6be75e2334849579a45d
2024-01-10 11:45:02 -05:00
ksankisa f554a075c7 SWDEV-404921 - Add dynamic LDS size hidden kernel argument
Change-Id: If99019058e4bfdc988ef5fc4cef001b772380453
2024-01-06 08:46:55 +05:30