Commit Graph

1728 Commits

Author SHA1 Message Date
Jatin Chaudhary c23913f6e7 SWDEV-474146 - use __bf16 to do operations
Change-Id: I568dfa97238fd760f5362a8e560c33402f96cff3
2025-02-12 07:03:05 -05:00
Steven Chung 782976f5c2 SWDEV-496674 - Convert non-templated typedefs to templates for consistent mangling
Change-Id: I952d15f20afc85c0118403f82e75360197049ef5
2025-02-04 16:37:00 -05:00
Aidan Belton-Schure 152cee3737 SWDEV-443561 - Add tools dispatch table
Change-Id: I3445554e486ab7b94592571f52c1530cb918d021
2025-02-04 04:57:38 -05:00
taosang2 32daa8f384 SWDEV-501963 - Add missing codes for gfx950
Cherry-pick https://gerrit-git.amd.com/c/compute/ec/clr/+/1162997

Change-Id: I6b3c6bf55c61cffd43cd6f17b75998f751b75723
2025-01-31 14:34:49 -05:00
Jatin Chaudhary 1fdbf35d14 SWDEV-508617 - There is no NaN for E4M3 and FNUZ
Change-Id: I330b041019990231c098073f94d9d40a3c13ba76
2025-01-30 11:48:34 -05:00
Jatin Chaudhary e6fb89190a SWDEV-507817 - fix the return type of one of the atomicMin variants
Change-Id: I9915eb174d5677e21adbabae5819c9e306338ab3
2025-01-29 11:52:19 -05:00
Jatin Chaudhary bd7d40a4d8 SWDEV-491248 - Fix build_mask
thread_rank() gives thread index in a block. Limit the range to the
current warp size.

Change-Id: Ib5c9831236096485cf99ba7ab0b911a3b10de31c
2025-01-22 04:46:01 -05:00
Konstantin Zhuravlyov dd4378611a SWDEV-341212 - HIP header changes for supporting SPIR-V
This removes almost all uses of the deprecated
__AMDGCN_WAVEFRONT_SIZE macro, which is unavailable
when targeting SPIR-V, and adds a SPIR-V compatible
formulation of warpSize (which should end up as the
sole definition of warpSize once we remove support
for treating it as a compile time constant). It
is incomplete in that the cooperative_groups
implementation will need additional surgery.

Squashed commit of the following:

commit 6840826c3fec8516857dc4f2092d84358550f588
Author: Alex Voicu <alexandru.voicu@amd.com>
Date:   Fri Dec 6 23:36:32 2024 +0000

    Add deprecation warning for constexpr uses of `warpSize`.

commit a72307a7353034c2de53fd164e016967945fd0d1
Author: Alex Voicu <alexandru.voicu@amd.com>
Date:   Fri Dec 6 23:12:14 2024 +0000

    Prepare HIP RT for SPIR-V.

commit 5e40dd746ac4f8c93b521ef048ff9d494905ba95
Author: Alex Voicu <alexandru.voicu@amd.com>
Date:   Fri Dec 6 22:46:05 2024 +0000

    Revert stale change.

commit 231fe91c53dba4cabd832fc84eaa6ddb402271a0
Merge: a48905ec9 78f62d323
Author: Alex Voicu <alexandru.voicu@amd.com>
Date:   Fri Dec 6 22:37:24 2024 +0000

    Merge branch 'amd-staging' of https://github.com/ROCm/clr into amd-staging

commit a48905ec9cfe0e017cc64943195be82b530117d7
Author: Alex Voicu <alexandru.voicu@amd.com>
Date:   Tue Sep 17 03:14:56 2024 +0100

    Add scaffolding for SPIR-V support.

Change-Id: I2e84bbe90df58a5f9a8709b619905f04fa5b96dc
2025-01-20 08:42:24 -05:00
Jaydeep Patel 1aa8383b09 SWDEV-509664 - Specify type explicitly.
Change-Id: Ia0c0478682fa15eae7a31a2360310f08151716d4
2025-01-17 13:09:48 -05:00
Rahul Manocha 93cff75928 SWDEV-492165 - Add support for 16 bit atomicAdd and atomicCAS
Change-Id: I7f1e4876fe2960dff1775d27cb6443a89e146c86
2025-01-13 13:35:40 -05:00
zichguan-amd b8ba4ccf9c SWDEV-495789 - revert Fix ambiguity of fma for _Float16 for libc++ (#1976)
Change-Id: I45ae4711a047f4484a018b9409c9f6ecf09720ce
2024-12-29 10:56:32 -05:00
Jatin Chaudhary fccf0fa2f0 SWDEV-377518 - Fix bf16/fp8 header to be compileable with hiprtc
Change-Id: I2093a39d79a46da7e102266c04c2a71e03dcb88e
2024-12-17 08:57:15 -05:00
Ioannis Assiouras e8b2fdab96 SWDEV-483134 - Remove hipExtHostAlloc API
Change-Id: I60777ef5c56b60dd8100d0d794ca10fb3b96a555
2024-12-16 17:13:49 -05:00
Istvan Kiss 3c863dad91 SWDEV-502543 - Update doxygen to surface funtions and Coop Group API
Change-Id: Id4df63b8ae64a1113f85d89aa250ac9f7cc8b9bb
2024-12-14 14:11:37 -05:00
Sourabh Betigeri f1c05e9026 SWDEV-421020 - Adds hipGraphAddBatchMemOp, SetGetParams and execSetParams APIs
Change-Id: Ieccecfe6173cc68fd3c01f86c99f7cc09fe194a3
2024-12-13 06:23:39 +00:00
Sourabh Betigeri 03dbcd8ca7 SWDEV-440866 - [hip-roclr] Adds support to batch memory operations APIs
Change-Id: I5ac63a6626af8c2b4ac382c52dfe1aaf0b3716b8
2024-12-12 19:29:24 -05:00
Maneesh Gupta 072f94c204 Revert "SWDEV-491314 - enable _sync() functions with 64-bit mask argument"
This reverts commit 4e2fd192eb.

Reason for revert: Introduces regression SWDEV-503319

Change-Id: I888c5b95d904146e4782e8c57d736878fcdde678
2024-12-11 11:16:48 -05:00
Jatin Chaudhary 98b33886cd SWDEV-503299 - Do not use operator to check for nan
Some libs use __HIP_NO_HALF_OPERATORS__ and __HIP_NO_HALF_CONVERSIONS__
which results in operators being hidden and can cause errors.

Change-Id: I83c194d7d727cba30b46d7c296f7d396549f5fca
2024-12-11 00:33:44 -05:00
Jatin Chaudhary 5122b8c999 SWDEV-501779 - add correct function qualifier for fp16 functions
Some functions were __device__ only, but should be __host__ and
__device__, changed them to __HOST_DEVICE__.
Some functions were __HOST_DEVICE__ but were using ockl functions,
changed them to __device__ only.

Change-Id: Ife9e7abe60415bda68f5f9a101e6e7c39ad51064
2024-12-05 10:17:18 -05:00
Sourabh Betigeri 2ca644cf22 Revert "SWDEV-440866 - [hip-roclr] Adds support to batch memory operations APIs"
This reverts commit bd5d8e9baf.

Reason for revert: hipInfo fails on windows. Updating llvm amd-mainline-closed

Change-Id: I57e1fa1945188b0bc0a799c4f3d540f2b7713003
2024-12-02 16:46:12 -05:00
Sameer Sahasrabuddhe 4e2fd192eb SWDEV-491314 - enable _sync() functions with 64-bit mask argument
Change-Id: Ieb13a9e1b2fc49ff225a05a51056d1212d95ae57
2024-12-01 10:16:59 -05:00
Sourabh Betigeri bd5d8e9baf SWDEV-440866 - [hip-roclr] Adds support to batch memory operations APIs
Change-Id: I449ffca44bbb04d13348d112e896d603c70fd485
2024-11-30 17:54:32 -05:00
Aidan Belton-Schure 9652d69575 SWDEV-436099 Use new amdgcn_ballot builtin
Change-Id: I024fabc6c5b3f39c66885eb7615953f4d0432e9a
2024-11-27 04:34:50 -05:00
Julia Jiang 3bd32d7d4b SWDEV-497618 - Define GL group of doxygen for HIP API
Change-Id: Ia43f25201df960b905e5fd45481f486e7bffabdb
2024-11-15 11:25:51 -05:00
Sameer Sahasrabuddhe 556390f9c7 SWDEV-490198: _sync() will be enabled by default in 6.4
Change-Id: Id029424a9c0f6b144a7aa0e96fe8acc4a138ec51
2024-10-25 09:54:40 -04:00
Rahul Manocha e729f08704 SWDEV-468039,SWDEV-482579 - Enable FP8 SW Conversions on pre gfx940 archs
1) SW Conversions for ocp and fnuz are enabled on pre mi300 archs
2) for mi300 only fnuz is enabled
3) for gfx1200 only ocp is enabled

Change-Id: I90373752a2d15eff20d5deec874ed396ba4e1788
2024-10-17 11:49:22 -04:00
Jatin Chaudhary b977101893 SWDEV-486137 - match behavior of int variants of hadd/uhadd/rhadd/urhadd
Match cases and handle cases where it can overflow.

Change-Id: I3d6f802686af230a622ef9891a844135ad3d1ae5
2024-10-09 13:47:33 -04:00
Jaydeep Patel d6193a2f23 SWDEV-483436 - User spt stream as def with -fgpu-default-stream=per-thread for hipMemsetAsync.
Change-Id: Ia85c2b4c40fc9250754d3b64fb9fd1c615362572
2024-09-29 01:42:33 -04:00
Jonathan R. Madsen 07c9c7fe56 Fix HIP API trace versioning
Change-Id: I33f2be4668c96e2225d4ca9a253e61ec2dc65102
2024-09-25 10:32:14 -04:00
Julia Jiang bb03ef11a3 SWDEV481762 - Updated definition of 'DEPRECATED' in header file
Change-Id: I88986b8e1815f3d816595f3eb2da8a6c1c1c2993
Jenifer helped make a combined PSDB build, together with the change in hip repos
https://gerrit-git.amd.com/c/compute/ec/hip/+/1114046
Combined PSDB verification passed.
http://rocm-ci.amd.com/job/compute-psdb-staging-hip/17293/
2024-09-05 15:41:04 -04:00
victzhan 8be00b6602 SWDEV-477218 - Implement hipDeviceGetTexture1DLinearMaxWidth
Change-Id: I8103f710abeb869f5f84be61c57a30b24356def6
2024-09-05 15:09:38 -04:00
Ioannis Assiouras 2c84211b58 SWDEV-470372 - Added hipExtHostAlloc API
This change adds a new HIP API `hipExtHostAlloc` which preserves
the functionality of `hipHostMalloc`.

Change-Id: I13504c6fc13465ddd7aed329795bb4f2fef1baff
2024-08-27 08:26:03 -04:00
Jatin Chaudhary bb52d9ed62 SWDEV-480489 - fix unsafeAtomicAdd
Integration into pytorch pointed out some issues, value narrowing, to
fix this we are now using unions. Also removed check for -munsafe*
compiler flag. The check is now just on builtin detection.

Change-Id: I49364503fa429bd862952f9b29879072afa6d553
2024-08-27 06:29:11 -04:00
Sameer Sahasrabuddhe 6df2da65cd SWDEV-480725: missing __ockl_wfall __ockl_wfany in amd_hip_bf16.h
Change-Id: Iff4aeec411bfeaf4cc187c515e2da3d5898f89cb
2024-08-25 22:49:14 -04:00
Satyanvesh Dittakavi d69cc35750 SWDEV-473942 - SWDEV-431367 - Correct atomicMax(_system) and atomicMin(_system)
- Fixes -0.0 and +0.0 comparison. For atomicMax if the value on
address is -0.0 and on val is +0.0, gfx90a's unsafe atomics will swap
them. This behavior should be consistent with cas loop as well.

- _system variants of atomicMax and atomicMin are resulting in
incorrect output. Updated these to use the similar implementation as
atomicMax and atomicMin.

Change-Id: I20df36ee29ae0434a6b564f2ba71193fe41cfa59
2024-08-13 10:38:50 -04:00
Rahul Manocha 39bbc0341d SWDEV-468039 - FP8 host only conversion support on mi200
Change-Id: I0891f42d1b7c0d94d099fe26df5db3eff64ba564
2024-08-07 20:51:00 -04:00
Jaydeep Patel 6344ddb2f3 SWDEV-470886 - Add maybe_undef attribute for shfl device function due to not all lanes of wave define var and compiler needs to know about this.
Change-Id: I3a683887e033305ac55362f356838b491a6d50f2
2024-08-05 00:53:13 -04:00
Sourabh Betigeri 7298b80112 SWDEV-365151 - Fix the fns32 to do 32 bit computation and adds a wrapper to ease porting from CUDA
Change-Id: I0b5a9ca11c98f8c1c40cfba7f4e057bfda2d756e
2024-07-26 11:20:14 -04:00
Sourabh Betigeri 4fbd7abbb2 SWDEV-475394 - Fix for the return type to be in-line with CUDA
Change-Id: I7c833571d47b4e86a86e4a0095b61947d16ecab6
2024-07-25 16:33:33 -04:00
Rahul Manocha 353f15afa6 SWDEV-468039 - FP8 OCP headers
Change-Id: Iecd32c5a0357781da07395d32f894415954b7b22
2024-07-25 12:42:23 -04:00
Jatin Chaudhary e43176bde9 SWDEV-466747 - add shfl functions in bfloat16
Change-Id: Ide7d7e1d449783cced8867abf43ff45f5bce113a
2024-07-23 10:51:02 -04:00
Jatin Chaudhary 6a655a77e7 SWDEV-466747 - optimize conversions for bfloat16 operations
Since we made the members public, we can optimize some operations which
do not require redundant conversions to half_raw types.

Change-Id: I31555ef18e695d8e24b89f0418187fa4e932a38a
2024-07-17 18:37:25 -04:00
Julia Jiang dd30e0e893 SWDEV-472710 - Adding gitattributes and remove trailing spaces
Change-Id: Ic8ad2071745f0ffe6a2e120bfebb6d90bf270f87
2024-07-15 12:39:56 -04:00
Marko Arandjelovic 152f343124 SWDEV-472345 - Fix coalesced group size
In case when the tile size is greater than the number of active threads,
the coalesced group size should be equal to the number of active threads.

Change-Id: I1d41322f2428a07862a590cb5d34b01243383b7c
2024-07-12 04:29:53 -04:00
sdashmiz 57e79802cd SWDEV-421021 - Add APIs cuMemcpyNodeGet/Set params
Signed-off-by: sdashmiz <shadi.dashmiz@amd.com>
Change-Id: I24bc0da56aad34c9d5876a3d83b59515f11dc3ea
2024-07-10 12:53:01 -04:00
Jatin Chaudhary ecd812b2d8 SWDEV-460834 - add unsafe atomic add for fp16 and bf16
Change-Id: I6de5c2c425c9f8ac7f6c4e5c83c8b8b7ac8fe4cb
2024-07-02 16:45:17 -04:00
Jatin Chaudhary 36d58c6518 SWDEV-450109 - add missing coop group operations
any, all, ballot, match_any, match_all for coalesced_group and
tiled_group. add binary_partition.

Change-Id: I8ed0e654f36d788653352a0617beb5b8d44749bc
2024-06-20 10:42:06 -04:00
Ioannis Assiouras 8f42ad6aa3 SWDEV-464648 - code and comment cleanups
Change-Id: I5ba3f1bff500b3cd5903c2f441017735e688f83f
2024-06-07 22:38:09 +01:00
Fabian Ritter 819e537dc5 Add __device__ specifier to HIP's warpSize constexpr variable
The warpSize variable is set to the value of the __AMDGCN_WAVEFRONT_SIZE macro,
which is a meaningless default in host code.
The resolution for SWDEV-449015 will introduce diagnostics for uses of this
macro in host code, which includes the current definition of the warpSize
variable. With the __device__ specifier, the definition of the warpSize
variable will not cause these diagnostics.

This change does not stop the variable from being used in host code since clang
intentionally does not diagnose uses of __device__ constexpr variables in host
code.

Change-Id: I0317217affe94fdf2dfd9ad0f134e68f5173245f
2024-06-04 09:32:52 -04:00
Saleel Kudchadker f18a3405d7 SWDEV-301667 - Add enum for TASK command
Change-Id: I5d2712d913bdbc42bc6eaccf9eb2f8cb9976b22e
2024-05-28 14:18:46 -04:00