* [SRC] Enable unroll=1 for gfx950
* Fix typo from rebase in generate.py
* Support for unroll=1 and gfx90a when building for all GPU targets
---------
Signed-off-by: nileshnegi <Nilesh.Negi@amd.com>
* Add fault injection of starting warps with random variations
This is done by inserting randomly delays after __syncthreads().
The feature can be turned off by FAULT_INJECTION=OFF in cmake.
* Remove manually introduced bug for demo purpose
* Use only one thread per warp for checking wall clock
* removed gfx940 and gfx941
* removed gfx940 and gfx941
* Update "gfx94" to "gfx942" in init.cc
* Updated remaining "gfx94" updates to "gfx942"
* Update filenames and variables from gfx940 to gfx942
---------
Co-authored-by: akolliasAMD <akollias@amd.com>
* misc/msccl: read graphCaptureStatus for every collective call
* fix a bug in checking whether UBR is enabled in MSCCLPP
* cmake: Fix patch reversal order
* misc/msccl: add logging
* erase handle key from mscclpp communicator during deregistration
* remove check on buffer size being a multiple of 32 from registration/deregistration routines since these checks are applied during enqueue
* add check for greater than zero buffer size in mscclpp registration
* Fix collective trace
* Use nontemporal for st_global
* Fix previous commit
* Add HDP flush to data receive path
* Fix previous commit
* Control flushing by NCCL_NET_FORCE_FLUSH and RCCL_NET_HDP_FLUSH
* Introduce RCCL_NET_HDP_FLUSH and RCCL_NET_GDR_FLUSH
Both are on by default. Turn both off will skip all flush will likely
result in data error.
* Enable GDR copy by default
* Remove GDR flush env var because it is disabled by GDC flush
* Output kernel collective trace at comm destroy by default
* Limit kernel timeout messages to 100
* Use system relaxed atomic for loadInt
* Refine timeout messages and use atomic for setting offset from CPU
* Add kernel trace for barrier timeout
* Add backup barrier to avoid race in atomicAdd
* Use different counters for different warps
* Rework barrier implementation
* Fix for other GFX
* Use __hip_atomic_store and __hip_atomic_load
* Fix bug in previous commit
* Don't reset barrier values in running kernel
* Update trace format
* Fix typo
* Switch back to hip_atomic_fetch_add
* Use same barrier implementation for all GFX
* Remove extra threadfence
* Turn off HDP flush by default
Please use RCCL_NET_HDP_FLUSH=1 to switch on HDP flush
* Remove unnecessary changes from alterative barrier implementation
* Added back __threadfence_block
* Revert back to threadfence for gfx other than gfx94x
* Added RCCL env params to control setting the SO_REUSEADDR and SO_LINGER socket options. This can allow control over the number of file descriptors created during bootstrapping.
* Casted the linger value to `int` sooner to avoid a scope of unknown typed-ness.
* Added CHANGELOG entry for this feature.
* ext-src: add MSCCLPP memory registration APIs
* update mem-reg patch with mscclpp helper routine to check if buffer is registered
* RCCL integration of MSCCL++ user-buffer registration APIs
* only include mscclpp_nccl header if ENABLE_MSCCLPP is defined
* ext-src: update mscclpp mem-reg patch
* add helper routine to patch
* check handle before MSCCL++ deregister
* fix typo to replace send buff with recv buff
* in case of no mscclpp registration, dduring deRegister call, ont fall back to rccl deRegister which will return an error
* Apply suggestions from code review
Whitespace suggestions and reducing diffs to avoid future merge conflicts
Co-authored-by: corey-derochie-amd <161367113+corey-derochie-amd@users.noreply.github.com>
* rename helper functions and change their return type
* set RCCL user-buffer registration to occur if attempting MSCCL++ registration with a buffer in managed memory
---------
Co-authored-by: isaki001 <Ioannis.Sakiotis@amd.com>
Co-authored-by: isaki001 <36317038+isaki001@users.noreply.github.com>
Co-authored-by: corey-derochie-amd <161367113+corey-derochie-amd@users.noreply.github.com>
Nilesh M Negi
isaki001
gilbertlee-amd
Wenkai Du
corey-derochie-amd
Avinash
Mustafa Abduljabbar
Tim
Nusrat Islam
Bertan Dogancay
Pedram Alizadeh
akolliasAMD
mberenjk
Dingming Wu
Vijay Srinivasan
Arm Patinyasakdikul
BertanDogancay