AI/rocm-systems
2
0
Fork 0
Código Incidencias Pull Requests Acciones Paquetes Proyectos Lanzamientos Wiki Actividad
1.683 Commits 3 Ramas 0 Etiquetas
858fa4e65dafd89659a6723af35d8a966a89858f
Gráfico de commits

1564 Commits

Autor SHA1 Mensaje Fecha
Istvan Kiss 858fa4e65d Add documentation for NPS4 and CPX partition modes (#1555) 
[ROCm/rccl commit: 28ab8603d2]
 2025-03-31 09:25:25 -06:00
Nilesh M Negi 1a2eca1756 Revert "[GRAPH] Increase default nChannels to 112 for gfx950 (#1596)" (#1620) 
* Revert "[GRAPH] Increase default nChannels to 112 for gfx950 (#1596)"

This reverts commit cf17cff5b6.

* [DOC] Update Changelog

* [DOC] Update CHANGELOG

[ROCm/rccl commit: b17338d164]
 2025-03-28 17:57:06 -05:00
Bertan Dogancay b737d8c222 Merge pull request #1559 from BertanDogancay/2.23 
[SYNC] 2.23.4-1

[ROCm/rccl commit: 532f54c244]
 2025-03-28 17:06:56 -04:00
Nilesh M Negi a7ec191754 [TEST] Switch to googletest release 1.12.0 (#1621) 
Signed-off-by: nileshnegi <Nilesh.Negi@amd.com>

[ROCm/rccl commit: 0e2c461c6c]
 2025-03-28 12:39:42 -05:00
Nilesh M Negi 210f90ae0f [SRC] Enable unroll=1 for gfx950 (#1602) 
* [SRC] Enable unroll=1 for gfx950

* Fix typo from rebase in generate.py

* Support for unroll=1 and gfx90a when building for all GPU targets

---------

Signed-off-by: nileshnegi <Nilesh.Negi@amd.com>

[ROCm/rccl commit: 307bc10781]
 2025-03-27 18:21:35 -05:00
BertanDogancay 8ed27fde74 Merge remote-tracking branch 'nccl/master' into develop 
[ROCm/rccl commit: 0b2062c560]
 2025-03-27 12:53:04 -05:00
isaki001 f0c853438c Disable mscclpp (#1614) 
* disable mscclpp by default

[ROCm/rccl commit: 9dc23d9265]
 2025-03-25 15:21:16 -05:00
Nilesh M Negi 290ca7deca [CI] Fix warnings from pytest in Azure CI (#1617) 
* [CI] Fix warnings from pytest

* [CI] Append to LD_LIBRARY_PATH instead of overwrite

---------
Signed-off-by: nileshnegi <Nilesh.Negi@amd.com>

[ROCm/rccl commit: 565f0ade60]
 2025-03-22 23:22:53 -05:00
Nilesh M Negi 8cfbc0fbd1 [UT] Increase stack size for StandaloneTests to 480 (#1616) 
Signed-off-by: nileshnegi <Nilesh.Negi@amd.com>

[ROCm/rccl commit: d6b987a53f]
 2025-03-21 21:33:32 -05:00
dependabot[bot] 68672b9b3c Bump jinja2 from 3.1.5 to 3.1.6 in /docs/sphinx (#1591) 
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.5 to 3.1.6.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/3.1.5...3.1.6)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[ROCm/rccl commit: fe3f3aae17]
 2025-03-21 17:17:32 -06:00
dependabot[bot] 83c59c859a Bump cryptography from 43.0.1 to 44.0.1 in /docs/sphinx (#1543) 
Bumps [cryptography](https://github.com/pyca/cryptography) from 43.0.1 to 44.0.1.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/43.0.1...44.0.1)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[ROCm/rccl commit: dd61df75ab]
 2025-03-21 17:17:19 -06:00
gilbertlee-amd 4f67522420 Removing the experimental clique kernel files (#1610) 
[ROCm/rccl commit: 626dc50ab5]
 2025-03-20 18:10:01 -06:00
Wenkai Du e86b217182 Add fault injection of starting warps with random variations (#1593) 
* Add fault injection of starting warps with random variations

This is done by inserting randomly delays after __syncthreads().
The feature can be turned off by FAULT_INJECTION=OFF in cmake.

* Remove manually introduced bug for demo purpose

* Use only one thread per warp for checking wall clock

[ROCm/rccl commit: 90ad586d94]
 2025-03-20 16:11:43 -07:00
gilbertlee-amd 12c1fe8fdf Psuedo-randomly adding zero-byte sends in AllToAllv unit test (#1597) 
[ROCm/rccl commit: 9a4e49ff1a]
 2025-03-20 17:00:48 -06:00
amd-jmacaran 72454ece3e [Azure CI] Minor environment setup fixes 
- Add extra deletion to ensure source workspace is clean for the job.
- pytest expects function names to start with test_


[ROCm/rccl commit: 805448261d]
 2025-03-20 15:36:10 -04:00
corey-derochie-amd e95578ef4c removed gfx940 and gfx941 (#1606) 
* removed gfx940 and gfx941

* removed gfx940 and gfx941

* Update "gfx94" to "gfx942" in init.cc

* Updated remaining "gfx94" updates to "gfx942"

* Update filenames and variables from gfx940 to gfx942

---------

Co-authored-by: akolliasAMD <akollias@amd.com>

[ROCm/rccl commit: 6505639cf4]
 2025-03-20 09:34:53 -06:00
Joseph Macaranas a95b2c9fb7 Introduce framework to run Azure Pipeline jobs (#1608) 
- PR runs targeting develop will require approval from at least one person in a predefined list.
- Nightly runs are separated and will not require approval.
- Sample pytest script is provided for expanding test coverage.


[ROCm/rccl commit: 6403afff4a]
 2025-03-18 17:29:44 -04:00
Wenkai Du c6f4c8d17a GDRCOPY support: Off by default (#1605) 
[ROCm/rccl commit: bd0092e8f1]
 2025-03-18 08:17:01 -07:00
Avinash 5a25b110af Memory leak fix when numIBDevices = 0 (#1429) 
* Initial commit for testing

* Fix memory leak in checkOptions

* Fix memory leak in checkOption

* x

* Delete cmake-3.28.2-linux-x86_64.sh

* gcn changes

* gcn memleak fixes

* gcn leak fix

* memory leak fixes for parseRome4P2H and ncclTopoAddGPU

* Keeping only necessary file for fixes

Deleting temporary scripts I created for debugging and testing

* changing to GCN_ARCH_NAME_LEN

* Added sanity check directory

* refactoring scripts

* Updated to sanity checks folder

* Initial fixes

* changes in tools

* pointing RCCL lib build to debug version

* Removed second pthread_detach

* Removing sanity checks

* Keeping only code changes

* addressing memory leaks in ncclIbinit

---------

Co-authored-by: Chao Chen <cchen104@amd.com>

[ROCm/rccl commit: ccb0820743]
 2025-03-17 11:21:19 -05:00
mberenjk a3a598efb3 Skipping AllReduce test on more than 8 ranks for FP8 type on Hyabusa (#1598) 
* Skipping AllReduce FP8 test on 9 to 16 ranks (gfx90a) as it's using Tree algorithm not RING

---------

Co-authored-by: Marzieh Berenjkoub <mberenjk@amd.com>

[ROCm/rccl commit: 5f691aaf65]
 2025-03-17 10:22:49 -05:00
Mustafa Abduljabbar 74d6537141 Multi-node reduce_scatter improved auto-selection for LL and LL128 on gfx942 (#1604) 
* Add reduce_scatter LL and LL128 thresholds

* Always honor user choice for protocol

[ROCm/rccl commit: f67b2cc908]
 2025-03-17 11:21:01 -04:00
Wenkai Du 0bd40f5a87 Enable LL128 on gfx942 (#1549) 
[ROCm/rccl commit: 245c2de909]
 2025-03-16 15:10:05 -07:00
Nilesh M Negi cf17cff5b6 [GRAPH] Increase default nChannels to 112 for gfx950 (#1596) 
Signed-off-by: nileshnegi <Nilesh.Negi@amd.com>

[ROCm/rccl commit: 1df73e209e]
 2025-03-14 14:47:03 -07:00
Wenkai Du afd04a5117 Limit P2P channels per peer to not exceeding max channels (#1594) 
* Limit P2P channels per peer to not exceeding max channels

* [UT] test single GPU cases for all collectives

* [UT] fix out of range root value

[ROCm/rccl commit: 4237caad69]
 2025-03-11 09:32:09 -07:00
Tim a79fa36b77 mscclpp compatibility check for ubr (#1573) 
[ROCm/rccl commit: f6c6d451a9]
 2025-03-09 22:10:47 -04:00
isaki001 14be1c9a7a fix the size of the recv buffer in AllGather UBR test (#1564) 
[ROCm/rccl commit: 59c55842f1]
 2025-03-05 11:42:15 -06:00
Nusrat Islam e7c90e0a46 misc/msccl: force use of mscclpp (#1581) 
[ROCm/rccl commit: ac823818aa]
 2025-03-04 12:48:59 -06:00
Bertan Dogancay d1247bbf2a [Transport] Fix IntraNet (#1582) 
[ROCm/rccl commit: d88cca3098]
 2025-03-04 13:30:36 -05:00
Wenkai Du 086fa823db NPKit: enable reduce scatter profiling (#1580) 
[ROCm/rccl commit: f957c4fe22]
 2025-03-04 10:03:56 -08:00
Nilesh M Negi 751370bb70 [BUILD] Enable multiple GPU targets in MSCCLPP (#1574) 
Signed-off-by: nileshnegi <Nilesh.Negi@amd.com>

[ROCm/rccl commit: 063c6cfc11]
 2025-03-01 22:28:42 -06:00
dependabot[bot] 977d04cb9a Bump rocm-docs-core from 1.15.0 to 1.17.0 in /docs/sphinx (#1558) 
Bumps [rocm-docs-core](https://github.com/ROCm/rocm-docs-core) from 1.15.0 to 1.17.0.
- [Release notes](https://github.com/ROCm/rocm-docs-core/releases)
- [Changelog](https://github.com/ROCm/rocm-docs-core/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/ROCm/rocm-docs-core/compare/v1.15.0...v1.17.0)

---
updated-dependencies:
- dependency-name: rocm-docs-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[ROCm/rccl commit: a6b2ca224e]
 2025-02-28 16:32:38 -07:00
Nusrat Islam f70f406463 misc/msccl: Read graph capture status for every collective call (#1576) 
* misc/msccl: read graphCaptureStatus for every collective call

* fix a bug in checking whether UBR is enabled in MSCCLPP

* cmake: Fix patch reversal order

* misc/msccl: add logging

[ROCm/rccl commit: 23c0b7bd84]
 2025-02-28 17:16:07 -06:00
Wenkai Du 3be905ca83 Improve RDMA flushing by write dummy payload with RO=0 (#1570) 
* Improve RDMA flushing by write dummy payload with RO=0

* Rename env var for disabling this change to RCCL_GDR_FLUSH_GPU_MEM_NO_RELAXED_ORDERING

[ROCm/rccl commit: 60c1264d27]
 2025-02-27 16:20:32 -08:00
Nilesh M Negi a3f44b4a10 [BUILD] Fix generate.py for gfx950 (#1575) 
* [BUILD] Fix generate.py for gfx950

Signed-off-by: nilenegi <Nilesh.Negi@amd.com>

* [BUILD] Cleaner way to check for gfx targets

Signed-off-by: nilenegi <Nilesh.Negi@amd.com>

---------

Signed-off-by: nilenegi <Nilesh.Negi@amd.com>

[ROCm/rccl commit: 02b8e504f0]
 2025-02-26 18:43:37 -06:00
Bertan Dogancay a9d09c6551 Use bit reversal based mapping for multi-node (#1572) 
[ROCm/rccl commit: 85eb1f16bc]
 2025-02-26 09:48:03 -05:00
Pedram Alizadeh acf5822a6c enable building rccl for gfx950 (#1571) 
[ROCm/rccl commit: f268553ee4]
 2025-02-25 16:13:48 -05:00
Nilesh M Negi c363118589 [BUILD] MSCCLPP: Fix OS check for CentOS (#1568) 
Signed-off-by: nileshnegi <Nilesh.Negi@amd.com>
Co-authored-by: corey-derochie-amd <corey.derochie@amd.com>

[ROCm/rccl commit: daaa6e155f]
 2025-02-25 13:03:04 -06:00
Nusrat Islam 3fbafef948 ext-src: tuning for allreduce8 kernel (#1560) 
This PR tunes the number of threadblocks used for larger (>1MB)
message sizes.

[ROCm/rccl commit: fdf75fd2c1]
 2025-02-21 19:34:38 -06:00
Nusrat Islam 4a5ab6cf75 ext-src: fix mscclpp allreduce for non-multiple of 128 message sizes (#1556) 
[ROCm/rccl commit: 83f8b191ff]
 2025-02-21 11:58:10 -06:00
gilbertlee-amd 4ca7e6873e Rail optimized trees (#1540) 
* Allow disabling rail-optimized trees via RCCL_DISABLE_RAIL_TREES, Graphviz-friendly output via RCCL_OUTPUT_TREES


[ROCm/rccl commit: ddc5d58b93]
 2025-02-20 15:18:29 -07:00
Nilesh M Negi 5f3134db50 [TOOLS] Update rcclDiagnostics script (#1557) 
* [TOOLS] Update rcclDiagnostics script

Signed-off-by: nileshnegi <Nilesh.Negi@amd.com>

* Fix typo in valid_marketing_names list

Co-authored-by: corey-derochie-amd <161367113+corey-derochie-amd@users.noreply.github.com>

---------

Signed-off-by: nileshnegi <Nilesh.Negi@amd.com>
Co-authored-by: corey-derochie-amd <161367113+corey-derochie-amd@users.noreply.github.com>

[ROCm/rccl commit: 159587be5c]
 2025-02-20 16:11:05 -06:00
akolliasAMD ae0b3c19d2 reverted the syncLDS back to syncthreads (#1554) 
[ROCm/rccl commit: aedbc95735]
 2025-02-19 10:44:32 -07:00
Wenkai Du 7eff149ceb Insert barrier after loading work items to LDS (#1551) 
[ROCm/rccl commit: baaa2ac64d]
 2025-02-18 10:17:27 -08:00
Wenkai Du 35987b9170 Enable GDRCopy only on gfx94x (#1550) 
* Enable GDRCopy only on gfx94x

* Use cudaFree instead of hipFree

* Add warning if failed to get device property

* Remove extra return

[ROCm/rccl commit: 32dc7ef47c]
 2025-02-17 13:28:19 -08:00
Sohaib Nadeem f7602c30f8 Remove COMPILING_TARGETS from CMakeLists.txt (#1533) 
COMPILING_TARGETS is not actually used for --offload-arch option,
instead GPU_TARGETS is being used implicitly when we call
find_package(hip REQUIRED) (See hip-config-amd.cmake).

[ROCm/rccl commit: 2f1c0bb213]
 2025-02-16 21:46:37 -06:00
Nikhil-Nunna f228a50646 Env conf debug (#1534) 
* Initial Script ready for review

* Added RCCL-tests and RCCL versions

* Added output folder and README

* Base format built

* Added ROCm version

* Added function to center titles and Vram information

* Added HIP version

* Cleaned formatting

* UCX version and MPI version

* Added NUMA balancing

* Added rocminfo

* Removed notes

* Changed regex for broadcom Nic

* Removed note by the ACS info

* Added Hostname to summary and details

* Print summary to terminal

* Added argparse

* Added flags and readme

* Added GPU ID

* fixed spelling

* renamed script again

* Added file descriptor and locked mem checks

* Added file descriptor and locked mem checks

* Removed extra spaces from summary table

* printing output file location

* Removed sudo in code and ACS flag

[ROCm/rccl commit: 4ba94d6662]
 2025-02-14 17:31:18 -06:00
Pedram Alizadeh 37f46f1669 reverting the (Reduce NPKit latency overhead in MSCCL kernel) PR #893 (#1525) 
[ROCm/rccl commit: 0e5f4d0662]
 2025-02-14 11:03:43 -05:00
corey-derochie-amd 30eecfdb25 Revert "replacing rccl_float8 with hip_fp8 and address compatibility issue (#…" (#1545) 
This reverts commit 9463a79dd9.

[ROCm/rccl commit: 824b81c034]
 2025-02-13 10:00:22 -07:00
mberenjk 9463a79dd9 replacing rccl_float8 with hip_fp8 and address compatibility issue (#1538) 
* replacing rccl_float8 with hip_fp8 and address compatibility issue with gfx942
---------

Co-authored-by: corey-derochie-amd <161367113+corey-derochie-amd@users.noreply.github.com>
Co-authored-by: Marzieh Berenjkoub <mberenjk@amd.com>

[ROCm/rccl commit: d437d6e41c]
 2025-02-13 10:34:17 -06:00
Wenkai Du 170cc1afde Print KL/CL/KE events for all warps (#1544) 
* Print KL/CL/KE events for all warps

* Fix count off-by-one issue

* Fix opCount in KE and restore CPU thread option

* Simplify count calculation

[ROCm/rccl commit: ebf7e2305e]
 2025-02-12 13:36:31 -08:00