Added a precision for AllGather and ReduceScatter sizes since NCCL uses the size per rank.
이 커밋은 다음에 포함됨:
1 부모
eb4c43ff3d
커밋
dcf818955f
1 파일이 변경됨
+4
@@ -78,6 +78,8 @@ And the Bus Bandwidth is therefore computed as :
|
||||
|
||||
`B = S/t * (n-1)/n = algbw * (n-1)/n`
|
||||
|
||||
Note that here, S is the size in bytes of the total array, which for NCCL is equal to `recvcount*sizeof(datatype)*n` as the `recvcount` argument is the count per rank.
|
||||
|
||||
### AllGather
|
||||
|
||||
The AllGather operation requires only to perform the assignation part of the allReduce operation :
|
||||
@@ -94,6 +96,8 @@ And the Bus Bandwidth is therefore computed as :
|
||||
|
||||
`B = S/t * (n-1)/n = algbw * (n-1)/n`
|
||||
|
||||
Note that here, S is the size in bytes of the total array, which for NCCL is equal to `sendcount*sizeof(datatype)*n` as the `sendcount` argument is the count per rank.
|
||||
|
||||
### Broadcast
|
||||
|
||||
The broadcast operation representation is similar to allGather :
|
||||
|
||||
새 이슈에서 참조
사용자 차단