Commit Graph
64512 Commits
Author SHA1 Message Date
Jonathan R. Madsen 27103d771b Install perfetto tools option (#58)
* Install perfetto tools option

- E.g. traced, perfetto, etc.

* Fix copying of perfetto directory

* Require curl for installing perfetto tools

* Fix to locating tools/ninja

[ROCm/rocprofiler-systems commit: 8146426e8b]
2022-05-11 15:05:09 -05:00
Jonathan R. Madsen 8146426e8b Install perfetto tools option (#58)
* Install perfetto tools option

- E.g. traced, perfetto, etc.

* Fix copying of perfetto directory

* Require curl for installing perfetto tools

* Fix to locating tools/ninja
2022-05-11 15:05:09 -05:00
Divya Shikre 231a61a394 Update get_frequencies to handle failures.
Show an optional debug log (RSMI_DEBUG_BITFIELD=2) to
the user in the following scenarios:
1. If more than one current frequency is found
2. If frequencies are not read in increasing order of
   their value
If current frequency is not available, index for it is
set to -1, values will not have * next to it in the
output. This will also be handled in rocm_smi.py.

Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: I477ec065f7513c8045d6392f12ef6cb835a6b8f6


[ROCm/rocm_smi_lib commit: afe996c2ed]
2022-05-11 15:33:15 -04:00
Divya Shikre afe996c2ed Update get_frequencies to handle failures.
Show an optional debug log (RSMI_DEBUG_BITFIELD=2) to
the user in the following scenarios:
1. If more than one current frequency is found
2. If frequencies are not read in increasing order of
   their value
If current frequency is not available, index for it is
set to -1, values will not have * next to it in the
output. This will also be handled in rocm_smi.py.

Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: I477ec065f7513c8045d6392f12ef6cb835a6b8f6
2022-05-11 15:33:15 -04:00
Ammar ELWazir 3df8bc7c77 Removing Backward compatability
removing the backward compatability file and making sure to use the right paths

Change-Id: I518d52c82e0c5878bd334713e7b1758bba79762d


[ROCm/roctracer commit: 6b16d37d65]
2022-05-11 14:43:35 -04:00
Ammar ELWazir 6b16d37d65 Removing Backward compatability
removing the backward compatability file and making sure to use the right paths

Change-Id: I518d52c82e0c5878bd334713e7b1758bba79762d
2022-05-11 14:43:35 -04:00
Graham Sider d5f074902c kfdtest: Update CMake message on LLVM build search
Old CMake message was misleading, update to reflect what it's actually
doing.

Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I8fd18a9095ea350cc10494b4382706027b447ba3
2022-05-11 13:39:49 -04:00
Graham Sider 27fed6c0d4 kfdtest: Update CMake message on LLVM build search
Old CMake message was misleading, update to reflect what it's actually
doing.

Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I8fd18a9095ea350cc10494b4382706027b447ba3


[ROCm/ROCR-Runtime commit: d5f074902c]
2022-05-11 13:39:49 -04:00
ROCm CI Service Account 0da08a8100 SWDEV-322620 - Virtual Memory Management (#2673)
Add device attribute for virtual memory management

Change-Id: Ie123412a7d1783f42d7c4274dcc3562a829a1e4c


[ROCm/hip commit: 96ec4ae295]
2022-05-11 10:31:15 -07:00
ROCm CI Service Account 96ec4ae295 SWDEV-322620 - Virtual Memory Management (#2673)
Add device attribute for virtual memory management

Change-Id: Ie123412a7d1783f42d7c4274dcc3562a829a1e4c
2022-05-11 10:31:15 -07:00
Satyanvesh Dittakavi a206abcaa1 SWDEV-335780 - Address OOM issue with BERT MLPerf
Memory free was not happening due to no flush

Change-Id: Iee9d2c9f8c50aa9b6012ca9d8fd792a51973b8c8


[ROCm/clr commit: f6e21144ae]
2022-05-11 13:03:32 -04:00
Satyanvesh Dittakavi f6e21144ae SWDEV-335780 - Address OOM issue with BERT MLPerf
Memory free was not happening due to no flush

Change-Id: Iee9d2c9f8c50aa9b6012ca9d8fd792a51973b8c8
2022-05-11 13:03:32 -04:00
German Andryeyev a7d73a753d SWDEV-311271 - Enable MGPU support for memory pool
Change-Id: I36850de282d62139b5cfe342df97bb1204cb7869


[ROCm/clr commit: 8b132beb92]
2022-05-11 11:04:24 -04:00
German Andryeyev 8b132beb92 SWDEV-311271 - Enable MGPU support for memory pool
Change-Id: I36850de282d62139b5cfe342df97bb1204cb7869
2022-05-11 11:04:24 -04:00
German Andryeyev 8624574866 SWDEV-336024 - Clear device heap to 0
The heap must be cleared once per device, but ROCclr doesn't
create a queue per device in HIP. Hence, the clear operation will
be performed during the first queue creation.

Change-Id: I52ceb06d67d11cde6d019c5ab510059f426a9bfb


[ROCm/clr commit: 04bfd93569]
2022-05-11 11:03:56 -04:00
German Andryeyev 04bfd93569 SWDEV-336024 - Clear device heap to 0
The heap must be cleared once per device, but ROCclr doesn't
create a queue per device in HIP. Hence, the clear operation will
be performed during the first queue creation.

Change-Id: I52ceb06d67d11cde6d019c5ab510059f426a9bfb
2022-05-11 11:03:56 -04:00
Divya Shikre 853a6e517c Add DEBUG_LOG macro
Add DEBUG_LOG that will optionally print error
message when RSMI_DEBUG_BITFIELD is set to 2.

Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: I6017e92d8a9e5f9861ae29ece0488d4bc198f996


[ROCm/rocm_smi_lib commit: 99be3451d7]
2022-05-11 11:03:24 -04:00
Divya Shikre 99be3451d7 Add DEBUG_LOG macro
Add DEBUG_LOG that will optionally print error
message when RSMI_DEBUG_BITFIELD is set to 2.

Signed-off-by: Divya Shikre <DivyaUday.Shikre@amd.com>
Change-Id: I6017e92d8a9e5f9861ae29ece0488d4bc198f996
2022-05-11 11:03:24 -04:00
Maneesh Gupta 43e1d22491 [CI] skip tests listed in config_amd_linux.json (#2674)
[ROCm/hip commit: 9628d0bda7]
2022-05-11 16:10:51 +05:30
Maneesh Gupta 9628d0bda7 [CI] skip tests listed in config_amd_linux.json (#2674) 2022-05-11 16:10:51 +05:30
ROCm CI Service Account f6a7172fe4 SWDEV-326794 - Document for hipMemoryType mismatch with CUDA (#2645)
Change-Id: Ib4349fc7446edac39ac09fefaaa4babfc7b81ca8

[ROCm/hip commit: 3786aa8518]
2022-05-11 12:36:38 +05:30
ROCm CI Service Account 3786aa8518 SWDEV-326794 - Document for hipMemoryType mismatch with CUDA (#2645)
Change-Id: Ib4349fc7446edac39ac09fefaaa4babfc7b81ca8
2022-05-11 12:36:38 +05:30
Chauncey Hui d63201c1d4 SWDEV-2 - Change OpenCL version number from 3462 to 3463
[ROCm/clr commit: 6c458c7534]
2022-05-11 03:00:08 -04:00
Chauncey Hui 6c458c7534 SWDEV-2 - Change OpenCL version number from 3462 to 3463 2022-05-11 03:00:08 -04:00
Ammar ELWazir 392e15598d Changing Installation docs
using build.sh rather than cmake in the readme

Change-Id: If3b80641497c0c967ec3340cb9ef546bf44824c3


[ROCm/roctracer commit: ed0e1f5cb8]
2022-05-11 01:31:52 -04:00
Ammar ELWazir ed0e1f5cb8 Changing Installation docs
using build.sh rather than cmake in the readme

Change-Id: If3b80641497c0c967ec3340cb9ef546bf44824c3
2022-05-11 01:31:52 -04:00
Ammar ELWazir dfe33f2c15 Changing the set CMAKE_CXX_FLAGS set for fPIC to known cmake ideal way
Change-Id: I898de3d05feffee2d7d37cf62ac33afe2ecde85a


[ROCm/roctracer commit: 7060b76927]
2022-05-10 22:38:13 -05:00
Ammar ELWazir 7060b76927 Changing the set CMAKE_CXX_FLAGS set for fPIC to known cmake ideal way
Change-Id: I898de3d05feffee2d7d37cf62ac33afe2ecde85a
2022-05-10 22:38:13 -05:00
Jonathan R. Madsen 57ef312d26 Option rename + minor fixes (#57)
- Set choices of OMNITRACE_BACKEND option
- rename OMNITRACE_SHMEM_SIZE_HINT_KB option
- rename OMNITRACE_BUFFER_SIZE_KB option
- rename OMNITRACE_COMBINE_PERFETTO_TRACES
- rename OMNITRACE_BACKEND option
- default to OMNITRACE_COLLAPSE_PROCESSES for combining perfetto traces
- OMNITRACE_PERFETTO_FILL_POLICY option
- fix unused variables due to constexpr in add_critical_trace
- rename perfetto config from "track_event" to "omnitrace"
- fix build-release.sh + python
- handle config file updating OMNITRACE_DL_VERBOSE in omnitrace-dl
- rename roctrace.cfg to omnitrace.cfg
- accept "on" and "off" for get_sampling_cpus()

[ROCm/rocprofiler-systems commit: 346f8cd0bc]
2022-05-10 17:30:45 -05:00
Jonathan R. Madsen 346f8cd0bc Option rename + minor fixes (#57)
- Set choices of OMNITRACE_BACKEND option
- rename OMNITRACE_SHMEM_SIZE_HINT_KB option
- rename OMNITRACE_BUFFER_SIZE_KB option
- rename OMNITRACE_COMBINE_PERFETTO_TRACES
- rename OMNITRACE_BACKEND option
- default to OMNITRACE_COLLAPSE_PROCESSES for combining perfetto traces
- OMNITRACE_PERFETTO_FILL_POLICY option
- fix unused variables due to constexpr in add_critical_trace
- rename perfetto config from "track_event" to "omnitrace"
- fix build-release.sh + python
- handle config file updating OMNITRACE_DL_VERBOSE in omnitrace-dl
- rename roctrace.cfg to omnitrace.cfg
- accept "on" and "off" for get_sampling_cpus()
2022-05-10 17:30:45 -05:00
Laurent Morichetti 12623a5f24 Fix the roctracer tests
14/15 tests pass, 1/15 intermittent failure (tool flushing test).

Change-Id: I36ed2900a1c51e584718993badeaefd48ad450a2


[ROCm/roctracer commit: a98476fe11]
2022-05-10 14:58:08 -07:00
Laurent Morichetti a98476fe11 Fix the roctracer tests
14/15 tests pass, 1/15 intermittent failure (tool flushing test).

Change-Id: I36ed2900a1c51e584718993badeaefd48ad450a2
2022-05-10 14:58:08 -07:00
Tao Sang a87da88ec3 SWDEV-319818 - Release restriction on hipResourceViewDesc
Let more types support hipResourceViewDesc.
Add missing symbols to fix cuda build failing.

Change-Id: Ife694cc6491427093863252e257e820b0bb4fa0f


[ROCm/hipother commit: 3fc1dca59f]
2022-05-10 15:57:31 -04:00
Tao Sang 3fc1dca59f SWDEV-319818 - Release restriction on hipResourceViewDesc
Let more types support hipResourceViewDesc.
Add missing symbols to fix cuda build failing.

Change-Id: Ife694cc6491427093863252e257e820b0bb4fa0f
2022-05-10 15:57:31 -04:00
Tao Sang 1a9a5f9bfc SWDEV-319818 - Release restriction on hipResourceViewDesc
Let more types support hipResourceViewDesc.
Add missing symbols to fix cuda build failing.

Change-Id: Ife694cc6491427093863252e257e820b0bb4fa0f


[ROCm/clr commit: a19b767e8a]
2022-05-10 15:57:31 -04:00
Tao Sang a19b767e8a SWDEV-319818 - Release restriction on hipResourceViewDesc
Let more types support hipResourceViewDesc.
Add missing symbols to fix cuda build failing.

Change-Id: Ife694cc6491427093863252e257e820b0bb4fa0f
2022-05-10 15:57:31 -04:00
Laurent Morichetti 196af97ad6 Disallow copying or moving trace buffers
Change-Id: I104b8240a76c6d96ae176b0b26bdc2e4e5e3c180


[ROCm/roctracer commit: 3f402eb6e9]
2022-05-10 12:08:06 -07:00
Laurent Morichetti 3f402eb6e9 Disallow copying or moving trace buffers
Change-Id: I104b8240a76c6d96ae176b0b26bdc2e4e5e3c180
2022-05-10 12:08:06 -07:00
Laurent Morichetti 9294225192 Fix memory leaks in roctracer
Each thread has a thread-local record_pair_stack. The stack is
dynamically allocated on first use, but is not detroyed when the
thread exits.

Replaced record_pair_stack pointers with record_pair_stack instances,
the intances are constructed on first odr-use, and destructed when the
thread exits.

Also, converted the cb_journal and act_journal to instances.

Change-Id: I186ac29da477f194880a1ab599f4be5715a23063


[ROCm/roctracer commit: 67481bd295]
2022-05-10 12:08:06 -07:00
Laurent Morichetti 67481bd295 Fix memory leaks in roctracer
Each thread has a thread-local record_pair_stack. The stack is
dynamically allocated on first use, but is not detroyed when the
thread exits.

Replaced record_pair_stack pointers with record_pair_stack instances,
the intances are constructed on first odr-use, and destructed when the
thread exits.

Also, converted the cb_journal and act_journal to instances.

Change-Id: I186ac29da477f194880a1ab599f4be5715a23063
2022-05-10 12:08:06 -07:00
Laurent Morichetti 4fddfcc5c5 Optimize rotcx markers
Improve the roctx markers performance when the tracer is not engaged
(the application is not running with rocprof).

The performance of roctx push/pop, measured with:

-----------------------------------------------------------------------
  auto start = std::chrono::steady_clock::now();
  for (int i = 0; i < 10000000; ++i) {
    roctxRangePush ("A");
    roctxRangePop ();
  }
  auto end = std::chrono::steady_clock::now();
  std::cout << "ns = " << std::chrono::nanoseconds(end - start).count()
      / 10000000 << std::endl;
-----------------------------------------------------------------------

w/o rocprof | with rocprof | commit
       92ns |       770ns  | 0d6e132: Cleanup CallbackTable::Get
       28ns |       712ns  | 6421bd5: Cleanup ROCTX's implementation
       20ns |       664ns  | 7f0e5e5: Remove the roctx range message...
        6ns |       665ns  | this commit

Change-Id: Id679dcbd0fb190a3179be98a9b2c1db151efee3d


[ROCm/roctracer commit: a794247c55]
2022-05-10 12:08:06 -07:00
Laurent Morichetti a794247c55 Optimize rotcx markers
Improve the roctx markers performance when the tracer is not engaged
(the application is not running with rocprof).

The performance of roctx push/pop, measured with:

-----------------------------------------------------------------------
  auto start = std::chrono::steady_clock::now();
  for (int i = 0; i < 10000000; ++i) {
    roctxRangePush ("A");
    roctxRangePop ();
  }
  auto end = std::chrono::steady_clock::now();
  std::cout << "ns = " << std::chrono::nanoseconds(end - start).count()
      / 10000000 << std::endl;
-----------------------------------------------------------------------

w/o rocprof | with rocprof | commit
       92ns |       770ns  | 0d6e132: Cleanup CallbackTable::Get
       28ns |       712ns  | 6421bd5: Cleanup ROCTX's implementation
       20ns |       664ns  | 7f0e5e5: Remove the roctx range message...
        6ns |       665ns  | this commit

Change-Id: Id679dcbd0fb190a3179be98a9b2c1db151efee3d
2022-05-10 12:08:06 -07:00
Laurent Morichetti 9b78c65ce1 Remove the roctx range message stack
The range message stack is mirrored in case ranges are pushed or popped
while tracing is stopped (by the tracer tool?). When a stop event is
reported, the tracer tool emits RangePop events by unwinding the stack,
then when the start event is reported, it emits RangePush events again
by unwinding the stack. The issue is that the RangePush events should
be emitted in reverse order.

For example:

RangePush(M1); RangePush(M2); \
  TracerStop; RangePop; RangePop; \
...; \
  TracerStart; RangePush(M2); RangePush(M1); \ <- In the wrong order
RangePop; RangePop;

It could be fixed by reversing the stack in RangeStackIterate but is it
worth it? The roctx range markers are supposed to be unintrusive so that
they can be left in the application even when it isn't being traced.

Simplifying the roctx API and reducing its added latency by removing
the range message stack mirroring seems like the better choise.

TODO: A future change should make roctx events immune to tracer start
and tracer stop requests. Or simply remove roctracer_start/stop.

Change-Id: Ie4d76afb5ce8d263848dcf1b599af394db56ddab


[ROCm/roctracer commit: 3d0198c395]
2022-05-10 12:08:06 -07:00
Laurent Morichetti 3d0198c395 Remove the roctx range message stack
The range message stack is mirrored in case ranges are pushed or popped
while tracing is stopped (by the tracer tool?). When a stop event is
reported, the tracer tool emits RangePop events by unwinding the stack,
then when the start event is reported, it emits RangePush events again
by unwinding the stack. The issue is that the RangePush events should
be emitted in reverse order.

For example:

RangePush(M1); RangePush(M2); \
  TracerStop; RangePop; RangePop; \
...; \
  TracerStart; RangePush(M2); RangePush(M1); \ <- In the wrong order
RangePop; RangePop;

It could be fixed by reversing the stack in RangeStackIterate but is it
worth it? The roctx range markers are supposed to be unintrusive so that
they can be left in the application even when it isn't being traced.

Simplifying the roctx API and reducing its added latency by removing
the range message stack mirroring seems like the better choise.

TODO: A future change should make roctx events immune to tracer start
and tracer stop requests. Or simply remove roctracer_start/stop.

Change-Id: Ie4d76afb5ce8d263848dcf1b599af394db56ddab
2022-05-10 12:08:06 -07:00
Laurent Morichetti 4a04400f85 Cleanup ROCTX's implementation
Remove thread_data_init. The C++ standard guarantees that the thread
local variable is initialized before its first odr-use and destructed
when the thread exits. Use a global initializer to set the reference
from the message stack instance in the map.

Remove roctracer_error_string. This does not belong to this library.
ROCTX does not expose errors to the application. The only functions
returning errors are returning -1 (Push/Pop).

Remove memory leaks due to strdup on the ranges messages. The memory
for the messages is guaranteed to be valid for the duration of the
callback, and it is the application's responsibility to strdup the
strings if it needs to extend the message's lifetime.

Add a lock to the RegisterApiCallback implementation. Iterating the
message stack map must be synchronized as a new thread could be adding
a new value to the map.

Change-Id: Iaf5b07ebc9efe4061cb01327d4c7034888727816


[ROCm/roctracer commit: 713db1fce5]
2022-05-10 12:08:06 -07:00
Laurent Morichetti 713db1fce5 Cleanup ROCTX's implementation
Remove thread_data_init. The C++ standard guarantees that the thread
local variable is initialized before its first odr-use and destructed
when the thread exits. Use a global initializer to set the reference
from the message stack instance in the map.

Remove roctracer_error_string. This does not belong to this library.
ROCTX does not expose errors to the application. The only functions
returning errors are returning -1 (Push/Pop).

Remove memory leaks due to strdup on the ranges messages. The memory
for the messages is guaranteed to be valid for the duration of the
callback, and it is the application's responsibility to strdup the
strings if it needs to extend the message's lifetime.

Add a lock to the RegisterApiCallback implementation. Iterating the
message stack map must be synchronized as a new thread could be adding
a new value to the map.

Change-Id: Iaf5b07ebc9efe4061cb01327d4c7034888727816
2022-05-10 12:08:06 -07:00
Laurent Morichetti bac7f1c162 Merge "Cleanup CallbackTable::Get" into amd-staging
[ROCm/roctracer commit: 6e4055503c]
2022-05-10 14:55:20 -04:00
Laurent Morichetti 6e4055503c Merge "Cleanup CallbackTable::Get" into amd-staging 2022-05-10 14:55:20 -04:00
Laurent Morichetti 08289f356a Merge "Remove unused open_output_file/close_output_file" into amd-staging
[ROCm/roctracer commit: e8909158b3]
2022-05-10 14:55:10 -04:00
Laurent Morichetti e8909158b3 Merge "Remove unused open_output_file/close_output_file" into amd-staging 2022-05-10 14:55:10 -04:00