76333 Υποβολές
Συγγραφέας SHA1 Μήνυμα Ημερομηνία
David Francis 47ddda3c6d kfdtest: Use scalar path for PollMemoryIsa Shader on gfx940
A gfx940 code path was erroneously added to this shader.

It's unneccesary; without this path, the shader uses
the scalar store, which works just fine on gfx940 without changes.

Remove it.

Signed-off-by: David Francis <David.Francis@amd.com>
Change-Id: I825cbbebbdb25c4a7c2f16e228c2bea6a6bcc30c


[ROCm/ROCR-Runtime commit: 2a01e5c33b]
2023-04-14 10:03:38 -04:00
Ori Messinger c234f84245 kfdtest: Update blacklist for Aqua Vanjanran
Signed-off-by: Ori Messinger <Ori.Messinger@amd.com>
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: David Francis <David.Francis@amd.com>
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Change-Id: I8f822bb71e8e5dbee6bdb62f77cbe5ea83faabb5
2023-04-14 10:03:38 -04:00
Ori Messinger 378f9999ed kfdtest: Update blacklist for Aqua Vanjanran
Signed-off-by: Ori Messinger <Ori.Messinger@amd.com>
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: David Francis <David.Francis@amd.com>
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Change-Id: I8f822bb71e8e5dbee6bdb62f77cbe5ea83faabb5


[ROCm/ROCR-Runtime commit: c234f84245]
2023-04-14 10:03:38 -04:00
David Francis 30da9a3cf9 kfdtest: Update shaders to compile on gfx940
gfx940 changed the semantics of the glc and slc coherency options
on vector stores and loads. This means that shaders that use
those bits no longer compile on gfx940.

Add precompilation if statements to those shaders to use the
new coherency bits.

Also add gfx940 to ASMTest so that compilation is tested.

Note: One of the tests enabled by this patch on gfx940,
KFDEvictTest.QueueTest, does not pass on gfx940 emulators.

Signed-off-by: David Francis <David.Francis@amd.com>
Change-Id: I942f9d2536e9eb5510c4d5af30df6ff1a95c8cf7
2023-04-14 10:03:38 -04:00
David Francis 78f489fb95 kfdtest: Update shaders to compile on gfx940
gfx940 changed the semantics of the glc and slc coherency options
on vector stores and loads. This means that shaders that use
those bits no longer compile on gfx940.

Add precompilation if statements to those shaders to use the
new coherency bits.

Also add gfx940 to ASMTest so that compilation is tested.

Note: One of the tests enabled by this patch on gfx940,
KFDEvictTest.QueueTest, does not pass on gfx940 emulators.

Signed-off-by: David Francis <David.Francis@amd.com>
Change-Id: I942f9d2536e9eb5510c4d5af30df6ff1a95c8cf7


[ROCm/ROCR-Runtime commit: 30da9a3cf9]
2023-04-14 10:03:38 -04:00
Graham Sider ae659e5427 libhsakmt: Fix queue destroy SVM path free size
Use q->total_mem_alloc_size for munmap in SVM codepath of free_queue.

Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I2fecaa1ddb337b1fe71f9cbba45a0c9467eff0c0
2023-04-14 10:03:38 -04:00
Graham Sider 543fe60c96 libhsakmt: Fix queue destroy SVM path free size
Use q->total_mem_alloc_size for munmap in SVM codepath of free_queue.

Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I2fecaa1ddb337b1fe71f9cbba45a0c9467eff0c0


[ROCm/ROCR-Runtime commit: ae659e5427]
2023-04-14 10:03:38 -04:00
Mukul Joshi a713fb766e libhsakmt: Fix memory leak on queue destroy for GFX9.4.3
Currently, on queue destroy, context save restore memory is freed
only for a single XCC. Instead, we need to free the entire context
save restore memory, which was allocated for all XCCs.

Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Change-Id: I51ebb12fa8d5ebed41979d68e74f7c5392dca062
2023-04-14 10:03:38 -04:00
Mukul Joshi dc0d800908 libhsakmt: Fix memory leak on queue destroy for GFX9.4.3
Currently, on queue destroy, context save restore memory is freed
only for a single XCC. Instead, we need to free the entire context
save restore memory, which was allocated for all XCCs.

Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Change-Id: I51ebb12fa8d5ebed41979d68e74f7c5392dca062


[ROCm/ROCR-Runtime commit: a713fb766e]
2023-04-14 10:03:38 -04:00
David Belanger 252a2cf959 libhsakmt: EOP Removal
Do not allocate the EOP buffer when not required.

Signed-off-by: David Belanger <david.belanger@amd.com>
Change-Id: I1664a3f0a882219a72278174006cdb8d46fd4f5e
2023-04-14 10:03:38 -04:00
David Belanger 7f91f54b27 libhsakmt: EOP Removal
Do not allocate the EOP buffer when not required.

Signed-off-by: David Belanger <david.belanger@amd.com>
Change-Id: I1664a3f0a882219a72278174006cdb8d46fd4f5e


[ROCm/ROCR-Runtime commit: 252a2cf959]
2023-04-14 10:03:38 -04:00
Mukul Joshi 8994c3ba0e kfdtest: Program COMPUTE_PGM_RSRC3 for GFX 9.4.3
Program ACCUM_OFFSET to match the number of VGPRS used
by the shader as part of Dispatch setup.

Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Change-Id: Icfa1fbe4de2a62f00743de567f3ed382d3378b17
2023-04-14 10:03:38 -04:00
Mukul Joshi c5b17ec68f kfdtest: Program COMPUTE_PGM_RSRC3 for GFX 9.4.3
Program ACCUM_OFFSET to match the number of VGPRS used
by the shader as part of Dispatch setup.

Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Change-Id: Icfa1fbe4de2a62f00743de567f3ed382d3378b17


[ROCm/ROCR-Runtime commit: 8994c3ba0e]
2023-04-14 10:03:38 -04:00
David Yat Sin f43a284b8e Change error reported when receiving code 128
We used to report HSA_STATUS_ERROR_INVALID_ISA when receiving error code
128, but there are several other reasons why we could be exceeding
number of VGPRs, so updating the error code.

Change-Id: I6a6980d5b07b09c93d00dee5207a0d52399bc77e
2023-04-14 09:12:07 -04:00
David Yat Sin 6812573d06 Change error reported when receiving code 128
We used to report HSA_STATUS_ERROR_INVALID_ISA when receiving error code
128, but there are several other reasons why we could be exceeding
number of VGPRs, so updating the error code.

Change-Id: I6a6980d5b07b09c93d00dee5207a0d52399bc77e


[ROCm/ROCR-Runtime commit: f43a284b8e]
2023-04-14 09:12:07 -04:00
gobhardw a56b32e4eb SWDEV-387560 rocprofiler changes for CI test scripts
Change-Id: Ie727424eae47502395e0779614bd5e863ed9a197
2023-04-14 16:19:15 +05:30
gobhardw 56437acbd1 SWDEV-387560 rocprofiler changes for CI test scripts
Change-Id: Ie727424eae47502395e0779614bd5e863ed9a197


[ROCm/rocprofiler commit: a56b32e4eb]
2023-04-14 16:19:15 +05:30
Satyanvesh Dittakavi 53dcdf2b46 SWDEV-393060 - Address directed tests build failures with CLR
Change-Id: I76b93ee073d21ec5fae0db5a129ada89f750855d


[ROCm/clr commit: a3104c60e1]
2023-04-14 03:53:51 -04:00
Satyanvesh Dittakavi a3104c60e1 SWDEV-393060 - Address directed tests build failures with CLR
Change-Id: I76b93ee073d21ec5fae0db5a129ada89f750855d
2023-04-14 03:53:51 -04:00
Hao Zhou e37e2d67c1 Merge amd-staging into amd-master 20230414
Signed-off-by: Hao Zhou <Hao.Zhou@amd.com>
Change-Id: I518af7182bb6537c9c03a30d53c44d2143f3064f


[ROCm/rocm_smi_lib commit: ecb1303732]
2023-04-14 12:17:58 +08:00
Hao Zhou ecb1303732 Merge amd-staging into amd-master 20230414
Signed-off-by: Hao Zhou <Hao.Zhou@amd.com>
Change-Id: I518af7182bb6537c9c03a30d53c44d2143f3064f
2023-04-14 12:17:58 +08:00
Jenkins 2ce7397ac0 Merge 'develop' into 'amd-staging'
Change-Id: I97da62d6b6267a604bc782299a739a21b672d113


[ROCm/hip-tests commit: 3c958a5022]
2023-04-13 23:10:48 +00:00
Jenkins 3c958a5022 Merge 'develop' into 'amd-staging'
Change-Id: I97da62d6b6267a604bc782299a739a21b672d113
2023-04-13 23:10:48 +00:00
Pedram Alizadeh aeffdf872b Disabled hipgraph tests! (#725)
[ROCm/rccl commit: 53c1c38f0e]
2023-04-13 17:42:05 -04:00
Pedram Alizadeh 53c1c38f0e Disabled hipgraph tests! (#725) 2023-04-13 17:42:05 -04:00
Rahul Garg d0f51600f7 SWDEV-391858 - Revert "SWDEV-373396 - OCLCreateBuffer: make the CPU checkResult() 4 times faster"
This reverts commit 50208c1a00.
Reason for revert: To fix ocltst issues.

Change-Id: Iac828e36e5288805be91f1b1efc7b95dda51b877


[ROCm/clr commit: eb5df70929]
2023-04-13 21:29:19 +00:00
Rahul Garg eb5df70929 SWDEV-391858 - Revert "SWDEV-373396 - OCLCreateBuffer: make the CPU checkResult() 4 times faster"
This reverts commit 6bf7514adb.
Reason for revert: To fix ocltst issues.

Change-Id: Iac828e36e5288805be91f1b1efc7b95dda51b877
2023-04-13 21:29:19 +00:00
Graham Sider fd48f14ceb libhsakmt: Update FD creation logic
In multi-partition modes, e.g. CPX, we want to create new file
descriptor despite using the same render node. Update
open_drm_render_device to use a gpu_id to fd map partitioned by render
node. Different gpu_id's requesting the same render node will be added
to that render node's map list for fetching its fd. Different gpu_id's
requesting different render nodes as well as the same gpu_id's
requesting the same render node will behave as they did previously.

Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: Ie153d42355d4d75b1c6ba6ff40fac3295bc87009
2023-04-13 15:25:09 -04:00
Graham Sider 89ce41694f libhsakmt: Update FD creation logic
In multi-partition modes, e.g. CPX, we want to create new file
descriptor despite using the same render node. Update
open_drm_render_device to use a gpu_id to fd map partitioned by render
node. Different gpu_id's requesting the same render node will be added
to that render node's map list for fetching its fd. Different gpu_id's
requesting different render nodes as well as the same gpu_id's
requesting the same render node will behave as they did previously.

Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: Ie153d42355d4d75b1c6ba6ff40fac3295bc87009


[ROCm/ROCR-Runtime commit: fd48f14ceb]
2023-04-13 15:25:09 -04:00
Mukul Joshi 97a669a979 libhsakmt: Update context save handling for multi XCC
Allocate debug area big enough for all XCCs in the partition. Also, fix
the cu_num calculations as driver now reports cu_num as the total number
of CUs in the partition.

Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Change-Id: I6e80d57196b770bb3c2506bc58cb366c0046084b
2023-04-13 15:25:09 -04:00
Mukul Joshi 0f7cfe5e4b libhsakmt: Update context save handling for multi XCC
Allocate debug area big enough for all XCCs in the partition. Also, fix
the cu_num calculations as driver now reports cu_num as the total number
of CUs in the partition.

Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Change-Id: I6e80d57196b770bb3c2506bc58cb366c0046084b


[ROCm/ROCR-Runtime commit: 97a669a979]
2023-04-13 15:25:09 -04:00
Graham Sider 6be4461a0d libhsakmt: Add Aqua Vanjaram support
Add gfx version for VGPR size per CU calc, add FAMILY_AV to KfdFamilyId,
add blacklist filter to kfdtest.exclude.

Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I9b8072e45f4d497e0a8fd3f8f97f1425238e8b42
2023-04-13 15:25:09 -04:00
Graham Sider ea4e2a82bb libhsakmt: Add Aqua Vanjaram support
Add gfx version for VGPR size per CU calc, add FAMILY_AV to KfdFamilyId,
add blacklist filter to kfdtest.exclude.

Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Change-Id: I9b8072e45f4d497e0a8fd3f8f97f1425238e8b42


[ROCm/ROCR-Runtime commit: 6be4461a0d]
2023-04-13 15:25:09 -04:00
Charis Poag d38ba30a0f [SWDEV-392571] Fix concise info when missing VRAM info
Updates:
    * [rocm-smi] Added larger app width size, which helps
      display missing device info
    * [rocm-smi] Added better context when rsmi_ret_ok
      does not return with RSMI_STATUS_SUCCESS
    * [rocm-smi] Removed all references to an
      undefined function (printLogNoDev())
    * [rocm-smi] Fixed not detecting non-int
      values when setting the voltage curve
    * [rocm-smi] Added better context on missing
      sysfs file when setting clock overdrive
      values
    * [rocm-smi] Fixed getMemInfo() calls not
      referencing tuple values (making it easier
      to read)
    * [rocm-smi] Silenced concise info spitting
      out errors for missing VRAM files, instead
      display which metric is "unsupported" if
      the files are missing
    * [rocm-smi] Updated function descriptions for
      rsmi_ret_ok & getMemInfo
    * [rocm-smi] Updated getMemInfo to provide a
      quiet call, to silence for concise info calls.
      This provides a way to keep the output clean.
    * [rocm-smi-lib] Added when using debug sysfs
      files, to state, which enums are enabled
      for debug

Change-Id: I0e9e0c97ccf71467ced0e1a1f71803327a8be2b7
Signed-off-by: Charis Poag <Charis.Poag@amd.com>


[ROCm/rocm_smi_lib commit: 6be92b9e26]
2023-04-13 15:11:35 -04:00
Charis Poag 6be92b9e26 [SWDEV-392571] Fix concise info when missing VRAM info
Updates:
    * [rocm-smi] Added larger app width size, which helps
      display missing device info
    * [rocm-smi] Added better context when rsmi_ret_ok
      does not return with RSMI_STATUS_SUCCESS
    * [rocm-smi] Removed all references to an
      undefined function (printLogNoDev())
    * [rocm-smi] Fixed not detecting non-int
      values when setting the voltage curve
    * [rocm-smi] Added better context on missing
      sysfs file when setting clock overdrive
      values
    * [rocm-smi] Fixed getMemInfo() calls not
      referencing tuple values (making it easier
      to read)
    * [rocm-smi] Silenced concise info spitting
      out errors for missing VRAM files, instead
      display which metric is "unsupported" if
      the files are missing
    * [rocm-smi] Updated function descriptions for
      rsmi_ret_ok & getMemInfo
    * [rocm-smi] Updated getMemInfo to provide a
      quiet call, to silence for concise info calls.
      This provides a way to keep the output clean.
    * [rocm-smi-lib] Added when using debug sysfs
      files, to state, which enums are enabled
      for debug

Change-Id: I0e9e0c97ccf71467ced0e1a1f71803327a8be2b7
Signed-off-by: Charis Poag <Charis.Poag@amd.com>
2023-04-13 15:11:35 -04:00
Charis Poag fc18ccd37a [SWDEV-392571] Fix concise info when missing VRAM info
Updates:
    * [rocm-smi] Added larger app width size, which helps
      display missing device info
    * [rocm-smi] Added better context when rsmi_ret_ok
      does not return with RSMI_STATUS_SUCCESS
    * [rocm-smi] Removed all references to an
      undefined function (printLogNoDev())
    * [rocm-smi] Fixed not detecting non-int
      values when setting the voltage curve
    * [rocm-smi] Added better context on missing
      sysfs file when setting clock overdrive
      values
    * [rocm-smi] Fixed getMemInfo() calls not
      referencing tuple values (making it easier
      to read)
    * [rocm-smi] Silenced concise info spitting
      out errors for missing VRAM files, instead
      display which metric is "unsupported" if
      the files are missing
    * [rocm-smi] Updated function descriptions for
      rsmi_ret_ok & getMemInfo
    * [rocm-smi] Updated getMemInfo to provide a
      quiet call, to silence for concise info calls.
      This provides a way to keep the output clean.
    * [rocm-smi-lib] Added when using debug sysfs
      files, to state, which enums are enabled
      for debug

Change-Id: I0e9e0c97ccf71467ced0e1a1f71803327a8be2b7
Signed-off-by: Charis Poag <Charis.Poag@amd.com>


[ROCm/amdsmi commit: 6be92b9e26]
2023-04-13 15:11:35 -04:00
Kaiming Ouyang 5eba5f178d Add a comment to shutdown() in ncclSocketClose
[ROCm/rccl commit: 006b6bc7dc]
2023-04-13 09:13:44 -07:00
Kaiming Ouyang 006b6bc7dc Add a comment to shutdown() in ncclSocketClose 2023-04-13 09:13:44 -07:00
Kaiming Ouyang e35b05a872 Shutdown socket before close in ncclSocketClose()
[ROCm/rccl commit: 367e9b61c3]
2023-04-13 09:11:52 -07:00
Kaiming Ouyang 367e9b61c3 Shutdown socket before close in ncclSocketClose() 2023-04-13 09:11:52 -07:00
Jonathan R. Madsen 557adea45a Linux Perf Support + Causal Profiling Updates (#276)
* causal backtrace updates

- fix initial causal sampling period value

* causal delay updates

- tweak handling of sleep_for_overhead

* Fix experiment global scaling for prog pts

- results in drastically improved predictions

* pthread_mutex_gotcha updates

- disable all wrappers during causal profiling

* validate-causal-json.py updates

- support decimal stddev
- fix setting stddev from command-line

* causal perform_experiment_impl update

- handle start failing because finalizing

* deprecate causal::component::sample_rate

- appears to not help at all

* Rework sample info

* Increase causal unwind_depth

- use OMNITRACE_MAX_UNWIND_DEPTH

* validate-causal-json updates

- min experiments
  - exclude reporting predictions with less than X experiments at a given speedup
- percent samples
  - only print samples within X% of the peak (default: 95%)

* Update timemory submodule

- extensions to sampling for signals delivered via non-timer method
  - e.g. via HW counter overflow

* dwarf_entry::operator< updates

- sort via file

* causal profiling docs updates

- info about backends
- info about installing/enabling perf

* config updates: causal backend

- CausalBackend enum
- OMNITRACE_CAUSAL_BACKEND: perf, timer, auto
- omnitrace-causal option: --backend

* debug update

- use spin_mutex instead of std::mutex

* address_range::contains update

- range from 0-100 contains range from 10-100 but was returning false because high was == 100 not < 100

* symbol::operator< update

- handle load address differences

* sampling updates (non-causal)

- update get_timer to get_trigger + dynamic_cast

* container::static_vector updates

- support construction from container::c_array
- update_size private member func for handling atomic m_size

* Move perf files

- moved library/causal/perf.{hpp,cpp} to library/perf.{hpp,cpp}

* causal example update

- created impl.hpp (forward decls)
- renamed {cpu,rng}_func_impl to {cpu,rng}_impl_func
- only create two threads which run N iterations instead of two threads each iteration

* Update timemory submodule

- updates to unwind::processed_entry
- updates to procfs::maps

* Updated causal documentation

- fixed line numbers changed by modifications to causal example

* omnitrace-causal exe updates

- set OMNITRACE_THREAD_POOL_SIZE to zero by default

* core/containers updates

- static_vector: provide data() member function
- c_array pop_front() and pop_back() member functions

* core: config and argparse updates + perf

- core/perf.{hpp,cpp}
  - forward decl of enums
  - config-related capabilities
- argparse: --sample-overflow
- renamed some config functions
  - e.g. get_sampling_cpu_freq -> get_sampling_cputime_freq
- added config settings related to overflow sampling via perf
- added timer_sampling and overflow_sampling categories

* Update timemory submodule

- sampling allocator flushing

* binary updates

- lookup_ipaddr_entry
- use bfd_find_nearest_line instead of bfd_find_nearest_line_discriminator
  - discriminators are not used
- explicit instantiations of inlined_symbol::serialize

* Bump VERSION to 1.10.0

* sampling and perf updates

- support overflow sampling via Linux Perf
- update perf namespace
- update perf::perf_event
  - update record ctor: pointer instead of const ref
  - update open member func: return optional string
  - add m_batch_size member variable
- sampling updates
  - support overflow sampling
  - flush allocators
  - increase buffer size from 1024 to 2048
  - restructure post-processing in light of perf overflow supports
  - improve offload memory usage only load buffers for thread
  - load_offload_buffer(tid) uses thread-specific filepos
- component updates
  - backtrace_metrics::operator-=
  - backtrace_metrics::operator-
  - backtrace::sample does not record for overflow signal
  - callchain: perf overflow sample

* core updates

- component::sampling_percent does not report self + uses_percent_units

* causal updates

- tweak get_line_info
- overloads for set_current_selection (uint64_t, c_array, std::array)
- delay
  - use sampling::pause/sampling::resume
- experiment
  - experiment::sample derives from unwind::processed_entry
  - experiment::samples is vector instead of set
  - fixed samples
  - overloads for is_selected (uint64_t, c_array, std::array)
  - scaling factor defaults to 100 instead of 50
  - serialize updates follow change to experiment::sample
  - modify algorithm for increasing/decreasing experiment length
- sample_data
  - use map<uintptr, uint64_t> instead of set<sample_data>
  - get_samples returns vector<sample_data> instead of set<sample_data>
- sampling
  - support overflow via Linux Perf
  - update causal_offload_buffer
  - flush sampling allocator
- backtrace
  - overflow component

* libomnitrace-dl updates

- handle dl::InstrumentMode::PythonProfile

* testing updates (causal)

- causal line 155 -> causal line 100
- causal line 165 -> causal line 110

* formatting

* exit_gotcha updates

- exit_info for abort()
- message about non-zero exit code

* testing updates

- fail regex for causal tests
- validate-causal-json: >= min_experiments instead of > min_experiments
- handle OMNITRACE_DEBUG_SETTINGS in omnitrace_write_test_config

* causal sampling updates

- add new lines where appropriate

* causal data updates

- reorder diagnostic info when experiment fails to start

* binary updates

- symbol address range from address to address + symsize + 1
  - add 1 based on debug info

* causal data updates

- sample_selection wait_ns defaults to 1,000 instead of 10,000
- sample_selection wait scaled by iteration number
- save_line_info_impl verbosity
- print latest_eligible_pc when experiment does not start

* causal sampling + component updates

- perf backend disables component::backtrace
- ensure get_sampling_(realtime|cputime|overflow)_signal do not malloc

* causal: remove period stats

* validate-causal-json update

- fix --help

* causal data updates

- improve eligible pc history reporting when experiment fails to start

* causal data updates

- fix compute_eligible_lines_impl
  - eligible address ranges returning too many ranges
  - occasionally, overwrite all *true* eligible address ranges

* causal data updates

- reduce scoped ranges to symbol ranges
- is_eligible_address() returns true contains (not just coarse)
- revert some sample_selection behavior

* binary address_multirange updates

- make coarse_range private
- fix operator+=(pair<coarse, uintptr_t>)

* causal example update

- fix nsync to default to once per iteration

* binary analysis updates

- tweak header file includes

* causal updates

- remove factoring in sleep_for_overhead
- invoke delay::process() even if experiment is not active

* causal data updates

- update latest_eligible_pc structure

* update omnitrace-install.py.in

- fix support for fedora
  - /etc/os-release does not have ID_LIKE
  - fallback to RHEL 8.7 if version not specified

* update omnitrace-install.py.in

- fix support for debian
  - /etc/os-release does not have ID_LIKE
  - version mapping

* Update documentation

- update docs on installation

* causal data and experiment updates

- data: reset_sample_selection

* causal set_current_selection debugging

- debug messages for failed e2e runs

* causal data and backtrace component updates

- data: set_current_selection returns the number of eligible addresses added
- backtrace: if cputime signal has selected zero IPs > 5x, then realtime signal starts contributing call-stacks

* core library updates

- move config::parse_numeric_range to utility namespace
- add core/utility.cpp
- support range:increment, e.g. 5-25:10 expands to '5 15 25' instead of '5 10 15 20 25'

* omnitrace-causal update

- end-to-end expands all speedups
- support range:increment in speedups

* causal backtrace updates

- remove select_ival (realtime signal always contributes when select_count == 0)

* containers: static_vector update

- explicit c_array constructor
- explicit std::array constructor

* causal data updates

- remove set_current_selection(uint64_t)
- remove set_current_selection(std::array)
- sample_selection increase default wait time
- report eligible PC candidates
- move reset_sample_selection to perform_experiment_impl
- decrease latest_eligible_pc array size
- set_current_selection does not guard for experiment::active

* core debug updates

- OMNITRACE_PRINT_COLOR macros

* causal data updates

- tweak to experiment never started message

* causal gotcha updates

- remove unused code

* critical trace updates

- remove unused code

* omnitrace-causal

- OMNITRACE_LAUNCHER

* causal data updates

- don't fail on end-to-end + omnitrace-causal

* causal backtrace updates

- reintroduce select_ival behavior

* causal data updates

- tweak verbose messages about number of PC candidates

* core mproc updates

- utilities for waiting on child PID and diagnosing status
  - omnitrace::mproc::wait_pid
  - omnitrace::mproc::diagnose_status

* omnitrace-run updates

- support --fork argument for executing via fork in current process + execvpe on child instead of execvpe in current process

* omnitrace-causal updates

- wait_pid and diagnose_status just call equivalent functions in omnitrace::mproc

* ubuntu-focal workflow update

- attempt to launch ubuntu-focal-codecov job with CAP_SYS_ADMIN and use perf backend

* tests reorg and updates

- remove binary-rewrite-sampling and runtime-instrument-sampling tests
- rename *-preload tests (which use omnitrace-sample exe) to *-sampling
- split tests/CMakeLists.txt into several tests/omnitrace-<category>-tests.cmake files
- tweak to causal-both-omni-func test
  - add args: -n 2 -b timer

* update validate-causal-json.py

- better reasoning info for adjusting tolerance
- always apply tolerance adjustments in CI mode

* causal e2e tests update

- add label "causal-e2e" label
- tweak params
  - old: 80 12 432525 500000000
  - new: 80 50 432525 100000000
- disable processor affinity for slow-func/line-100 tests
  - artificially inflates some speedups with perf

* unblocking_gotcha updates

- overload operator() according to gotcha function index

* blocking_gotcha updates

- overload operator() according to gotcha function index
- fix bug where potentially post block functors (e.g. pthread_mutex_trylock) throw error if lock is not acquired.

* parse_numeric_range update

- support unordered_set

* config update

- OMNITRACE_DEBUG_{TIDS,PIDS} use parse_numeric_range

[ROCm/rocprofiler-systems commit: 9de3a6b0b4]
2023-04-13 02:14:35 -05:00
Jonathan R. Madsen 9de3a6b0b4 Linux Perf Support + Causal Profiling Updates (#276)
* causal backtrace updates

- fix initial causal sampling period value

* causal delay updates

- tweak handling of sleep_for_overhead

* Fix experiment global scaling for prog pts

- results in drastically improved predictions

* pthread_mutex_gotcha updates

- disable all wrappers during causal profiling

* validate-causal-json.py updates

- support decimal stddev
- fix setting stddev from command-line

* causal perform_experiment_impl update

- handle start failing because finalizing

* deprecate causal::component::sample_rate

- appears to not help at all

* Rework sample info

* Increase causal unwind_depth

- use OMNITRACE_MAX_UNWIND_DEPTH

* validate-causal-json updates

- min experiments
  - exclude reporting predictions with less than X experiments at a given speedup
- percent samples
  - only print samples within X% of the peak (default: 95%)

* Update timemory submodule

- extensions to sampling for signals delivered via non-timer method
  - e.g. via HW counter overflow

* dwarf_entry::operator< updates

- sort via file

* causal profiling docs updates

- info about backends
- info about installing/enabling perf

* config updates: causal backend

- CausalBackend enum
- OMNITRACE_CAUSAL_BACKEND: perf, timer, auto
- omnitrace-causal option: --backend

* debug update

- use spin_mutex instead of std::mutex

* address_range::contains update

- range from 0-100 contains range from 10-100 but was returning false because high was == 100 not < 100

* symbol::operator< update

- handle load address differences

* sampling updates (non-causal)

- update get_timer to get_trigger + dynamic_cast

* container::static_vector updates

- support construction from container::c_array
- update_size private member func for handling atomic m_size

* Move perf files

- moved library/causal/perf.{hpp,cpp} to library/perf.{hpp,cpp}

* causal example update

- created impl.hpp (forward decls)
- renamed {cpu,rng}_func_impl to {cpu,rng}_impl_func
- only create two threads which run N iterations instead of two threads each iteration

* Update timemory submodule

- updates to unwind::processed_entry
- updates to procfs::maps

* Updated causal documentation

- fixed line numbers changed by modifications to causal example

* omnitrace-causal exe updates

- set OMNITRACE_THREAD_POOL_SIZE to zero by default

* core/containers updates

- static_vector: provide data() member function
- c_array pop_front() and pop_back() member functions

* core: config and argparse updates + perf

- core/perf.{hpp,cpp}
  - forward decl of enums
  - config-related capabilities
- argparse: --sample-overflow
- renamed some config functions
  - e.g. get_sampling_cpu_freq -> get_sampling_cputime_freq
- added config settings related to overflow sampling via perf
- added timer_sampling and overflow_sampling categories

* Update timemory submodule

- sampling allocator flushing

* binary updates

- lookup_ipaddr_entry
- use bfd_find_nearest_line instead of bfd_find_nearest_line_discriminator
  - discriminators are not used
- explicit instantiations of inlined_symbol::serialize

* Bump VERSION to 1.10.0

* sampling and perf updates

- support overflow sampling via Linux Perf
- update perf namespace
- update perf::perf_event
  - update record ctor: pointer instead of const ref
  - update open member func: return optional string
  - add m_batch_size member variable
- sampling updates
  - support overflow sampling
  - flush allocators
  - increase buffer size from 1024 to 2048
  - restructure post-processing in light of perf overflow supports
  - improve offload memory usage only load buffers for thread
  - load_offload_buffer(tid) uses thread-specific filepos
- component updates
  - backtrace_metrics::operator-=
  - backtrace_metrics::operator-
  - backtrace::sample does not record for overflow signal
  - callchain: perf overflow sample

* core updates

- component::sampling_percent does not report self + uses_percent_units

* causal updates

- tweak get_line_info
- overloads for set_current_selection (uint64_t, c_array, std::array)
- delay
  - use sampling::pause/sampling::resume
- experiment
  - experiment::sample derives from unwind::processed_entry
  - experiment::samples is vector instead of set
  - fixed samples
  - overloads for is_selected (uint64_t, c_array, std::array)
  - scaling factor defaults to 100 instead of 50
  - serialize updates follow change to experiment::sample
  - modify algorithm for increasing/decreasing experiment length
- sample_data
  - use map<uintptr, uint64_t> instead of set<sample_data>
  - get_samples returns vector<sample_data> instead of set<sample_data>
- sampling
  - support overflow via Linux Perf
  - update causal_offload_buffer
  - flush sampling allocator
- backtrace
  - overflow component

* libomnitrace-dl updates

- handle dl::InstrumentMode::PythonProfile

* testing updates (causal)

- causal line 155 -> causal line 100
- causal line 165 -> causal line 110

* formatting

* exit_gotcha updates

- exit_info for abort()
- message about non-zero exit code

* testing updates

- fail regex for causal tests
- validate-causal-json: >= min_experiments instead of > min_experiments
- handle OMNITRACE_DEBUG_SETTINGS in omnitrace_write_test_config

* causal sampling updates

- add new lines where appropriate

* causal data updates

- reorder diagnostic info when experiment fails to start

* binary updates

- symbol address range from address to address + symsize + 1
  - add 1 based on debug info

* causal data updates

- sample_selection wait_ns defaults to 1,000 instead of 10,000
- sample_selection wait scaled by iteration number
- save_line_info_impl verbosity
- print latest_eligible_pc when experiment does not start

* causal sampling + component updates

- perf backend disables component::backtrace
- ensure get_sampling_(realtime|cputime|overflow)_signal do not malloc

* causal: remove period stats

* validate-causal-json update

- fix --help

* causal data updates

- improve eligible pc history reporting when experiment fails to start

* causal data updates

- fix compute_eligible_lines_impl
  - eligible address ranges returning too many ranges
  - occasionally, overwrite all *true* eligible address ranges

* causal data updates

- reduce scoped ranges to symbol ranges
- is_eligible_address() returns true contains (not just coarse)
- revert some sample_selection behavior

* binary address_multirange updates

- make coarse_range private
- fix operator+=(pair<coarse, uintptr_t>)

* causal example update

- fix nsync to default to once per iteration

* binary analysis updates

- tweak header file includes

* causal updates

- remove factoring in sleep_for_overhead
- invoke delay::process() even if experiment is not active

* causal data updates

- update latest_eligible_pc structure

* update omnitrace-install.py.in

- fix support for fedora
  - /etc/os-release does not have ID_LIKE
  - fallback to RHEL 8.7 if version not specified

* update omnitrace-install.py.in

- fix support for debian
  - /etc/os-release does not have ID_LIKE
  - version mapping

* Update documentation

- update docs on installation

* causal data and experiment updates

- data: reset_sample_selection

* causal set_current_selection debugging

- debug messages for failed e2e runs

* causal data and backtrace component updates

- data: set_current_selection returns the number of eligible addresses added
- backtrace: if cputime signal has selected zero IPs > 5x, then realtime signal starts contributing call-stacks

* core library updates

- move config::parse_numeric_range to utility namespace
- add core/utility.cpp
- support range:increment, e.g. 5-25:10 expands to '5 15 25' instead of '5 10 15 20 25'

* omnitrace-causal update

- end-to-end expands all speedups
- support range:increment in speedups

* causal backtrace updates

- remove select_ival (realtime signal always contributes when select_count == 0)

* containers: static_vector update

- explicit c_array constructor
- explicit std::array constructor

* causal data updates

- remove set_current_selection(uint64_t)
- remove set_current_selection(std::array)
- sample_selection increase default wait time
- report eligible PC candidates
- move reset_sample_selection to perform_experiment_impl
- decrease latest_eligible_pc array size
- set_current_selection does not guard for experiment::active

* core debug updates

- OMNITRACE_PRINT_COLOR macros

* causal data updates

- tweak to experiment never started message

* causal gotcha updates

- remove unused code

* critical trace updates

- remove unused code

* omnitrace-causal

- OMNITRACE_LAUNCHER

* causal data updates

- don't fail on end-to-end + omnitrace-causal

* causal backtrace updates

- reintroduce select_ival behavior

* causal data updates

- tweak verbose messages about number of PC candidates

* core mproc updates

- utilities for waiting on child PID and diagnosing status
  - omnitrace::mproc::wait_pid
  - omnitrace::mproc::diagnose_status

* omnitrace-run updates

- support --fork argument for executing via fork in current process + execvpe on child instead of execvpe in current process

* omnitrace-causal updates

- wait_pid and diagnose_status just call equivalent functions in omnitrace::mproc

* ubuntu-focal workflow update

- attempt to launch ubuntu-focal-codecov job with CAP_SYS_ADMIN and use perf backend

* tests reorg and updates

- remove binary-rewrite-sampling and runtime-instrument-sampling tests
- rename *-preload tests (which use omnitrace-sample exe) to *-sampling
- split tests/CMakeLists.txt into several tests/omnitrace-<category>-tests.cmake files
- tweak to causal-both-omni-func test
  - add args: -n 2 -b timer

* update validate-causal-json.py

- better reasoning info for adjusting tolerance
- always apply tolerance adjustments in CI mode

* causal e2e tests update

- add label "causal-e2e" label
- tweak params
  - old: 80 12 432525 500000000
  - new: 80 50 432525 100000000
- disable processor affinity for slow-func/line-100 tests
  - artificially inflates some speedups with perf

* unblocking_gotcha updates

- overload operator() according to gotcha function index

* blocking_gotcha updates

- overload operator() according to gotcha function index
- fix bug where potentially post block functors (e.g. pthread_mutex_trylock) throw error if lock is not acquired.

* parse_numeric_range update

- support unordered_set

* config update

- OMNITRACE_DEBUG_{TIDS,PIDS} use parse_numeric_range
2023-04-13 02:14:35 -05:00
coleramos425 f0e316f2f6 Add kernel filter example to analyze mode docs
Signed-off-by: coleramos425 <colramos@amd.com>


[ROCm/rocprofiler-compute commit: 9c981cb6bf]
2023-04-12 16:24:57 -05:00
coleramos425 9c981cb6bf Add kernel filter example to analyze mode docs
Signed-off-by: coleramos425 <colramos@amd.com>
2023-04-12 16:24:57 -05:00
Rahul Garg e4c04014b5 SWDEV-393637 - Temporally disable failing tests (#236)
[ROCm/hip-tests commit: 87a4d9edb8]
2023-04-12 11:52:55 -07:00
Rahul Garg 87a4d9edb8 SWDEV-393637 - Temporally disable failing tests (#236) 2023-04-12 11:52:55 -07:00
Jenkins 7973950e2d Merge 'develop' into 'amd-staging'
Change-Id: Ia4e86a95f7437edb507e0a9c7dbd7b983578c175


[ROCm/hip-tests commit: a3b30998db]
2023-04-12 11:12:05 +00:00
Jenkins a3b30998db Merge 'develop' into 'amd-staging'
Change-Id: Ia4e86a95f7437edb507e0a9c7dbd7b983578c175
2023-04-12 11:12:05 +00:00
Jonathan R. MadsenκαιJoseSantosAMD 778e87c69c Casual Profiling GUI (#265)
* Long workload name layout fix (#269)

* changed layout to fit experiment names

- added span to show full name
- shortened name to fit dropwdown
- changed layout for added consistency

* layout Fixes

- refresh button is to the right
- header is more consistent across different width screens

* header layout update

- center div makes turns into multiple lines if not all items fit

* slight improvement for header/graph spacing

* Fixed refresh button shape and function

- moved find_causal_files to parser so that main and gui can access
- resized refresh width to allow for same shape across different screens

* all graphs now have the same width

- graphs now have same width

- chart headers start well below the header with filters

---------

Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>

* Causal GUI: Linting, synced y-range, remove unused imports/variables, and bug fixes (#274)

* GUI: python linting workflow

- runs flake8 on code in source/python/gui

* GUI: flake8 settings in source/python/gui/setup.cfg

- ignore E501 errors (line too long)
- ignore W503 errors (line break before binary operator)

* GUI: setup.py updates

- remove unused imports

* GUI: __main__.py updates

- stddev is float value
- remove unused imports
- effectively propagate the --stddev argument

* GUI: gui.py updates

- remove unused imports
- fix light mode
- sync initial y range of all causal plots
- fix error bars for causal data
- set x-ticks to 5
- set y-ticks to 10
- only display top 99% of samples
- separate global declarations and assignments to global values
- remove unused variable assignments
- fix mislabeled function_regex and exp_regex
- change if X == False to if not X

* GUI: header.py updates

- remove unused imports
- fix mislabeled function_regex and exp_regex

* GUI: parser.py updates

- add set_num_stddev function for manipulating global num_stddev value
- remove unused variables
- fix latency point object (duplicated __init__ function)
- fix handling latency in JSON
- fix formatting of validation format error message
- replace if X == False with if not X
- fix unused dataframe creation in add_latency
- fix flake8 do not assign lambda for name_wo_ext (use def)

* GUI: gui.py updates

- replace misnamed "func_list" with "experiment_list"
- replace misnamed "exp_list" with "progpt_list"

* GUI: fix python workflow

- quote python versions to avoid truncating 3.10 to 3.1

---------

Co-authored-by: JoseSantosAMD <87447437+JoseSantosAMD@users.noreply.github.com>

[ROCm/rocprofiler-systems commit: cc14b52584]
2023-04-11 23:36:24 -05:00
Jonathan R. MadsenκαιJoseSantosAMD cc14b52584 Casual Profiling GUI (#265)
* Long workload name layout fix (#269)

* changed layout to fit experiment names

- added span to show full name
- shortened name to fit dropwdown
- changed layout for added consistency

* layout Fixes

- refresh button is to the right
- header is more consistent across different width screens

* header layout update

- center div makes turns into multiple lines if not all items fit

* slight improvement for header/graph spacing

* Fixed refresh button shape and function

- moved find_causal_files to parser so that main and gui can access
- resized refresh width to allow for same shape across different screens

* all graphs now have the same width

- graphs now have same width

- chart headers start well below the header with filters

---------

Co-authored-by: Jonathan R. Madsen <jrmadsen@users.noreply.github.com>

* Causal GUI: Linting, synced y-range, remove unused imports/variables, and bug fixes (#274)

* GUI: python linting workflow

- runs flake8 on code in source/python/gui

* GUI: flake8 settings in source/python/gui/setup.cfg

- ignore E501 errors (line too long)
- ignore W503 errors (line break before binary operator)

* GUI: setup.py updates

- remove unused imports

* GUI: __main__.py updates

- stddev is float value
- remove unused imports
- effectively propagate the --stddev argument

* GUI: gui.py updates

- remove unused imports
- fix light mode
- sync initial y range of all causal plots
- fix error bars for causal data
- set x-ticks to 5
- set y-ticks to 10
- only display top 99% of samples
- separate global declarations and assignments to global values
- remove unused variable assignments
- fix mislabeled function_regex and exp_regex
- change if X == False to if not X

* GUI: header.py updates

- remove unused imports
- fix mislabeled function_regex and exp_regex

* GUI: parser.py updates

- add set_num_stddev function for manipulating global num_stddev value
- remove unused variables
- fix latency point object (duplicated __init__ function)
- fix handling latency in JSON
- fix formatting of validation format error message
- replace if X == False with if not X
- fix unused dataframe creation in add_latency
- fix flake8 do not assign lambda for name_wo_ext (use def)

* GUI: gui.py updates

- replace misnamed "func_list" with "experiment_list"
- replace misnamed "exp_list" with "progpt_list"

* GUI: fix python workflow

- quote python versions to avoid truncating 3.10 to 3.1

---------

Co-authored-by: JoseSantosAMD <87447437+JoseSantosAMD@users.noreply.github.com>
2023-04-11 23:36:24 -05:00