Commit Graph

72 Commits

Author SHA1 Message Date
ansurya df20c17f12 Fix texture tests, always pass row pitch to HSA API for linear layout images (#1785)
=> New ROCr calculates pitch as per HSA specification and addrlib is used to check whether HW can support that configuration. Hence few texture tests are failing with HSA_EXT_STATUS_ERROR_IMAGE_PITCH_UNSUPPORTED.

=> Determine pitch for linear images and always pass rowpitch to HSA API's.
2020-01-16 08:54:30 +05:30
Maneesh Gupta 00bd5d1cec Revert PRs that break ROCm builds (#1781)
Fixes SWDEV-218626 and SWDEV-218629

Changes:
- Revert "`static inline` in a header, just like excess sugar in a diet, causes bloat (#1692)"
   This reverts commit be70b9f7e7.
- Revert "Fix rocFFT build failure (#1777)"
   This reverts commit 753277422a.
2020-01-08 15:11:58 +05:30
Siu Chi Chan f4555c835a Detect when an explicit printf buffer flush is required (#1766)
* Detect when an explicit printf buffer flush is required
in a device/stream synchronization function.

* hip_module.cpp: add missing hc_am.hpp header
2020-01-07 09:06:38 -08:00
Alex Voicu 75a11330aa Fix late-coming issues. (#1724)
Implementation for hipMemcpyWithStream.
2019-12-23 19:11:24 +05:30
Alex Voicu be70b9f7e7 static inline in a header, just like excess sugar in a diet, causes bloat (#1692) 2019-12-23 19:09:38 +05:30
Sarbojit2019 153a959280 Revert [HIP] Fixed hipStreamAddCallback (#1674)
This reverts commit 45613311d7.
Addresses SWDEV#212675.
2019-11-20 11:55:46 +05:30
Sarbojit2019 45613311d7 [HIP] Fixed hipStreamAddCallback [SWDEV#165185] (#1425)
Fixed hipStreamAddCallback() as requested in SWDEV#165185
Added unit test to test the behavior
2019-11-07 13:18:12 +05:30
Jeff Daily 85080905c0 hipEventRecord only needs one lock; remove locked_eventIsReady 2019-11-06 15:56:32 +00:00
Jeff Daily 7986090d9c remove ihipEvent_t::refreshEventStatus(), new hipEventElapsedTime
This fixes possible races in hipEventElapsedTime.
2019-11-06 15:46:46 +00:00
Aryan Salmanpour 359dc79101 [hip] add support for implicit kernel argument for multi-grid sync (#1456)
* [hip] add support for implicit kernel argument for multi-grid sync

* modified code for calculating the prev_sum

* change the impCoopArg type to size_t

* add memory clean up

* launch init_gws and main kernels into two separate loops
2019-10-24 17:43:30 +05:30
satyanveshd 3d661e4706 Reimplement hipMemGetInfo (#1447)
Addresses SWDEV-136570. hipMemGetInfo changed to compute free memory based on information from kfd instead of relying on hc::am_tracker.
2019-10-01 12:40:36 +05:30
Jeff Daily 1eb3dbf065 consolidate thread local storage (#915)
* all thread local access now through single struct

* clean up old commented-out code, more use of GET_TLS()

* fewer calls to GET_TLS by passing tls as a funtion argument

* revert unnecessary change to printf

* fix failing tests due to TLS change

* fix merge conflicts in ihipOccupancyMaxActiveBlocksPerMultiprocessor
2019-08-05 09:51:02 +00:00
Jeff Daily f096a3239e remove stream locks where it is safe to do so 2019-07-22 17:38:51 +00:00
Aryan Salmanpour 999f45fc11 [hip] Move _criticalData of ihipStream_t class to private section and use criticalData() to access it (#1177) 2019-07-04 00:42:19 +00:00
Aryan Salmanpour 96dc74897d [hip] implement the hipExtLaunchMultiKernelMultiDevice API (#1165)
* [hip] implement the hipExtLaunchMultiKernelMultiDevice API

* add a guard to check the HCC version for acquire_locked_hsa_queue() API which was introdued in HCC for ROCm 2.5

* modified code based on the requested changes

* changes to lock all streams before launching kernels for each device and unlock them after the dispatches

* check each stream to be valid before starting to lock all the streams
2019-06-20 05:59:05 +05:30
Evgeny 19ca7a2a08 prof layer includes refactoring 2019-06-11 20:13:29 -05:00
Alex Voicu ccfb764a59 Add HIPRTC, glorious ersatz for NVRTC (#1097)
* Add ersatz for NVRTC.

* Fix extraneous paren and use correct namespace.

* Use lowerCamelCase (yuck, yuck) consistently.

* Link against FS when building hiprtc lib.

* Correctly mark Manipulators. Fix dual compile.

* Add unit tests. Extend HIT to accept linker options.

* Make sure the HIPRTC library is installed.

* Better logging. Try to auto-detect the target.

* Stop specifying the target explicitly.

* Add missing flavour of `hipModuleLaunchKernel`.

* Program was already destroyed.

* Don't use `--genco`. Fix mangled name trimming.

* Fix HIPRTC breakage due to upstream noise.

* [dtests] Replace RUN -> TEST in hiprtc tests

Change-Id: Ie499e92dfe4e5c94634b1c2b76cf52d241bcfea3

* [hit] Set HIP_PATH to HIP_ROOT_DIR for all tests

Change-Id: Ib0ad1f99bc71c03e363e055dd508a7a4a210680a
2019-05-16 18:28:54 +05:30
Jeff Daily 2b3037a6ea In hipFree, synchronize owner of memory (#1018)
* In hipFree, if memory is associated with a device, synchronize that device's streams.

This changes the behavior from synchronizing the currently set TLS device.

* All devices sync in hipFree for _appId=-1 case.

* Revert "All devices sync in hipFree for _appId=-1 case."

This reverts commit 1efb34d6a8426661e45bc5f763422a1147aeac10.

* add HIP_SYNC_FREE env var
2019-04-16 08:35:55 +05:30
Yaxun Sam Liu 98b9e92908 hip-clang: fix kernel not found on multi-gpu
__hipRegisterFunction is called during by .init functions during program initialization.
It calls hipModuleGetFunction to locate kernel symbol in code objects. hipModuleGetFunction
assumes current device when locating kernel symbols. This works for HCC but not for hip-clang,
since hip-clang needs to locate kernel symbols for different devices without switching
between devices.

This patch introduces a new hsa agent parameter to ihipModuleGetFunction, which allows
__hipRegisterFunction to choose the correct hsa agent when locating kernel symbols. By
default it uses this_agent(), therefore this patch has no impact on HCC.
2019-03-31 10:08:20 -04:00
Maneesh Gupta eb03d50de9 Merge pull request #962 from gargrahul/add_2d_copy_fallback
Add 2D fallback to use copy kernel
2019-03-25 07:46:43 +00:00
Siu Chi Chan 24d08beef8 reimplement HIP_INIT as hip_impl::hip_init(), add hip_init() to some of the inlined API (#966)
* reimplement HIP_INIT as a function, expose it as hip_impl::hip_init()
so that it could be called from hipLaunchKernelGGL and other inlined
HIP functions

* Don't call hip_init from ihipPreLaunchKernel
2019-03-20 05:11:15 +00:00
Rahul Garg 918d7e3a40 Add 2D fallback to use copy kernel 2019-03-14 13:03:06 +05:30
Wen-Heng (Jack) Chung b4d658a48f Introduce hash key to HIP module implementation
A hash calculated via FNV-1a algorithm is introduced in ihipModule_t, the
internal of hipModule_t. The hash is used by HIP module APIs such as

- read_agent_global_from_module

to determine whether the agent-scope globals for a module have been iterated.

This commit fixes one issue that applications which load / unload modules
frequently would occasionally fail. After deep investigation of the issue it
turns out the old implementation in read_agent_global_from_module uses
hipModule_t as the key, which is not robust enough, as hipModule_t instances
are allocated dynamically so there are cases that one memory address may be
used by multiple hipModule_t instances. The real solution is to introduce a
uniquely identifiable hash for the code object associated with the HIP module.
And that's the rationale behind this commit.
2019-01-08 17:33:40 +00:00
Maneesh Gupta 0dd26b4f63 Merge pull request #608 from gargrahul/add_pinned_2d_sdma_copy
Added support for pinned 2D SDMA copy
2018-12-12 07:44:16 +05:30
Maneesh Gupta 7ce082415b Merge pull request #773 from fronteer/master
Support of printing process ID for HIP tracing
2018-11-23 11:16:22 +05:30
Qianfeng Zhang 81cf7cabfa Add support of printing process ID for HIP Tracing 2018-11-22 18:58:06 +08:00
Evgeny e5ba097afd renaming HIP_INIT_CB_API to HIP_INIT_API 2018-11-13 15:33:26 +00:00
Evgeny b8b1637ef7 adding activity prof layer 2018-11-13 15:33:26 +00:00
Yaxun Sam Liu f5d8842f6a Add HIP_DUMP_CODE_OBJECT 2018-10-26 14:14:00 -04:00
Yaxun Sam Liu 1299b65e15 Add HIP_DB=fatbin for debugging fat binary issues 2018-08-17 11:53:45 -04:00
Rahul Garg 1e57764378 Added support for pinned 2D SDMA copy 2018-07-31 14:05:35 +05:30
Maneesh Gupta 0c2f985553 Update hip_hcc_internal.h
Adding missing include for hip_hcc_internal in order to build with HCC
2018-07-04 09:33:51 +05:30
Siu Chi Chan a1f3b587fb remove the _ready flag in ihipStreamCallback_t and the mutex that protects it. 2018-06-04 17:29:04 -04:00
Laurent Morichetti 0c1893ea62 Add support for clang offload bundles and <<<>>> kernel launch. 2018-04-04 13:32:33 -07:00
Maneesh Gupta 1ba06f63c4 Apply .clangformat to all repo source files
Change-Id: I7e79c6058f0303f9a98911e3b7dd2e8596079344
2018-03-12 11:29:03 +05:30
Maneesh Gupta cebb070d30 Implement hipStreamAddCallback
Change-Id: Ib851e4d86ba9c8406ca37b88162ea483ccbc9d36
2017-12-19 16:06:14 +05:30
Alex Voicu 5ca3176a91 Re-sync with upstream. 2017-12-03 23:14:58 +00:00
Alex Voicu 5127ce67e8 This is primarily intended as an additional cleanup of the module functionality, in the aftermath of adopting module based dispatch. The main effort was associated with refactoring the questionable ihipModuleGetSymbol. It was quaintly written and misleading, in that it had little to do with getting symbols, and was exactly retrieving a kernel object. Error handling is modified so as to reduce branching depth. Functions which serve as interfaces to the HSA RT are moved in a separate helper header. Code object readers are properly deleted. Some leftover dead functionality pertaining to associating namespace scope variables with their allocated memory is removed. Executable loading is changed to use a string which holds the ELF image of the code object being loaded, thus avoiding some corner cases where using a istream would fail. 2017-12-03 23:09:06 +00:00
Ben Sander 9bba97fdcc Fix some cppcheck style issues. 2017-12-01 20:45:34 +00:00
Ben Sander 4313686d6e Fix warning from default cppchek. 2017-12-01 20:45:33 +00:00
Alex Voicu dc67ca3feb Merge remote-tracking branch 'origin/master' into feature_use_module_based_dispatch_instead_of_pfe
# Conflicts:
#	src/hip_module.cpp
2017-11-28 17:29:11 +00:00
Rahul Garg f97c5f9a64 -Moved coGlobals in hipModule class (takes care of multi module case)
-Used mutex scope for updating coGlobals
2017-11-20 16:23:18 +05:30
Alex Voicu 3d248927e4 Merge remote-tracking branch 'origin/master' into feature_use_module_based_dispatch_instead_of_pfe
# Conflicts:
#	tests/src/runtimeApi/stream/hipStreamSync2.cpp
2017-11-08 10:26:30 +00:00
Alex Voicu adaf6b8dff Merge remote-tracking branch 'origin/master' into feature_use_module_based_dispatch_instead_of_pfe 2017-11-07 00:01:22 +00:00
Ben Sander 4a2e6f8955 Make hipEvent_t thread safe.
Support re-recording of same event by different threads.

- Add criticalData structure to hipEvent_t, similar to mechanism used
  for streams, contexts, device.  Events are always locked
  after streams to avoid deadlock.
- ihipEvent_t::locked_copyCrit can be used to copy critical state
  including marker.  The critical state in the event can then
  be re-recorded.
- refactor hipEventElapsedTime.  Remmove stale debug code, native signal
  refs.
2017-11-06 23:49:25 +00:00
Maneesh Gupta a62d5aa875 Merge pull request #249 from bensander/warn_event
Add HIP_DB=warn + message if sync on dangerous event.
2017-11-06 07:25:40 +05:30
Alex Voicu bb1176001f Merge remote-tracking branch 'origin/master' into feature_use_module_based_dispatch_instead_of_pfe
# Conflicts:
#	src/hip_module.cpp
2017-11-03 10:53:39 +00:00
Ben Sander 09d866a639 Merge pull request #237 from bensander/use_ctxptr_for_p2p
Use ctxptr for p2p
2017-11-01 18:55:25 +01:00
Ben Sander 70c25bdf8e Add HIP_DB=warn + message if sync on dangerous event. 2017-11-01 10:44:34 -07:00
Alex Voicu c2482d1255 This switches HIP from its currently convoluted macro + pfe based dispatch mechanism to a more natural one partially based on the existing module API. The basic idea is that HCC will always correctly emit __global__ functions: as empty-bodied stubs, on host, and as kernels, on device. It then becomes trivial to obtain the mangled name on host, at dispatch, from the function's address, and then to use the mangled name to retrieve the kernel. This should address all problems stemming from serialisation, dubious mismatches due to the manufactured functor, macro-isms et al. It also immediately enables support for generalised globals as a consequence of that being available in the module API. Finally, it will make debug much easier, since the actual names of the __global__ functions will automatically be used in traces etc. One detail is that due to how dispatch works now (hipLaunchKernel and hipLaunchKernelGGL are themselves variadic function templates which deduce the function type of the callee), in certain cases it may be necesssary to insert explicit casts to ensure that the variadic argument list selects a viable overload - this can be observed in some unit tests. Eventually we may be able to remove this limitation, but for now it does not appear terribly onerous. The code is not extremely HIPpie, nor is it fully optimised, but rather is intended as a starting point for the HIP team to make its own. 2017-11-01 15:09:59 +00:00