Commit Graph

933 Commits

Author SHA1 Message Date
Siu Chi Chan 304a1e2dbe move executable_cache into program_state.cpp
[ROCm/clr commit: 1a2d332e76]
2019-05-24 17:27:25 -04:00
Siu Chi Chan 305eb4239e remove executables() from program_state
[ROCm/clr commit: e2c0122892]
2019-05-24 17:27:01 -04:00
Siu Chi Chan 0cae3e06c1 moving agent_globals_impl into hip_module
[ROCm/clr commit: 6852be819f]
2019-05-24 16:43:38 -04:00
Evgeny Mankov 204043c6e0 [HIP][HIPIFY] Make hipMemcpyParam2D coherent with cuMemcpy2D
+ Makes hip_Memcpy2D struct compatible with CUDA_MEMCPY2D struct
+ Add hipMemcpyParam2D support in nvcc fallback path
+ Update hipify-clang, tests and docs accordingly


[ROCm/clr commit: 9cb3e9aa5e]
2019-05-22 18:31:39 +03:00
Alex Voicu a4a3132c64 Add HIPRTC, glorious ersatz for NVRTC (#1097)
* Add ersatz for NVRTC.

* Fix extraneous paren and use correct namespace.

* Use lowerCamelCase (yuck, yuck) consistently.

* Link against FS when building hiprtc lib.

* Correctly mark Manipulators. Fix dual compile.

* Add unit tests. Extend HIT to accept linker options.

* Make sure the HIPRTC library is installed.

* Better logging. Try to auto-detect the target.

* Stop specifying the target explicitly.

* Add missing flavour of `hipModuleLaunchKernel`.

* Program was already destroyed.

* Don't use `--genco`. Fix mangled name trimming.

* Fix HIPRTC breakage due to upstream noise.

* [dtests] Replace RUN -> TEST in hiprtc tests

Change-Id: Ie499e92dfe4e5c94634b1c2b76cf52d241bcfea3

* [hit] Set HIP_PATH to HIP_ROOT_DIR for all tests

Change-Id: Ib0ad1f99bc71c03e363e055dd508a7a4a210680a


[ROCm/clr commit: a538eb705a]
2019-05-16 18:28:54 +05:30
Wenkai Du 3d75b10e0b Use NUMA distance for hop count calculation
[ROCm/clr commit: 56d2dc0022]
2019-05-15 21:50:35 +00:00
Maneesh Gupta e0e30536e6 Merge pull request #1083 from gargrahul/fix_hip_impl_visible_agents
Maintain HIP_VISIBLE_DEVICES for kernel launch

[ROCm/clr commit: c9fdb42b91]
2019-05-13 14:20:18 +05:30
Rahul Garg d44e800a17 Add fine grained host memory lock support (#1095)
* Add fine grained host memory lock support

* Fix default flag check


[ROCm/clr commit: e1f3dc0c80]
2019-05-13 11:48:26 +05:30
Siu Chi Chan 76f535b4ce migrate program_state logic from header into shared library (phase I) (#1077)
* Revert "Revert "Use COMgr to read Kernel Args Metadata (#1006)""

This reverts commit f8d108a815.

* Revert "Use COMgr to read Kernel Args Metadata (#1006)"

This reverts commit 10048a5631.

* Revert "improve program state commentary"

This reverts commit 5233d41c6c.

* Revert "load program state once per agent"

This reverts commit 9cee2c5311.

* start moving function_names() into the hip shared lib

* start moving code_object_blobs to a new "state" object

* Consolidate various program state related static objects into a
single program_state object

* minor clean up

* move more stuffs from functional_grid_launch into program_state

* debug make_kernarg

* moving lookup for kernargs size_align into program_state

* clean up old code for kernarg size and alignment

* update hip_module to use newer api in program_state

* Create public member functions for program_state

* move most program state functions into shared library

* Pass the data buffer size to load_executable
Otherwise, it can't figure what the data size is
just from the char* (since the data is not really a string)

* turning free functions in program state into members of program_state_impl

* change the free function globals() into a member of program_state_impl

* replace the static mutex used for populating globals

* moving associate_code_object_symbols_with_host_allocation into
program_state_impl

* move load_code_object_and_freeze_executable into program_state_impl

* moving executables and functions_names into program_state_impl

* moving kernels() into program_state_impl

* moving functions() into program_state_impl

* move get_kernargs into program_state_impl

* moving kernel_descriptor into program_state_impl

* moving kernargs_size_align calculation into program_state_impl

* Changing the handle to program_state_impl to a pointer

* moving program_state_impl into a separate inline source file

* fixing/cleaning up some header file includes

* moving member function for kernargs_size_align into program_state.cpp

* moving Kernel_descriptor into program_state.inl

* add a new class to manage agent globals

* moving all agent globals processing functions into agent_globals_impl

* load program state once per agent

re-merging PR991 against other program state changes

* fix per-agent program state member initialization

* cache executables based on elf name, isa, and agent.

This avoids program state reloading executables after a shared library is dlopened.

re-merging PR1057 against other program state changes

* protect executables cache by a global mutex

* return ref to executables cache

* adapt PR#981 Make hipModuleGetGlobal be in HIP runtime


[ROCm/clr commit: 05a1b696da]
2019-05-12 19:24:03 +05:30
Maneesh Gupta 30c7ed3e28 Merge pull request #1081 from mangupta/swdev-181624
Implement hipExtGetLinkTypeAndHopCount for ROCm devices

[ROCm/clr commit: c6c5e4cee8]
2019-05-07 16:15:41 +05:30
wkwchau 7eaaf6f1ae Return hipErrorInsufficientDriver status when CPU device not found (#1064)
* Return hipErrorInsufficientDriver status when CPU device not found - no exception thrown

* Return hipErrorInsufficientDriver status when CPU device not found


[ROCm/clr commit: ebf986dcee]
2019-05-07 15:58:25 +05:30
Rahul Garg 3f65bec096 Maintain HIP_VISIBLE_DEVICES for kernel launch
[ROCm/clr commit: 3be54a903c]
2019-05-07 05:09:02 +05:30
Maneesh Gupta f657eba4a5 Implement hipExtGetLinkTypeAndHopCount for ROCm devices
Change-Id: Ie5bb4f640ac6d189c7fceeab22627a7494fd10bd


[ROCm/clr commit: 2f43f110d9]
2019-05-06 15:54:31 +05:30
Sameer Sahasrabuddhe 4f69390332 minor cleanup: eliminate repetition
[ROCm/clr commit: c74a97f756]
2019-04-25 20:41:16 +05:30
Rahul Garg d69edbbb7f Add hipMallocManaged default functional support (#1036)
* Add hipMallocManaged default functional support

* Fix build error

* Add dtest


[ROCm/clr commit: 94769fc8dd]
2019-04-24 16:50:03 +05:30
Yaxun (Sam) Liu cb81018121 Fix missing arg in HIP_INIT_API
[ROCm/clr commit: 710e633bdd]
2019-04-18 16:18:31 -04:00
Maneesh Gupta dac817873f Merge pull request #1019 from scchan/lazy_binding
minor workaround for lazy binding

[ROCm/clr commit: 22660bed74]
2019-04-16 08:36:10 +05:30
Jeff Daily cf4e198a91 In hipFree, synchronize owner of memory (#1018)
* In hipFree, if memory is associated with a device, synchronize that device's streams.

This changes the behavior from synchronizing the currently set TLS device.

* All devices sync in hipFree for _appId=-1 case.

* Revert "All devices sync in hipFree for _appId=-1 case."

This reverts commit 1efb34d6a8426661e45bc5f763422a1147aeac10.

* add HIP_SYNC_FREE env var


[ROCm/clr commit: cf8fb43e6b]
2019-04-16 08:35:55 +05:30
Yaxun (Sam) Liu d8acabf24c Fix regression on multi-gpu due to PR#997
[ROCm/clr commit: 5c67ee11f4]
2019-04-05 22:54:41 -04:00
Siu Chi Chan f6837a4e7f minor workaround for lazy binding
[ROCm/clr commit: b5045af7e9]
2019-04-02 17:28:06 -04:00
Yaxun Sam Liu 12ac74bad1 hip-clang: fix kernel not found on multi-gpu
__hipRegisterFunction is called during by .init functions during program initialization.
It calls hipModuleGetFunction to locate kernel symbol in code objects. hipModuleGetFunction
assumes current device when locating kernel symbols. This works for HCC but not for hip-clang,
since hip-clang needs to locate kernel symbols for different devices without switching
between devices.

This patch introduces a new hsa agent parameter to ihipModuleGetFunction, which allows
__hipRegisterFunction to choose the correct hsa agent when locating kernel symbols. By
default it uses this_agent(), therefore this patch has no impact on HCC.


[ROCm/clr commit: 8f5c812a68]
2019-03-31 10:08:20 -04:00
Wen-Heng (Jack) Chung cfe930f9d6 Make hipModuleGetGlobal be in HIP runtime so it can be discovered at runtime (#981)
* Make hipModuleGetGlobal be in HIP runtime so it can be discovered at runtime

In HIP PR #929, quite a few HIP public APIs were made as inline functions with
hidden visibility. It was necessary to support applications with shared
libraries with GPU kernels launched via hipLaunchKernelGGL(), after HIP runtime
is initialized.

In empirical tests, the implementation has been proved to be a bit too
excessive, especially for hipModuleGetGlobal(). The function is used by another
type of client applications which relies on the existence of this function
within HIP runtime so global symbols from HSA code objects loaded dynamically
at runtime can be retrieved programmtically.

This commit moves hipModuleGetGlobal() back to src/hip_module.cpp, and makes it
visible and not inline, to fulfill requirements for applications
aforementioned. It does not change the behavior of applications depending on
hipLaunchKernelGGL().

* Add HIP_INIT_API into the implementation of hipModuleGetGlobal

Address review comments.

* Fix failing HIP unit tests


[ROCm/clr commit: 04915cea2f]
2019-03-29 03:45:04 +00:00
Maneesh Gupta d99bc4c540 Merge pull request #992 from gargrahul/handle_d2d_memcpy2d
Handle D2D in memcpy2D

[ROCm/clr commit: f9f4cee347]
2019-03-28 04:41:36 +00:00
Rahul Garg 73bb9a74bb Handle D2D in memcpy2D
[ROCm/clr commit: 50d623981e]
2019-03-28 02:21:45 +05:30
Jeff Daily 9cee2c5311 load program state once per agent
[ROCm/clr commit: 2845b4c4b8]
2019-03-27 18:19:10 +00:00
Rahul Garg 0d47ae4203 Let hipHostMalloc always share/map pinned host ptr
[ROCm/clr commit: 9b38380c03]
2019-03-26 10:19:13 +05:30
Rahul Garg 21d7bbab11 Avoid double mapping of devices to hostMalloc buffer
[ROCm/clr commit: ad11972f47]
2019-03-25 23:07:05 +05:30
Maneesh Gupta 817e064745 Merge pull request #970 from mangupta/swdev-172995
hipExtMallocWithFlags implementation

[ROCm/clr commit: c20d233585]
2019-03-25 07:46:53 +00:00
Maneesh Gupta 9e2774e81e Merge pull request #962 from gargrahul/add_2d_copy_fallback
Add 2D fallback to use copy kernel

[ROCm/clr commit: 9de28dfa5a]
2019-03-25 07:46:43 +00:00
Rahul Garg 66ce9921d5 2D Fallback needs hcc workweek 19101 or higher
[ROCm/clr commit: bec3995700]
2019-03-25 12:07:28 +05:30
Maneesh Gupta 505fc1e98c hipExtMallocWithFlags needs hcc workweek 19115 or higher
[ROCm/clr commit: 45255ab492]
2019-03-25 11:41:20 +05:30
Siu Chi Chan 597c06b6be reimplement HIP_INIT as hip_impl::hip_init(), add hip_init() to some of the inlined API (#966)
* reimplement HIP_INIT as a function, expose it as hip_impl::hip_init()
so that it could be called from hipLaunchKernelGGL and other inlined
HIP functions

* Don't call hip_init from ihipPreLaunchKernel


[ROCm/clr commit: fa9495841b]
2019-03-20 05:11:15 +00:00
Maneesh Gupta f1d064562d hipExtMallocWithFlags implementation
Change-Id: Iee9e119796472200b2933d5e23be60813f33bc75


[ROCm/clr commit: e44de376f7]
2019-03-19 11:59:22 +05:30
Evgeny a0c8ef2e96 tracing callback layer update
[ROCm/clr commit: 2aa88a4505]
2019-03-14 22:43:52 -05:00
Rahul Garg da6653482d Add 2D fallback to use copy kernel
[ROCm/clr commit: af72cde0a1]
2019-03-14 13:03:06 +05:30
Siu Chi Chan f7817c48d5 move triple_to_hsa_isa into the header
[ROCm/clr commit: 30053f6443]
2019-03-11 19:51:44 +00:00
Siu Chi Chan 3951fccb4e Fix memory leak introduced by previous change to Agent_global.
Make Agent_global manage the lifetime of the name string


[ROCm/clr commit: fa564a5345]
2019-03-11 19:51:32 +00:00
Aaron Enye Shi 3562ddf981 Fix Agent_global variables failing hipTestDeviceSymbol
Issue: Header uses std::vector<Agent_global> agent_globals which is created by hip_module.cpp
  - Move iterator fails to copy Agent_global from library source into header version
  - Due to different versions of std::string name in struct Agent_global
Fix: Change Agent_global to use char* name instead of std::string name


[ROCm/clr commit: fcde9fe3df]
2019-03-11 19:51:25 +00:00
Aaron Enye Shi 4326d67550 Fix hash_for undefined reference in hipTestConstant test
Issue: mismatch undefined symbols in different user env
  - Binary expects modified return value std::string&
  - Fails to match libhip_hcc.so: return value is std::string& but doesn't match modified C++ env
Fix: Change return value to char*, create new key std::string in header from char*


[ROCm/clr commit: f8aca3159c]
2019-03-11 19:51:18 +00:00
Maneesh Gupta 2a11a5529e Merge pull request #949 from gargrahul/single_stream_concurrent_kernels
Add extension for kernel concurrency on same stream

[ROCm/clr commit: 3955f2c131]
2019-03-06 17:34:54 +05:30
Alex Voicu 45f4ac5023 dlopen() fixes (#929)
* Initial attempt to switch over to internally linked state.

* Add missing CMake update.

* hipLaunchKernelGGLImpl must be inline as well. Ensure internal linkage.

* Ensure global retrieval uses internally linked state.

* Hide HC in the implementation. Minimise ADL woes.

* Strange software exists, and must be catered to.

* Use a less spammy mechanism for ensuring internal linkage / non-export.

* Remove leftover internal detail.


[ROCm/clr commit: ed48847237]
2019-03-06 17:31:44 +05:30
Rahul Garg f364b32e29 Add extension for kernel concurrency on same stream
[ROCm/clr commit: 263e82a67a]
2019-03-06 12:55:39 +05:30
Wen-Heng (Jack) Chung e34b0ccd48 Address code review comments to use hipDeviceptr_t
[ROCm/clr commit: 8b7baa0bd9]
2019-03-05 05:51:05 +00:00
Wen-Heng (Jack) Chung 2706bf46f2 Add hipMemsetD32 and hipMemsetD32Async
Add 2 extra memset functions which fills memory with integer-typed data

Also change the parameters of ihipMemset to better explain the semantic


[ROCm/clr commit: 392271f4db]
2019-03-04 17:00:33 +00:00
Wilkin Chau 19b361281a Fix hipMemset3D test
Calculate the allocated size based on the width, height and depth.


[ROCm/clr commit: 99540373cf]
2019-02-28 22:42:46 +00:00
Yaxun Sam Liu ed6efc2c0b Add __gnu_h2f_ieee and __gnu_f2h_ieee
The implementation is copied from HCC runtime.

For hcc it has no effect since apps can find them in either hcc runtime or HIP
runtime.

hip-clang needs it in HIP/HCC runtime so that HIP/HCC and HIP/VDI runtime are
swappable.


[ROCm/clr commit: 972ca06c4c]
2019-02-21 12:48:28 -05:00
lmoriche 2c2a4cfe29 Merge pull request #899 from lmoriche/hip_clang
Add code-object-v3 support for the HIP-Clang path.

[ROCm/clr commit: 002bb4a94c]
2019-02-06 19:50:51 -08:00
Laurent Morichetti 0ef706ff97 Add code-object-v3 support for the HIP-Clang path.
[ROCm/clr commit: 74578160bd]
2019-02-05 14:43:26 -08:00
Rahul Garg c5ba34378a Revert "Fixed issue of GPU device losing access to host pinned memory"
[ROCm/clr commit: e94153ed8b]
2019-02-04 22:45:13 -08:00
Rahul Garg c6b09da295 Fixed issue of GPU device losing access to host pinned memory
[ROCm/clr commit: f8b399d3b5]
2019-01-22 06:25:43 +05:30