Prefer using memfd_create() for the ring buffer.
We were using /dev/shm, but this won't work on systems that
either don't have /dev/shm or have mounted it with noexec, because
for everything other than gfx700 we map the ring buffer with PROT_EXEC.
memfd_create() is Linux specific and was added in Linux 3.17, so we
will fallback to using /dev/shm on systems where memfd_create() is
not available.
Change-Id: I58fb533eebc362f6d29dc3e316a80801014d50e8
[ROCm/ROCR-Runtime commit: b93ffafdc7]
Corrected semantics used in hsa_queue_load_write_index_relaxed.
The semantics that was used in hsa_queue_load_write_index_relaxed
didn't seem to match the name of the function.
I also removed a useless return keyword.
Change-Id: If3819d38fb367f122fc382edf8ee3771a23279ae
[ROCm/ROCR-Runtime commit: 5872b618de]
1. Renamed _fmm_map_to_gpu to _fmm_map_to_apu_local
to reflect the real semantics of this function
2. Renamed _fmm_map_to_gpu_gtt to _fmm_map_to_gpu
because this function is used to map both gtt
and local memory
3. Call _fmm_map_to_gpu in _fmm_map_to_apu_local
to get rid of duplicated codes
Change-Id: Id8e3ebfffe0a3c27ebdcac8a8f4dc3738d67d10a
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
[ROCm/ROCR-Runtime commit: 019f7cbd20]
1. Initialize pointers to NULL in vm_create_and_init_object
2. Added helper function to add/remove device ids to/from mapped arrary
3. Only map nodes that were not mapped currently
4. Remove unnecessary condition check on object frees
Change-Id: I7aed6d40c7464be0d168d5796229af55451e0f34
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
[ROCm/ROCR-Runtime commit: b4c89c1ea7]
Print data in PMC trace when the debug level is set to 7(pr_debug).
Change-Id: I9abbb8f6c3f7962fb637528578c1a58b7784042d
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
[ROCm/ROCR-Runtime commit: 6f7b55f2d8]
_fmm_unmap_from_gpu is called in fmm_map_to_gpu_nodes
to unmap buffer from nodes that is already mapped to
but not in the new map nodes list. Previously, the unmap
was called unconditionally even though the size of the
array to unmap is 0. This fixes the issue by calling
the unmap func only when the unmap array size is not 0.
Also releases the fmm_mutex on error returns
Change-Id: Iadd8383caf7ebb92f02618798c5efd138a352aaa
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
[ROCm/ROCR-Runtime commit: 061db45fe2]
Buffer mapping to devices and buffer registration to
devices can be changed b/t two pointer info queries.
Thus update buffer mapping info and registration info
only when mapping and registration changed. This is
done by free mapped_node_id_array on mapping to new
device and free registered_node_id_array on registration
and re-allocate them on next ptr info query.
Also uses fmm_mutex to avoid race conditions in case
of calling hsaKmtQueryPointerInfo concurrently with
calling of buffer mapping or registration
Change-Id: Ibc2e20be1fc0147066f873dfa44b21f5015104b7
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
[ROCm/ROCR-Runtime commit: f06e887725]
cmake command in making packages was not updated.
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
Change-Id: Iafd3d9f4941d782bd77cfd0efafe48a02221b002
[ROCm/ROCR-Runtime commit: 61d1c6ffac]
Previously even for debug build, -O2 is used.
So there wasn't debug information in the debug build.
Change-Id: I6334474e007480eb2db191bdfec5a71677c26a52
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
[ROCm/ROCR-Runtime commit: d4e6ec0ff0]
Remove "zombie" queue state and report queue creation failure via
exceptions. Make Shared object a final container and support array
objects with Shared. Add message printing to hsa_exception in
debug builds.
Change-Id: I459f38c80846018acbf45538874e95f91dd6b195
[ROCm/ROCR-Runtime commit: f312a7386e]
Queue intercept is exposed as two tools-only APIs via the API
intercept table.
Change-Id: Iac9602ed3143974d85c3569e9092295ad18037f8
[ROCm/ROCR-Runtime commit: 0c7dde2d1f]
Also separated a function for removing CPU mapping
and reserving address, as a refactoring of codes
Change-Id: I1feb85b0b2ec942487f899ec3192c7c47dd7c7d5
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
[ROCm/ROCR-Runtime commit: 07110fbd38]
Update build instructions in README.md to use absolute path on cmake
parameter, CMAKE_MODULE_PATH. Relative path causes build error. Tested
on cmake 3.5.1 ans cmake 3.5.2.
Change-Id: I1b8e8deb9f4941580580be8087a94655ae155d02
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
[ROCm/ROCR-Runtime commit: 310e3d7b8b]
Previously kfd device is used to map memory for CPU access.
However this is not compatible with how TTM handles CPU mapping
on eviction - memory won't be unmapped and remapped on restore.
This fixes the issue by mmapping memory using DRM render device.
This patch requires a coordinated kernel driver change to work.
To make it compatible with old kernel driver, some temporary codes
are included. Once the coordinated kernel driver is checked in,
the temporary codes can be removed.
Change-Id: Ie7b304c4a82b7e8d5ab703acb81d66430af4f0bc
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
[ROCm/ROCR-Runtime commit: 68a2d286ca]
On thunk level, gfx906 works same as gfx900 chip
Change-Id: I727bd904284616f3b1b9b911e41ad0f19318b3ee
Signed-off-by: Shaoyun Liu <Shaoyun.Liu@amd.com>
[ROCm/ROCR-Runtime commit: 55fc06dac3]
1. Add hsa ext api hsa_amd_register_vmfault_handler for debugger to register callback in case of VM fault.
2. Extend hsa_ven_amd_loader API to:
(1) iterate loaded code objects in executable:
hsa_ven_amd_loader_executable_iterate_loaded_code_objects
(2) get loaded code object info:
hsa_ven_amd_loader_loaded_code_object_get_info
3. Make the id of hsa_queue the same as the one used in communication with thunk (for amd_aql_queue)
Change-Id: I68910809e59e24297350d262606f00e96c14bcbd
[ROCm/ROCR-Runtime commit: ce6aee01ed]
Child process hsaKmtOpenKFD() call must re-initialize global variables
copied from parent process. This includes close all file handles, free
dynamically malloc buf. Double free issue is because destroy_device_
debugging_memory() free the memory in parent process hsaKmtCloseKFD()
but don't reset it to null pointer. As a result, child process free it
again. kfd_fd is closed in parent process but don't reset to 0, so
child process close it again.
Fix: reset kfd_fd to 0 after close, reset is_device_debugged pointer to 0
after free
Change-Id: I421b3decbcaa4111298b8e599aa16940d851a58c
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
[ROCm/ROCR-Runtime commit: 3501b2f40d]
Adds the thunk include and lib paths to the cache, removes paths
to indicator files from the cache, uses the cached path directory
(if any) as a search hint for indicator files.
Change-Id: I0859faa8d229a97abfaacb408d2c831e317aed5f
[ROCm/ROCR-Runtime commit: a8d818a6bc]
Because of HW design change, GPUVM aperture is no longer needed on GFX9
APUs. However, on APUs some functionalities still depend on GPUVM
aperture, so we choose to use SVM aperture instead to assume
the functionality of previous GPUVM aperture.
Change-Id: Ife7f0d598dd7989f2bcf7cdf3466d5a68703ca60
Signed-off-by: Yong Zhao <yong.zhao@amd.com>
[ROCm/ROCR-Runtime commit: 3b852b4437]
Use mbind to specify the NUMA node for system memory allocation. This
only works with HSA_USERPTR_FOR_PAGED_MEM=1.
Change-Id: I88e7815d5a5aefcc4c22358c1a4a1635d7677ef3
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: cb4814eadc]
Marking heap memory as executable using mprotect() is not allowed
by SELinux. mprotect() calls that try to do this will fail on systems
with SELinux enabled. This is also a security risk, so it should be
fixed even on systems that allow this.
Any memory we want to mark as executable must be allocated using mmap().
See https://www.akkadia.org/drepper/selinux-mem.html
The two places where we try to mark heap memory as executable both use
posix_memalign() to allocate the heap memory. In both cases, the
alignment value passed into this function is always equal to PAGE_SIZE,
which means that they are safe to replace with mmap(), which guarantees
alignment to PAGE_SIZE. In this case PAGE_SIZE has been set to
sysconf(_SC_PAGESIZE);
v2:
- Use MAP_PRIVATE instead of MAP_SHARED. This matches the behavior
of memory allocated by posix_memalign()
- Ignore alignment hints instead of returning error when we can't
accommodate them.
- Drop alignment parameter of allocate_exec_aligned_memory() since
the only alignment supported is sysconf(_SC_PAGESIZE).
- Remove extra parameter from fmm_release().
- Add error path to fmm_allocate_host_cpu() for when mmap fails.
v3:
- Avoid use after free.
Change-Id: I7d51279790d9700bc3fa761c44bfde1c1936019b
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: e2ed9cf79a]
src/perfctr.c: In function ‘destroy_shared_region’:
src/perfctr.c:154:10: error: logical ‘and’ of equal expressions [-Werror=logical-op]
if (sem && sem != SEM_FAILED) {
^~
src/perfctr.c: In function ‘update_block_slots’:
src/perfctr.c:323:11: error: logical ‘or’ of equal expressions [-Werror=logical-op]
if (!sem || sem == SEM_FAILED)
^~
v2:
- Initialize and reset sem to SEM_FAILED.
Change-Id: Id70361079b715c4946b13e4460e4fd85d9542c46
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
[ROCm/ROCR-Runtime commit: 4815d9b887]
TensorFlow was running out of VRAM due to padding up allocations
from legacy memory APIs. These allocations have been added to
the fragment allocator to improve VRAM utilization.
Change-Id: Ic680fff576a0434b3b17a4c91746da44e09957fa
[ROCm/ROCR-Runtime commit: 4f299a9909]
Fix a while loop that can cause forever loop when cpuid instruction
doesn't work properly.
Change-Id: Iefa49d23b40c994eb4369621974a7d3c4067e47a
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
[ROCm/ROCR-Runtime commit: 5815d9de9b]
Sean Keely
Oak Zeng
Amber Lin
Evgeny
Chris Freehill
Kent Russell
Shaoyun Liu
Qingchuan Shi
Philip Yang
Jan Vesely
Ramesh Errabolu
Yong Zhao
Felix Kuehling
Tom Stellard