Allocate coherent uncached memory when HSA_DISABLE_CACHE is set

Set the KFD_IOC_ALLOC_MEM_FLAGS_COHERENT flag  and
KFD_IOC_ALLOC_MEM_FLAGS_UNCACHED flag to allocate
uncached coherent memory when HSA_DISABLE_CACHE
environment variable is set. At KFD driver,
Single KFD_IOC_ALLOC_MEM_FLAGS_UNCACHED flag is
not sufficient to allocate uncached memory. We
have to use both two flags to allocate uncached
memory.

Change-Id: Ie490f37b2e696314e60048f5b1b57442431696e9
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>


[ROCm/ROCR-Runtime commit: ae0e74095e]

projects/rocr-runtime/src/fmm.c

@@ -1335,7 +1335,7 @@ void *fmm_allocate_device(uint32_t gpu_id, void *address, uint64_t MemorySizeInB
 		aperture = &gpu_mem[gpu_mem_id].gpuvm_aperture;
 	}
 
-	if (!flags.ui32.CoarseGrain)
+	if (!flags.ui32.CoarseGrain || svm.disable_cache)
 		ioc_flags |= KFD_IOC_ALLOC_MEM_FLAGS_COHERENT;
 
 	if (flags.ui32.Uncached || svm.disable_cache)
@@ -1544,7 +1544,7 @@ static void *fmm_allocate_host_gpu(uint32_t node_id, void *address,
 	else
 		aperture = svm.dgpu_alt_aperture; /* always coherent */
 
-	if (!flags.ui32.CoarseGrain)
+	if (!flags.ui32.CoarseGrain || svm.disable_cache)
 		ioc_flags |= KFD_IOC_ALLOC_MEM_FLAGS_COHERENT;
 
 	if (flags.ui32.Uncached || svm.disable_cache)