[PAL] Allow more heaps for non-SVM suballocations

On ReBar systems the invible heap is not present, so in theory we should
fail creating the suballocation chunk, however PAL doesn't report any
errors.

To make sure we never fail, allow creating the allocation in the visible
heap and system memory.

Change-Id: Iea9cc68d98b9cb396a2b7a37398b98b66274083b


[ROCm/clr commit: 330b674821]

projects/clr/rocclr/device/pal/palresource.cpp

@@ -1925,8 +1925,10 @@ bool MemorySubAllocator::CreateChunk(const Pal::IGpuMemory* reserved_va) {
   createInfo.alignment = device_->properties().gpuMemoryProperties.fragmentSize;
   createInfo.vaRange = Pal::VaRange::Default;
   createInfo.priority = Pal::GpuMemPriority::Normal;
-  createInfo.heapCount = 1;
+  createInfo.heapCount = 3;
   createInfo.heaps[0] = Pal::GpuHeapInvisible;
+  createInfo.heaps[1] = Pal::GpuHeapLocal;
+  createInfo.heaps[2] = Pal::GpuHeapGartUswc;
   createInfo.flags.peerWritable = device_->P2PAccessAllowed();
   createInfo.mallPolicy = static_cast<Pal::GpuMemMallPolicy>(device_->settings().mallPolicy_);
   GpuMemoryReference* mem_ref = GpuMemoryReference::Create(*device_, createInfo);