Split ihipCtx_t into ihipCtx_t and ihipDevice_t .

Major change to existing code base.
    Ctx holds streams, enables peers, and flags.
    Device holds accelerator, hsa-agent, device props.

Add hipCtx_t.

Add peer APIs that accept hipCtx_t (in addition to deviceId)

Compiles and passes directed tests.

Change-Id: Iddab1eb9edbf90caad2ef5959c6b811d658197f1


[ROCm/clr commit: d09b19bb6c]

projects/clr/hipamd/src/hip_device.cpp

@@ -283,15 +283,18 @@ hipError_t hipSetDeviceFlags( unsigned int flags)
 
     hipError_t e;
 
-    auto * hipDevice = ihipGetTlsDefaultCtx();
-    if(hipDevice){
-       hipDevice->_device_flags = hipDevice->_device_flags | flags;
+    auto * ctx = ihipGetTlsDefaultCtx();
+
+    // TODO : does this really OR in the flags or replaces previous flags:
+    // TODO : Review error handling behavior for this function, it often returns ErrorSetOnActiveProcess
+    if (ctx) {
+       ctx->_ctxFlags = ctx->_ctxFlags | flags;
        e = hipSuccess;
-    }else{
+    } else {
        e = hipErrorInvalidDevice;
     }
 
     return ihipLogStatus(e);
-}
+};