SWDEV-301667 - Add cache state for a device

- Add a global cache state for a device to indicate scopes of submitted
AQL packets
- Remove scopes for TS marker if hipEventReleaseToDevice is passed. Set
env ROC_EVENT_NO_FLUSH=1 to use NOP AQL for event records.
It would flush caches by default with system scope release.
- Calling finish() should ensure if caches are flushed, if not queue a
marker

Change-Id: Ibbbdbb1cd7ac61cb35649169212142545be159e0

rocclr/utils/flags.hpp

@@ -273,7 +273,9 @@ release(uint, ROC_AQL_QUEUE_SIZE, 4096,                                       \
 release(bool, ROC_SKIP_KERNEL_ARG_COPY, false,                                \
         "If true, then runtime can skip kernel arg copy")                     \
 release(bool, GPU_STREAMOPS_CP_WAIT, false,                                   \
-        "Force the stream wait memory operation to wait on CP.")
+        "Force the stream wait memory operation to wait on CP.")              \
+release(bool, ROC_EVENT_NO_FLUSH, false,                                      \
+        "Use NOP AQL packet for event records with no explicit flags.")
 
 namespace amd {