[SYCL] Make device id unique per backend

sycl/doc/EnvironmentVariables.md

@@ -53,25 +53,25 @@ subject to change. Do not rely on these variables in production code.
 
 This environment variable limits the SYCL RT to use only a subset of the system's devices. Setting this environment variable affects all of the device query functions (`platform::get_devices()` and `platform::get_platforms()`) and all of the device selectors. 
 
-The value of this environment variable is a comma separated list of filters, where each filter is a triple of the form "`backend:device_type:device_num`" (without the quotes). Each element of the triple is optional, but each filter must have at least one value. Possible values of "backend" are:
-- host
+The value of this environment variable is a comma separated list of filters, where each filter is a triple of the form "`backend`:`device_type`:`device_num`" (without the quotes). Each element of the triple is optional, but each filter must have at least one value. Possible values of `backend` are:
+- `host`
 - `level_zero`
-- opencl
-- cuda
-- \*
+- `opencl`
+- `cuda`
+- `*`
 
-Possible values of "`device_type`" are:
-- host
-- cpu
-- gpu
-- acc
-- \*
+Possible values of `device_type` are:
+- `host`
+- `cpu`
+- `gpu`
+- `acc`
+- `*`
 
-`Device_num` is an integer that indexes the enumeration of devices from the sycl-ls utility tool, where the first device in that enumeration has index zero in each backend. For example, `SYCL_DEVICE_FILTER`=2 will return all devices with index '2' from all different backends. If multiple devices satisfy this device number (e.g., GPU and CPU devices can be assigned device number '2'), then default_selector will choose the device with the highest heuristic point. 
+`device_num` is an integer that indexes the enumeration of devices from the sycl-ls utility tool, where the first device in that enumeration has index zero in each backend. For example, `SYCL_DEVICE_FILTER=2` will return all devices with index '2' from all different backends. If multiple devices satisfy this device number (e.g., GPU and CPU devices can be assigned device number '2'), then default_selector will choose the device with the highest heuristic point.  When `SYCL_DEVICE_ALLOWLIST` is set, it is applied before enumerating devices and affects `device_num` values.
 
-Assuming a filter has all three elements of the triple, it selects only those devices that come from the given backend, have the specified device type, AND have the given device index. If more than one filter is specified, the RT is restricted to the union of devices selected by all filters. The RT does not include the "host" backend and the host device automatically unless one of the filters explicitly specifies the "host" device type. Therefore, `SYCL_DEVICE_FILTER`=host should be set to enforce SYCL to use the host device only. 
+Assuming a filter has all three elements of the triple, it selects only those devices that come from the given backend, have the specified device type, AND have the given device index. If more than one filter is specified, the RT is restricted to the union of devices selected by all filters. The RT does not include the `host` backend and the `host` device automatically unless one of the filters explicitly specifies the `host` device type. Therefore, `SYCL_DEVICE_FILTER=host` should be set to enforce SYCL to use the `host` device only. 
 
-Note that all device selectors will throw an exception if the filtered list of devices does not include a device that satisfies the selector. For instance, `SYCL_DEVICE_FILTER`=cpu,level_zero will cause host_selector() to throw an exception. `SYCL_DEVICE_FILTER` also limits loading only specified plugins into the SYCL RT. In particular, `SYCL_DEVICE_FILTER`=level_zero will cause the cpu_selector to throw an exception since SYCL RT will only load the level_zero backend which does not support any CPU devices at this time. When multiple devices satisfy the filter (e..g, `SYCL_DEVICE_FILTER`=gpu), only one of them will be selected.
+Note that all device selectors will throw an exception if the filtered list of devices does not include a device that satisfies the selector. For instance, `SYCL_DEVICE_FILTER=cpu,level_zero` will cause `host_selector()` to throw an exception. `SYCL_DEVICE_FILTER` also limits loading only specified plugins into the SYCL RT. In particular, `SYCL_DEVICE_FILTER=level_zero` will cause the `cpu_selector` to throw an exception since SYCL RT will only load the `level_zero` backend which does not support any CPU devices at this time. When multiple devices satisfy the filter (e..g, `SYCL_DEVICE_FILTER=gpu`), only one of them will be selected.
 
 ### `SYCL_PRINT_EXECUTION_GRAPH` Options
 

sycl/include/CL/sycl/detail/device_filter.hpp

@@ -82,6 +82,9 @@ inline std::ostream &operator<<(std::ostream &Out,
   return Out;
 }
 
+std::vector<std::string> tokenize(const std::string &Filter,
+                                  const std::string &Delim);
+
 } // namespace detail
 } // namespace sycl
 } // __SYCL_INLINE_NAMESPACE(cl)

sycl/source/detail/config.hpp

@@ -186,11 +186,11 @@ static const std::array<std::pair<std::string, info::device_type>, 5>
 
 // Array is used by SYCL_DEVICE_FILTER and SYCL_DEVICE_ALLOWLIST
 static const std::array<std::pair<std::string, backend>, 6> SyclBeMap = {
-    {{"host", backend::host},
-     {"opencl", backend::opencl},
+    {{"opencl", backend::opencl},
      {"level_zero", backend::level_zero},
      {"cuda", backend::cuda},
      {"rocm", backend::rocm},
+     {"host", backend::host},
      {"*", backend::all}}};
 
 template <> class SYCLConfig<SYCL_DEVICE_FILTER> {

sycl/source/detail/device_filter.cpp

@@ -17,60 +17,76 @@ __SYCL_INLINE_NAMESPACE(cl) {
 namespace sycl {
 namespace detail {
 
+std::vector<std::string> tokenize(const std::string &Filter,
+                                  const std::string &Delim) {
+  std::vector<std::string> Tokens;
+  size_t Pos = 0;
+  std::string Input = Filter;
+  std::string Tok;
+
+  while ((Pos = Input.find(Delim)) != std::string::npos) {
+    Tok = Input.substr(0, Pos);
+    Input.erase(0, Pos + Delim.length());
+
+    if (!Tok.empty()) {
+      Tokens.push_back(std::move(Tok));
+    }
+  }
+
+  // Add remainder
+  if (!Input.empty())
+    Tokens.push_back(std::move(Input));
+
+  return Tokens;
+}
+
 device_filter::device_filter(const std::string &FilterString) {
-  size_t Cursor = 0;
-  size_t ColonPos = 0;
-  auto findElement = [&](auto Element) {
-    size_t Found = FilterString.find(Element.first, Cursor);
+  std::vector<std::string> Tokens = tokenize(FilterString, ":");
+  size_t I = 0;
+
+  auto FindElement = [&](auto Element) {
+    size_t Found = Tokens[I].find(Element.first);
     if (Found == std::string::npos)
       return false;
-    Cursor = Found;
     return true;
   };
 
   // Handle the optional 1st field of the filter, backend
   // Check if the first entry matches with a known backend type
   auto It =
-      std::find_if(std::begin(SyclBeMap), std::end(SyclBeMap), findElement);
+      std::find_if(std::begin(SyclBeMap), std::end(SyclBeMap), FindElement);
   // If no match is found, set the backend type backend::all
   // which actually means 'any backend' will be a match.
   if (It == SyclBeMap.end())
     Backend = backend::all;
   else {
     Backend = It->second;
-    ColonPos = FilterString.find(":", Cursor);
-    if (ColonPos != std::string::npos)
-      Cursor = ColonPos + 1;
-    else
-      Cursor = Cursor + It->first.size();
+    I++;
   }
+
   // Handle the optional 2nd field of the filter - device type.
   // Check if the 2nd entry matches with any known device type.
-  if (Cursor >= FilterString.size()) {
+  if (I >= Tokens.size()) {
     DeviceType = info::device_type::all;
   } else {
     auto Iter = std::find_if(std::begin(SyclDeviceTypeMap),
-                             std::end(SyclDeviceTypeMap), findElement);
+                             std::end(SyclDeviceTypeMap), FindElement);
     // If no match is found, set device_type 'all',
     // which actually means 'any device_type' will be a match.
     if (Iter == SyclDeviceTypeMap.end())
       DeviceType = info::device_type::all;
     else {
       DeviceType = Iter->second;
-      ColonPos = FilterString.find(":", Cursor);
-      if (ColonPos != std::string::npos)
-        Cursor = ColonPos + 1;
-      else
-        Cursor = Cursor + Iter->first.size();
+      I++;
     }
   }
 
   // Handle the optional 3rd field of the filter, device number
   // Try to convert the remaining string to an integer.
   // If succeessful, the converted integer is the desired device num.
-  if (Cursor < FilterString.size()) {
+  if (I < Tokens.size()) {
     try {
-      DeviceNum = stoi(FilterString.substr(Cursor));
+      DeviceNum = stoi(Tokens[I]);
       HasDeviceNum = true;
     } catch (...) {
       std::string Message =

sycl/source/detail/filter_selector_impl.cpp

@@ -24,35 +24,12 @@ namespace sycl {
 namespace ONEAPI {
 namespace detail {
 
-std::vector<std::string> tokenize(const std::string &Filter,
-                                  const std::string &Delim) {
-  std::vector<std::string> Tokens;
-  size_t Pos = 0;
-  std::string Input = Filter;
-  std::string Tok;
-
-  while ((Pos = Input.find(Delim)) != std::string::npos) {
-    Tok = Input.substr(0, Pos);
-    Input.erase(0, Pos + Delim.length());
-
-    if (!Tok.empty()) {
-      Tokens.push_back(std::move(Tok));
-    }
-  }
-
-  // Add remainder
-  if (!Input.empty())
-    Tokens.push_back(std::move(Input));
-
-  return Tokens;
-}
-
 filter create_filter(const std::string &Input) {
   filter Result;
   constexpr auto Error = "Invalid filter string! Valid strings conform to "
                          "BE:DeviceType:DeviceNum, where any are optional";
 
-  std::vector<std::string> Tokens = tokenize(Input, ":");
+  std::vector<std::string> Tokens = sycl::detail::tokenize(Input, ":");
   std::regex IntegerExpr("[[:digit:]]+");
 
   // There should only be up to 3 tokens.
@@ -106,7 +83,7 @@ filter create_filter(const std::string &Input) {
 
 filter_selector_impl::filter_selector_impl(const std::string &Input)
     : mFilters(), mRanker(), mNumDevicesSeen(0), mMatchFound(false) {
-  std::vector<std::string> Filters = detail::tokenize(Input, ",");
+  std::vector<std::string> Filters = sycl::detail::tokenize(Input, ",");
   mNumTotalDevices = device::get_devices().size();
 
   for (const std::string &Filter : Filters) {

sycl/source/detail/global_handler.cpp

@@ -52,8 +52,9 @@ ProgramManager &GlobalHandler::getProgramManager() {
 
 Sync &GlobalHandler::getSync() { return getOrCreate(MSync); }
 
-std::vector<PlatformImplPtr> &GlobalHandler::getPlatformCache() {
-  return getOrCreate(MPlatformCache);
+std::map<PlatformImplPtr, std::vector<DeviceImplPtr>> &
+GlobalHandler::getPlatformDeviceCache() {
+  return getOrCreate(MPlatformDeviceCache);
 }
 
 std::mutex &GlobalHandler::getPlatformMapMutex() {
@@ -79,7 +80,7 @@ void shutdown() {
   // First, release resources, that may access plugins.
   GlobalHandler::instance().MScheduler.Inst.reset(nullptr);
   GlobalHandler::instance().MProgramManager.Inst.reset(nullptr);
-  GlobalHandler::instance().MPlatformCache.Inst.reset(nullptr);
+  GlobalHandler::instance().MPlatformDeviceCache.Inst.reset(nullptr);
 
   // Call to GlobalHandler::instance().getPlugins() initializes plugins. If
   // user application has loaded SYCL runtime, and never called any APIs,

sycl/source/detail/global_handler.hpp

@@ -11,6 +11,7 @@
 #include <CL/sycl/detail/spinlock.hpp>
 #include <CL/sycl/detail/util.hpp>
 
+#include <map>
 #include <memory>
 
 __SYCL_INLINE_NAMESPACE(cl) {
@@ -22,8 +23,10 @@ class ProgramManager;
 class Sync;
 class plugin;
 class device_filter_list;
+class device_impl;
 
 using PlatformImplPtr = std::shared_ptr<platform_impl>;
+using DeviceImplPtr = std::shared_ptr<device_impl>;
 
 /// Wrapper class for global data structures with non-trivial destructors.
 ///
@@ -51,7 +54,8 @@ class GlobalHandler {
   Scheduler &getScheduler();
   ProgramManager &getProgramManager();
   Sync &getSync();
-  std::vector<PlatformImplPtr> &getPlatformCache();
+  std::map<PlatformImplPtr, std::vector<DeviceImplPtr>> &
+  getPlatformDeviceCache();
   std::mutex &getPlatformMapMutex();
   std::mutex &getFilterMutex();
   std::vector<plugin> &getPlugins();
@@ -77,7 +81,8 @@ class GlobalHandler {
   InstWithLock<Scheduler> MScheduler;
   InstWithLock<ProgramManager> MProgramManager;
   InstWithLock<Sync> MSync;
-  InstWithLock<std::vector<PlatformImplPtr>> MPlatformCache;
+  InstWithLock<std::map<PlatformImplPtr, std::vector<DeviceImplPtr>>>
+      MPlatformDeviceCache;
   InstWithLock<std::mutex> MPlatformMapMutex;
   InstWithLock<std::mutex> MFilterMutex;
   InstWithLock<std::vector<plugin>> MPlugins;