[SYCL][CUDA][HIP] device overlap property

[zjin-lcf]

  printf("Relevant properties of this CUDA device\n");
  printf(
      "(%s) Can overlap one CPU<>GPU data transfer with GPU kernel execution "
      "(device property \"deviceOverlap\")\n",
      deviceProp.deviceOverlap ? "X" : " ");
  // printf("(%s) Can execute several GPU kernels simultaneously (compute
  // capability >= 2.0)\n", deviceProp.major >= 2 ? "X": " ");
  printf(
      "(%s) Can overlap two CPU<>GPU data transfers with GPU kernel execution\n"
      "    (Compute Capability >= 2.0 AND (Tesla product OR Quadro "
      "4000/5000/6000/K5000)\n",
      (deviceProp.major >= 2 && deviceProp.asyncEngineCount > 1) ? "X" : " ");

Reference
https://github.com/NVIDIA/cuda-samples/tree/master/Samples/0_Introduction/simpleMultiCopy

[JackAKirk]

To make use of this functionality simply use a queue without the in-order property, the overlap behavior of memcpy and compute is as described in the sycl spec: it is taken care of in the runtime automatically behind the scenes; whereas in the CUDA Runtime API the user must explicitly manage the streams. This functionality was enabled in the ext_one_api backend in dpc++ since #6102

[zjin-lcf]

Adding a device property for overlap (deviceProp.deviceOverlap) may be helpful in SYCL.

[JackAKirk]

Adding a device property for overlap (deviceProp.deviceOverlap) may be helpful in SYCL.

It is supported by all modern gpu devices I believe. sm_20 is really old on Nvidia and only sm_50 and above is officially supported by dpc++.

[zjin-lcf]

deviceOverlap and asyncEngineCount may be added to the (vendor) SYCL specification.