[Caffe2] Windows build errors in generated file caffe2.pb.h

[skelleher]

Issue description

Windows build of caffe2 fails. Compiler errors in generated protobuf file caffe2.pb.h:

D:/code/pytorch/build\caffe2/proto/caffe2.pb.h(229): error : member "caffe2::TensorProto_Segment::kIndexInFileMessages" may not be initialized [D:\ code\pytorch\build\caffe2\caffe2_gpu.vcxproj]

The generated header looks OK:

class CAFFE2_API TensorProto_Segment : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:caffe2.TensorProto.Segment) */ {
 public:
  TensorProto_Segment();
  virtual ~TensorProto_Segment();

  //// ... snip .... ////

  static PROTOBUF_CONSTEXPR int const kIndexInFileMessages =
    0;
}

Master branch git hash 9f21ec7 .

System Info

• PyTorch or Caffe2:
  Caffe2
• How you installed PyTorch (conda, pip, source):
  Did not install PyTorch, as I am building from source.
• Build command you used (if compiling from source):
  Opened Developer Command Prompt for VS 2017 (but see below)
  Activated Python 2.7 environment
  Ran cmake-gui - selected "Visual Studio 14 2015 Win64" generator
  Configured Cmake options (mostly use defaults; see below)
  Ran scripts\build_windows.bat
• OS:
  Windows 10
• PyTorch version:
  N/A
• Python version:
  Anaconda 2.7.14 64-bit
• CUDA/cuDNN version:
  CUDA 9.1 / cuDNN 7.1.3
• GPU models and configuration:
  GeForce GTX 980 Ti - 6 GB
• GCC version (if compiling from source):
  N/A. VS 2017 (but using the included VS 2015 toolchain).
• CMake version:
  3.10.3
• Versions of any other relevant libraries:

[skelleher]

My cmake output is:

******** Summary ********
General:
CMake version : 3.10.3
CMake command : C:/Program Files/CMake/bin/cmake.exe
Git version : v0.1.11-8582-g9f21ec7ca-dirty
System : Windows
C++ compiler : C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
C++ compiler version : 19.0.24234.1
BLAS : Eigen
CXX flags : /DWIN32 /D_WINDOWS /W3 /GR /EHsc -DONNX_NAMESPACE=onnx_c2 /MP /bigobj
Build type : Release
Compile definitions :

BUILD_CAFFE2 : ON
BUILD_ATEN : OFF
BUILD_BINARY : ON
BUILD_CUSTOM_PROTOBUF : ON
Link local protobuf : ON
BUILD_DOCS : OFF
BUILD_PYTHON : OFF
BUILD_SHARED_LIBS : ON
BUILD_TEST : ON
USE_ASAN : OFF
USE_CUDA : ON
CUDA static link : OFF
USE_CUDNN : ON
CUDA version : 9.1
cuDNN version : 7.1.3
CUDA root directory : C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.1
CUDA library : C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.1/lib/x64/cuda.lib
cudart library : C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.1/lib/x64/cudart_static.lib
cublas library : C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.1/lib/x64/cublas.lib;C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.1/lib/x64/cublas_device.lib
cufft library : C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.1/lib/x64/cufft.lib
curand library : C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.1/lib/x64/curand.lib
cuDNN library : C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.1/lib/x64/cudnn.lib
nvrtc : C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.1/lib/x64/nvrtc.lib
CUDA include path : C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.1/include
NVCC executable : C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.1/bin/nvcc.exe
CUDA host compiler : $(VCInstallDir)bin
USE_TENSORRT : OFF
USE_ROCM : OFF
USE_EIGEN_FOR_BLAS : ON
USE_FFMPEG : OFF
USE_GFLAGS : OFF
USE_GLOG : OFF
USE_GLOO : OFF
USE_LEVELDB : OFF
USE_LITE_PROTO : OFF
USE_LMDB : OFF
USE_METAL : OFF
USE_MKL :
USE_MOBILE_OPENGL : OFF
USE_MPI : OFF
USE_NCCL : OFF
USE_NERVANA_GPU : OFF
USE_NNPACK : OFF
USE_OBSERVERS : ON
USE_OPENCL : OFF
USE_OPENCV : ON
OpenCV version : 3.4.0
USE_OPENMP : OFF
USE_PROF : OFF
USE_REDIS : OFF
USE_ROCKSDB : OFF
USE_ZMQ : OFF
Configuring done
Generating done

[skelleher]

I modified build_windows.bat to use /MD instead of /MT, since my application needs to link with several other libraries that were compiled for /MD:

if NOT DEFINED CMAKE_GENERATOR (
if DEFINED APPVEYOR_BUILD_WORKER_IMAGE (
....
-DUSE_LEVELDB=OFF ^
-DUSE_ROCKSDB=OFF ^
-DUSE_OPENCV=OFF ^
-DBUILD_SHARED_LIBS=ON ^
-DBUILD_PYTHON=OFF^
|| goto :label_error

[skelleher]

-DBUILD_SHARED_LIBS=ON seems to be the culprit. Builds fine if I leave it set to OFF.

I don't particularly care if Caffe2 and friends build as static or shared libs. The problem is that pytorch\cmake\public\cuda.cmake defaults to /MT unless BUILD_SHARED_LIBS=ON.

Building caffe2_gpu.lib with /MT prevents me from actually using it, since my application uses many other libs built with /MD (the Visual Studio default) and we don't have the option of recompiling them all from source.

[ezyang]

@peterjc123, I know this is touching the caffe2 bits as opposed to the pytorch bits, but do you have any insight here?

[skelleher]

The problem I was trying to solve was compiling caffe2 with /MD instead of /MT. Almost everywhere this is controlled by CAFFE2_USE_MSVC_STATIC_RUNTIME.

I think there's a bad assumption in cuda.cmake, where this switch is instead controlled by BUILD_SHARED_LIBS (I don't agree that this should control /MT or /MD; static libs often need to be built with /MD). But I enabled BUILD_SHARED_LIBS anyway, which failed to build.

I think, to be consistent, cuda.cmake should use CAFFE2_USE_MSVC_STATIC_RUNTIME e.g.:

if (MSVC)
  if (${CMAKE_BUILD_TYPE} MATCHES "Release")
    if (${CAFFE2_USE_MSVC_STATIC_RUNTIME})
      list(APPEND CUDA_NVCC_FLAGS "-Xcompiler" "-MT")
    else()
      list(APPEND CUDA_NVCC_FLAGS "-Xcompiler" "-MD")
    endif()
  elseif(${CMAKE_BUILD_TYPE} MATCHES "Debug")
    message(FATAL_ERROR
            "Caffe2 currently does not support the combination of MSVC, Cuda "
            "and Debug mode. Either set USE_CUDA=OFF or set the build type "
            "to Release")
    if (${CAFFE2_USE_MSVC_STATIC_RUNTIME})
      list(APPEND CUDA_NVCC_FLAGS "-Xcompiler" "-MTd")
    else()
      list(APPEND CUDA_NVCC_FLAGS "-Xcompiler" "-MDd")
    endif()
  else()
    message(FATAL_ERROR "Unknown cmake build type: " ${CMAKE_BUILD_TYPE})
  endif()
endif()

As for the caffe2.pb.h compiler errors with BUILD_SHARED_LIBS, it beats me.

[peterjc123]

@skelleher I don't quite get your idea. Is it that just simply change CAFFE2_USE_MSVC_STATIC_RUNTIME in Line 3 to BUILD_SHARED_LIBS will do the work? I'm not super familiar with Caffe2, so I don't know whether cuda.cmake is a standalone script or it is okay to accept some global variables here.

[xkszltl]

I remember I've seen this issue before.

It's related to a funny combination of nvcc and protobuf actually if my memory is correct.
PROTOBUF_CONSTEXPR is defined to constexpr.
However in nvcc 9.1 you need to get rid of it in order to do in-place initialization.

So...the solution is to patch protobuf...

BTW this issue is on protobuf 3.5.
master branch of protobuf should not have this issue but it has something else...

[orionr]

@skelleher I think you might be right on using CAFFE2_USE_MSVC_STATIC_RUNTIME instead of BUILD_SHARED_LIBS inside public/cuda.cmake to set /MT vs /MD. Thanks for bringing this up and I'll confirm with some of the team here.

[orionr]

We are looking at the PROTOBUF_CONSTEXPR issue above on Windows as a part of #11160

[orionr]

@skelleher this should be fixed as a part of #11266 . Closing.

[skelleher]

Note to anyone building caffe2 with CUDA 9.x on Windows; this build error was ultimately fixed by commit 9310eb1 / pull request #16251.

Looks like that commit was not part of the v1.0.1 tag (last stable release).

The commit adds a string replace to cmake/ProtoBufPatch.cmake to fix the build errors "*::kIndexInFileMessages may not be initialized"

Also note: ProtoBufPatch.cmake only executes if CAFFE2_LINK_LOCAL_PROTOBUF is defined.