[MSVC] Referencing a static from another crate doesn't seem to work

[tomaka]

The glutin library has a build script and the khronos_api build-dependency.
The khronos_api crate does nothing except export 4 pub static items (source code here).

This build script fails to compile with the MSVC++ nightlies with the following error:

     Running `rustc build.rs --crate-name build_script_build --crate-type bin -C prefer-dynamic -g --cfg feature="default" --cfg feature="window" --out-dir C:\projects\glutin\target\debug\build\glutin-40bab6d10148d98f --emit=dep-info,link -L dependency=C:\projects\glutin\target\debug -L dependency=C:\projects\glutin\target\debug\deps --extern gl_generator=C:\projects\glutin\target\debug\deps\libgl_generator-3f3da0d671af0734.rlib --extern khronos_api=C:\projects\glutin\target\debug\deps\libkhronos_api-af8e40aff286ad52.rlib`
error: linking with `link.exe` failed: exit code: 1120
note: "link.exe" "/NOLOGO" "/NXCOMPAT" "/LIBPATH:C:\projects\glutin\rustc-nightly-x86_64-pc-windows-msvc\rustc\bin\rustlib\x86_64-pc-windows-msvc\lib" "C:\projects\glutin\target\debug\build\glutin-40bab6d10148d98f\build_script_build.o" "/OUT:C:\projects\glutin\target\debug\build\glutin-40bab6d10148d98f\build_script_build.exe" "/OPT:REF,ICF" "C:\projects\glutin\target\debug\deps\libgl_generator-3f3da0d671af0734.rlib" "C:\projects\glutin\target\debug\deps\libxml-b8f632f5409f1c24.rlib" "C:\projects\glutin\target\debug\deps\libkhronos_api-af8e40aff286ad52.rlib" "C:\projects\glutin\target\debug\deps\liblog-8a6aba167994951e.rlib" "C:\projects\glutin\target\debug\deps\liblibc-ef5cbad4ef5c7a1e.rlib" "C:\projects\glutin\target\debug\deps\libbitflags-52cb4de1e3db33a5.rlib" "/LIBPATH:C:\projects\glutin\rustc-nightly-x86_64-pc-windows-msvc\rustc\bin\rustlib\x86_64-pc-windows-msvc\lib" "std-74fa456f.lib" "/LIBPATH:C:\projects\glutin\target\debug" "/LIBPATH:C:\projects\glutin\target\debug\deps" "/LIBPATH:C:\projects\glutin\rustc-nightly-x86_64-pc-windows-msvc\rustc\bin\rustlib\x86_64-pc-windows-msvc\lib" "/LIBPATH:C:\projects\glutin\.rust\bin\x86_64-pc-windows-msvc" "/LIBPATH:C:\projects\glutin\bin\x86_64-pc-windows-msvc" "ws2_32.lib" "userenv.lib" "advapi32.lib" "kernel32.lib" "shell32.lib" "msvcrt.lib" "compiler-rt.lib"
note:    Creating library C:\projects\glutin\target\debug\build\glutin-40bab6d10148d98f\build_script_build.lib and object C:\projects\glutin\target\debug\build\glutin-40bab6d10148d98f\build_script_build.exp

... (lots of warnings, see below) ...

build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN7WGL_XML20h414720ad029f8e51qaaE referenced in function _ZN4main20hdd85b714c95cf787jaaE
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN7EGL_XML20h414720ad029f8e51kaaE referenced in function _ZN4main20hdd85b714c95cf787jaaE
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN7GLX_XML20h414720ad029f8e51waaE referenced in function _ZN4main20hdd85b714c95cf787jaaE
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN6GL_XML20h414720ad029f8e51eaaE referenced in function _ZN4main20hdd85b714c95cf787jaaE
C:\projects\glutin\target\debug\build\glutin-40bab6d10148d98f\build_script_build.exe : fatal error LNK1120: 4 unresolved externals

The four unresolved symbols refer to the four static variables exported by khronos_api.
This only happens with the 64bits version. The 32bits build works fine. EDIT: oops, 32bits was in fact using MinGW, so nevermind.

In addition to this, there are also a lot of warnings generated:

build_script_build.o : warning LNK4217: locally defined symbol _ZN8registry24RegistryBuilder$LT$R$GT$16consume_registry15__STATIC_FMTSTR20h5aefe2547183a934OQcE imported in function _ZN8registry24RegistryBuilder$LT$R$GT$16consume_registry20h7736587482988118975E
build_script_build.o : warning LNK4217: locally defined symbol _ZN8registry24RegistryBuilder$LT$R$GT$16consume_registry15__STATIC_FMTSTR20h5aefe2547183a9349LcE imported in function _ZN8registry24RegistryBuilder$LT$R$GT$16consume_registry20h7736587482988118975E
build_script_build.o : warning LNK4217: locally defined symbol _ZN8registry24RegistryBuilder$LT$R$GT$16consume_registry10_FILE_LINE20h61ca6051b049c413zMcE imported in function _ZN8registry24RegistryBuilder$LT$R$GT$16consume_registry20h7736587482988118975E
build_script_build.o : warning LNK4217: locally defined symbol _ZN20MAX_LOG_LEVEL_FILTER20h52e87f418f7a0ca4EaaE imported in function _ZN8registry24RegistryBuilder$LT$R$GT$16consume_registry20h7736587482988118975E
build_script_build.o : warning LNK4217: locally defined symbol _ZN8registry24RegistryBuilder$LT$R$GT$16consume_registry15__STATIC_FMTSTR20h5aefe2547183a934EOcE imported in function _ZN8registry24RegistryBuilder$LT$R$GT$16consume_registry20h7736587482988118975E
build_script_build.o : warning LNK4217: locally defined symbol _ZN8registry24RegistryBuilder$LT$R$GT$16consume_registry3LOC20h7853c3b976861309ENcE imported in function _ZN8registry24RegistryBuilder$LT$R$GT$16consume_registry20h7736587482988118975E
build_script_build.o : warning LNK4217: locally defined symbol _ZN8registry24RegistryBuilder$LT$R$GT$16consume_registry15__STATIC_FMTSTR20h5aefe2547183a934WPcE imported in function _ZN8registry24RegistryBuilder$LT$R$GT$16consume_registry20h7736587482988118975E
build_script_build.o : warning LNK4217: locally defined symbol _ZN8registry24RegistryBuilder$LT$R$GT$16consume_registry10_FILE_LINE20h61ca6051b049c413mQcE imported in function _ZN8registry24RegistryBuilder$LT$R$GT$16consume_registry20h7736587482988118975E

... (around 150 lines of similar warnings)

The symbols of these warnings refer to functions of the other build-dependency, which is gl_generator. I don't really know what they mean. Maybe they are shown only because of the linking errors and would be hidden otherwise?

Here is the complete log from appveyor for reference.

[retep998]

Since build scripts link to their dependencies dynamically instead of statically, this means that statics have to be properly marked with dllimport and dllexport, unlike for functions where those attributes can be left off with only a minor performance hit. MinGW's ldpatches up those missing references for us which is why the MinGW version works, but MSVC's link does not.

[tomaka]

cc @alexcrichton I guess

[retep998]

See also #7196

[alexcrichton]

We currently do place dllimport on statics (and dllexport), but I haven't had a chance to investigate this thoroughly yet. I'm surprised that this doesn't work when the bootstrap works.

[tomaka]

I'm going to replace static by const in the khronos_api crate, so the test case above will no longer produce an error.

However any static + build script combination seems to trigger this problem, so it should be easy to reproduce.

[tomaka]

I have the same problem with my pocket-resources crate: https://github.com/tomaka/pocket-resources
When compiling the demo, I get the following:

error: linking with `link.exe` failed: exit code: 1120
note: "link.exe" "/NOLOGO" "/NXCOMPAT" "/LIBPATH:D:\Logiciels\Rust\bin\rustlib\x
86_64-pc-windows-msvc\lib" "D:\Projets\pocket-resources\demo\target\debug\build\
test-0957aa1c8f24e32d\build_script_build.o" "/OUT:D:\Projets\pocket-resources\de
mo\target\debug\build\test-0957aa1c8f24e32d\build_script_build.exe" "/OPT:REF,IC
F" "D:\Projets\pocket-resources\demo\target\debug\deps\libpocket_resources-3f7a5
7389309468a.rlib" "/LIBPATH:D:\Logiciels\Rust\bin\rustlib\x86_64-pc-windows-msvc
\lib" "std-74fa456f.lib" "/LIBPATH:D:\Projets\pocket-resources\demo\target\debug
" "/LIBPATH:D:\Projets\pocket-resources\demo\target\debug\deps" "/LIBPATH:D:\Log
iciels\Rust\bin\rustlib\x86_64-pc-windows-msvc\lib" "/LIBPATH:D:\Projets\pocket-
resources\demo\.rust\bin\x86_64-pc-windows-msvc" "/LIBPATH:D:\Projets\pocket-res
ources\demo\bin\x86_64-pc-windows-msvc" "/LIBPATH:C:\Users\pierre\.rust\bin\x86_
64-pc-windows-msvc" "ws2_32.lib" "userenv.lib" "advapi32.lib" "kernel32.lib" "sh
ell32.lib" "msvcrt.lib" "compiler-rt.lib"
note:    Creating library D:\Projets\pocket-resources\demo\target\debug\build\te
st-0957aa1c8f24e32d\build_script_build.lib and object D:\Projets\pocket-resource
s\demo\target\debug\build\test-0957aa1c8f24e32d\build_script_build.exp
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN7packa
ge15__STATIC_FMTSTR20h1f831eca1e1942baPaaE referenced in function _ZN7package20h
5585954171191908666E
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN7packa
ge15__STATIC_FMTSTR20h1f831eca1e1942bafbaE referenced in function _ZN7package20h
5585954171191908666E
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN7packa
ge15__STATIC_FMTSTR20h1f831eca1e1942baEbaE referenced in function _ZN7package20h
5585954171191908666E
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN7packa
ge15__STATIC_FMTSTR20h1f831eca1e1942ba0faE referenced in function _ZN7package20h
5585954171191908666E
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN7packa
ge16__STATIC_FMTARGS20h2be310a11ffe1ab8HgaE referenced in function _ZN7package20
h5585954171191908666E
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN7packa
ge15__STATIC_FMTSTR20h1f831eca1e1942ba8caE referenced in function _ZN7package12c
losure.1694E
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN7packa
ge15__STATIC_FMTSTR20h1f831eca1e1942baPdaE referenced in function _ZN7package12c
losure.1694E
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN7packa
ge15__STATIC_FMTSTR20h1f831eca1e1942baweaE referenced in function _ZN7package12c
losure.1694E
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN21path
_to_resource_name10_FILE_LINE20h548d6e231b25c39c1maE referenced in function _ZN2
1path_to_resource_name12closure.2253E
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN21path
_to_resource_name10_FILE_LINE20h548d6e231b25c39cnnaE referenced in function _ZN2
1path_to_resource_name12closure.2253E
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN21path
_to_resource_name10_FILE_LINE20h548d6e231b25c39cJnaE referenced in function _ZN2
1path_to_resource_name12closure.2253E
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN21path
_to_resource_name10_FILE_LINE20h548d6e231b25c39c5naE referenced in function _ZN2
1path_to_resource_name12closure.2253E
build_script_build.o : error LNK2019: unresolved external symbol __imp__ZN20path
_to_enum_variant15__STATIC_FMTSTR20h1f831eca1e1942barlaE referenced in function
_ZN20path_to_enum_variant12closure.2484E
D:\Projets\pocket-resources\demo\target\debug\build\test-0957aa1c8f24e32d\build_
script_build.exe : fatal error LNK1120: 13 unresolved externals

But interesingly if I add crate-type = ["dylib"] in pocket_resources' Cargo.toml it works.

[tomaka]

Apparently using panic! seems to store the line number and filename in a static.

For each usage of panic!, assert! or similar you get either a warning LNK4217 or an error. Warnings seem much more frequent than errors.

I tried to compile a complex project (outside of a build script), and I got hundreds and hundreds of warnings, and 9 errors.

In my opinion this wasn't detected earlier because warnings from the linker are simply ignored.

[Mr-Byte]

I'm having a similar issue during linking using cargo test. In my main crate I have a static INTERFACE_ID which is not public, but instead local to the module. A reference is then returned to this via an iid() static method. Calling cargo build will succeed, but when calling cargo test which links against the main crate, very similar linking errors occur. I'm not sure why this would occur, since the static is not public, other than the fact that the iid() method is inline. I'll attempt turning off inlining to see what results I get.

[alexcrichton]

OK, I had some time to investigate this and I believe I know what's going on now. The fundamental problem here is #7196 (we don't handle dllimport/dllexport properly). To recap, a constant FOO, when tagged with dllexport, is referenced through __imp_FOO instead of the normal path (I don't believe the symbol FOO is exposed from the dll). So our situation is:

• An upstream rlib declares a dllexport static. This rlib then contains this object file, but apparently the crucial part is that the symbol table for the rlib does not include the __imp_ name.
• Our downstream executable references the upstream constant, tagging it as dllimport. This means that we generate a symbol reference to __imp_FOO
• When linking these two objects together the linker is trying to find the symbol __imp_FOO but it never finds it because the symbol table for our first library didn't actually include that symbol. This is when the linker winds up generating an error.

Now one question I had personally was "why doesn't this break everything?!" We have a huge test suite, most of which is passing on MSVC, and most of which references a static from the standard library (aka this exact situation). It turns out that if the linker already included the object file for another reason (e.g. it resolved another function) then it will detect that we're looking for __imp_FOO but found FOO, wiring the two up after issuing a warning.

What this ends up meaning is that if any other symbol from our upstream library was used, then the linker error goes away. For example if you add pub fn foo() {} to the upstream library and then call it in the downstream library everything will link (with the linker issuing warnings).

So, to summary, there are a number of ways to work around this:

• Make sure some other symbol in the library is referenced (to make sure the whole library is linked). This could just be a dummy function.
• Make a dylib. The dllimport annotation is then actually correct!
• Don't expose statics/constants in the public interface. Sidesteps the problem entirely.

We currently don't tag any functions with dllimport (only dllexport), so I'm currently exploring that route to see if it works for constants as well (which would fix this problem), but I vaguely remember it specifically not working before, so my hopes are not that high.

Otherwise this is basically a dupe of #7196 (correctly dealing with dllimport/dllexport), so I'm going to close this in favor of that.

[alexcrichton]

Ah yeah unfortunately not using dllimport didn't work. If you're importing a function from a dll and forget dllimport I think you take a small perf hit (e.g. one jump instruction) to link anyway, but if you're importing a constant from a dll and forget dllimport it just hits a link error.