[DO NOT MERGE] build Enzyme as archive

[ZuseZ4]

r? ghost

try-job: dist-x86_64-linux

[rustbot]

This PR changes how LLVM is built. Consider updating src/bootstrap/download-ci-llvm-stamp.

[ZuseZ4]

@bors try

[bors]

⌛ Trying commit 8102bba with merge 75a4f91...

[bors]

💔 Test failed - checks-actions

[ZuseZ4]

@Kobzol this is the PR about which I asked you. The error shows up in all cases, no matter whether we build Enzyme and/or LLVM as static or dynamic.

In https://github.com/rust-lang/rust/pull/140799/files#diff-5a1e05f2688d271039171a547d407d0c8a96715ee64d35562fc76b4c9a874303 we link against Enzyme inside of
impl Step for Rustc {, which causes cranelift to fail since it can't find Enzyme.
However, there we might have multiple compiler backends, and I'm not sure how to limit adding the flag to the llvm backend. In apple dist CI we build both cranelift and llvm, so I can't just check if cranelift is build, I care if it's currently used.

That could be checked in impl Step for CodegenBackend {, since CodegenBackend encodes the specific backend.

I think in general I'm just confused by bootstrap and how we should fix it. Bjorn or you said Backends can be dlopen'ed and thus swapped at rustc runtime. So if we link Enzyme statically, we should only do that once when building rustc for the first time. When building it dynamically we should do it every time, and thus tell rustc where to find it, even when building cranelift, right? Any suggestions on how to handle this? Preferably I would get libEnzyme.so to work (since the artifact can be re-used by downstream users), but either approach is fine to get started.

I am also confused by the fact that my rust code first adds -L path/to/libs and then -lEnzyme, but in the log it shows up the other way around, first -lEnzyme and then -L correct/path/to/libs. Without that, it would probably work correctly. I assume cargo is clearing up the -L paths as part of uplifting?

2025-05-08T13:46:45.2387720Z [RUSTC-TIMING] object test:false 4.353
2025-05-08T13:46:56.6105770Z �[1m�[32m   Compiling�[0m cranelift-module v0.118.0
2025-05-08T13:46:56.6107250Z �[1m�[32m   Compiling�[0m cranelift-native v0.118.0
2025-05-08T13:46:56.6696930Z [RUSTC-TIMING] cranelift_native test:false 0.052
2025-05-08T13:46:56.6714790Z �[1m�[32m   Compiling�[0m cranelift-frontend v0.118.0
2025-05-08T13:46:56.9650020Z [RUSTC-TIMING] cranelift_module test:false 0.348
2025-05-08T13:46:56.9655060Z �[1m�[32m   Compiling�[0m cranelift-object v0.118.0
2025-05-08T13:46:57.5202320Z [RUSTC-TIMING] cranelift_object test:false 0.552
2025-05-08T13:46:58.4039250Z [RUSTC-TIMING] cranelift_frontend test:false 1.732
2025-05-08T13:47:54.3724330Z [RUSTC-TIMING] cranelift_codegen test:false 72.970
2025-05-08T13:47:54.3819550Z �[1m�[32m   Compiling�[0m rustc_codegen_cranelift v0.1.0 (/Users/runner/work/rust/rust/compiler/rustc_codegen_cranelift)
2025-05-08T13:48:11.4626320Z �[0m�[1m�[38;5;9merror�[0m�[0m�[1m: linking with `cc` failed: exit status: 1�[0m
2025-05-08T13:48:11.4627040Z �[0m  �[0m�[0m�[1m�[38;5;12m|�[0m
2025-05-08T13:48:11.4655380Z �[0m  �[0m�[0m�[1m�[38;5;12m= �[0m�[0m�[1mnote�[0m�[0m:  "cc" "-Wl,-exported_symbols_list" "-Wl,/var/folders/w5/_8wgjw3j5cg6mgrth3s2kg9m0000gn/T/rustcmcp2xw/list" "/var/folders/w5/_8wgjw3j5cg6mgrth3s2kg9m0000gn/T/rustcmcp2xw/symbols.o" "<1 object files omitted>" "/Users/runner/work/rust/rust/build/aarch64-apple-darwin/stage0-codegen/aarch64-apple-darwin/release/deps/rustc_codegen_cranelift-b15c61165536d833.8y3u4pq4q583k21r4o891eajv.rcgu.rmeta" "/Users/runner/work/rust/rust/build/aarch64-apple-darwin/stage0-codegen/aarch64-apple-darwin/release/deps/{libcranelift_native-f31891f35cd92336.rlib,libcranelift_object-593ba375fd7aa423.rlib,libobject-59e6da34f178a9d8.rlib,libcrc32fast-754b0d7985706cb0.rlib,libcfg_if-1d71ae575f7e0319.rlib,libmemchr-a7f29ee35edbf4bc.rlib,libcranelift_frontend-51cad2d86d61a70e.rlib,libcranelift_module-92ef458cd759dfe2.rlib,libanyhow-ab8b71bcf19b4421.rlib,libcranelift_codegen-2e601784203c7f13.rlib,libcranelift_assembler_x64-ea52ffe13a4b65e5.rlib,libtarget_lexicon-785301db9e6358f8.rlib,libcranelift_codegen_shared-05ad82b7eca83584.rlib,libregalloc2-bb4d0b587a33027e.rlib,libbumpalo-5a5cc5b64f2aabad.rlib,liblog-ecb60cf346cdc2ac.rlib,libsmallvec-40263346323bffe6.rlib,librustc_hash-b516b89ab9a7d6b3.rlib,liballocator_api2-78961c5bd9f58316.rlib,libgimli-5de4cd6f3f3dedea.rlib,libindexmap-ca49ddc68c96a2ba.rlib,libequivalent-8591a9e4196672c0.rlib,libhashbrown-4c8ea8ba893b37d0.rlib,libfoldhash-1f7efd45c13de509.rlib,libcranelift_control-5f41a037e2a41512.rlib,libarbitrary-9278e8e460ab801f.rlib,libcranelift_bforest-faaef89f732f5215.rlib,libcranelift_entity-0c308e7d0828cf87.rlib,libcranelift_bitset-db4f6ef73ec908ae.rlib}.rlib" "<sysroot>/lib/rustlib/aarch64-apple-darwin/lib/librustc_driver-44f0946cd3a3cff2.dylib" "<sysroot>/lib/rustlib/aarch64-apple-darwin/lib/{libcompiler_builtins-*}.rlib" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lm" "-lz" "-lc++" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-liconv" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lEnzymeStatic-20" "-lSystem" "-lc" "-lm" "-arch" "arm64" "-mmacosx-version-min=11.0.0" "-o" "/Users/runner/work/rust/rust/build/aarch64-apple-darwin/stage0-codegen/aarch64-apple-darwin/release/deps/librustc_codegen_cranelift-b15c61165536d833.dylib" "-Wl,-dead_strip" "-dynamiclib" "-Wl,-install_name" "-Wl,@rpath/librustc_codegen_cranelift-b15c61165536d833.dylib" "-nodefaultlibs" "-L/Users/runner/work/rust/rust/build/aarch64-apple-darwin/llvm/lib" "-Wl,-rpath,@loader_path/../lib"�[0m
2025-05-08T13:48:11.4680440Z �[0m  �[0m�[0m�[1m�[38;5;12m= �[0m�[0m�[1mnote�[0m�[0m: some arguments are omitted. use `--verbose` to show all linker arguments�[0m
2025-05-08T13:48:11.4681140Z �[0m  �[0m�[0m�[1m�[38;5;12m= �[0m�[0m�[1mnote�[0m�[0m: ld: warning: ignoring duplicate libraries: '-lEnzymeStatic-20', '-lm'�[0m
2025-05-08T13:48:11.4681790Z �[0m          ld: library 'EnzymeStatic-20' not found�[0m
2025-05-08T13:48:11.4682500Z �[0m          clang: error: linker command failed with exit code 1 (use -v to see invocation)�[0m
2025-05-08T13:48:11.4682910Z �[0m          �[0m
2025-05-08T13:48:11.4683090Z 
2025-05-08T13:48:11.4734740Z [RUSTC-TIMING] rustc_codegen_cranelift test:false 17.088
2025-05-08T13:48:11.4742020Z �[1m�[31merror�[0m�[1m:�[0m could not compile `rustc_codegen_cranelift` (lib) due to 1 previous error

[ZuseZ4]

@bors try

[bors]

🔒 Merge conflict

This pull request and the master branch diverged in a way that cannot be automatically merged. Please rebase on top of the latest master branch, and let the reviewer approve again.

How do I rebase?

Assuming self is your fork and upstream is this repository, you can resolve the conflict following these steps:

1. git checkout enzyme-static (switch to your branch)
2. git fetch upstream master (retrieve the latest master)
3. git rebase upstream/master -p (rebase on top of it)
4. Follow the on-screen instruction to resolve conflicts (check git status if you got lost).
5. git push self enzyme-static --force-with-lease (update this PR)

You may also read Git Rebasing to Resolve Conflicts by Drew Blessing for a short tutorial.

Please avoid the "Resolve conflicts" button on GitHub. It uses git merge instead of git rebase which makes the PR commit history more difficult to read.

Sometimes step 4 will complete without asking for resolution. This is usually due to difference between how Cargo.lock conflict is handled during merge and rebase. This is normal, and you should still perform step 5 to update this PR.

Error message

Auto-merging src/ci/github-actions/jobs.yml
CONFLICT (content): Merge conflict in src/ci/github-actions/jobs.yml
Auto-merging src/ci/docker/host-x86_64/dist-x86_64-linux/Dockerfile
Auto-merging src/bootstrap/src/core/build_steps/llvm.rs
Auto-merging src/bootstrap/src/core/build_steps/compile.rs
CONFLICT (content): Merge conflict in src/bootstrap/src/core/build_steps/compile.rs
warning: inexact rename detection was skipped due to too many files.
warning: you may want to set your merge.renamelimit variable to at least 1707 and retry the command.
Automatic merge failed; fix conflicts and then commit the result.

[rustbot]

⚠️ Warning ⚠️

• Some commits in this PR modify submodules.

[ZuseZ4]

@bors try

[bors]

⌛ Trying commit f6a6f9a with merge c81a806...

[bors]

💔 Test failed - checks-actions

[ZuseZ4]

@bors try

[bors]

⌛ Trying commit b4244db with merge d6d2593...

[bors]

☀️ Try build successful - checks-actions
Build commit: d6d2593 (d6d25934826d196238db81f76f1e486c1ed882ff)

[Kobzol]

@rust-timer build d6d2593

[rust-timer]

Finished benchmarking commit (d6d2593): comparison URL.

Overall result: ❌ regressions - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	2.2%	[0.2%, 6.6%]	17
Regressions ❌ (secondary)	1.3%	[0.1%, 5.5%]	109
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.2%	[0.2%, 6.6%]	17

Max RSS (memory usage)

Results (primary 2.1%, secondary -3.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.1%	[2.1%, 2.1%]	1
Regressions ❌ (secondary)	2.9%	[1.7%, 4.2%]	2
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-5.4%	[-9.0%, -2.6%]	7
All ❌✅ (primary)	2.1%	[2.1%, 2.1%]	1

Cycles

Results (primary 5.8%, secondary 3.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	5.8%	[5.2%, 6.4%]	2
Regressions ❌ (secondary)	3.3%	[1.9%, 6.6%]	21
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	5.8%	[5.2%, 6.4%]	2

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 689.033s -> 688.949s (-0.01%)
Artifact size: 372.09 MiB -> 381.31 MiB (2.48%)