"cargo test" failing because of docblocks?

[daschl]

Hello,

As part of using rust-bindgen to build libcouchbase bindings, I'm running cargo test to run the automatically created sanity tests.

As it turns out the function tests are fine, but somehow the rust doc tests get tripped up by some docs in the c code (I think?).

~/code/rust/couchbase-rs/couchbase-sys$ cargo test
       Fresh rustc-serialize v0.3.22
       Fresh num-traits v0.1.36
       Fresh serde v0.8.21
       Fresh cfg-if v0.1.0
       Fresh winapi v0.2.8
       Fresh utf8-ranges v0.1.3
       Fresh dtoa v0.2.2
       Fresh lazy_static v0.2.2
       Fresh syntex_pos v0.50.0
       Fresh itoa v0.1.1
       Fresh nom v1.2.4
       Fresh bitflags v0.7.0
       Fresh glob v0.2.11
       Fresh phf_shared v0.7.20
       Fresh regex-syntax v0.3.9
       Fresh unicode-xid v0.0.3
       Fresh libc v0.2.18
       Fresh serde_json v0.8.4
       Fresh cexpr v0.2.0
       Fresh log v0.3.6
       Fresh phf v0.7.20
       Fresh pkg-config v0.3.8
       Fresh winapi-build v0.1.1
       Fresh memchr v0.1.11
       Fresh rand v0.3.15
       Fresh aho-corasick v0.5.3
       Fresh phf_generator v0.7.20
       Fresh phf_codegen v0.7.20
       Fresh kernel32-sys v0.2.2
       Fresh term v0.4.4
       Fresh thread-id v2.0.0
       Fresh syntex_errors v0.50.0
       Fresh thread_local v0.2.7
       Fresh syntex_syntax v0.50.0
       Fresh regex v0.1.80
       Fresh syntex v0.50.0
       Fresh aster v0.34.0
       Fresh quasi v0.26.0
       Fresh env_logger v0.3.5
       Fresh quasi_codegen v0.26.0
       Fresh target_build_utils v0.1.2
       Fresh libloading v0.3.1
       Fresh clang-sys v0.12.0
       Fresh libbindgen v0.1.5
   Compiling couchbase-sys v0.1.0 (file:///Users/daschl/code/rust/couchbase-rs/couchbase-sys)
     Running `rustc --crate-name couchbase_sys src/lib.rs -g --test -C metadata=85396d413f7666df -C extra-filename=-85396d413f7666df --out-dir /Users/daschl/code/rust/couchbase-rs/target/debug/deps --emit=dep-info,link -L dependency=/Users/daschl/code/rust/couchbase-rs/target/debug/deps -L native=/usr/local/Cellar/libcouchbase/2.6.4/lib -l couchbase`
    Finished debug [unoptimized + debuginfo] target(s) in 3.71 secs
     Running `/Users/daschl/code/rust/couchbase-rs/target/debug/deps/couchbase_sys-85396d413f7666df`

running 182 tests
test bindgen_test_layout___darwin_pthread_handler_rec ... ok
test bindgen_test_layout___sFILE ... ok
test bindgen_test_layout___sbuf ... ok
test bindgen_test_layout__bindgen_ty_1 ... ok
test bindgen_test_layout__bindgen_ty_14 ... ok
test bindgen_test_layout__bindgen_ty_15 ... ok
test bindgen_test_layout__bindgen_ty_18 ... ok
test bindgen_test_layout__bindgen_ty_19 ... ok
test bindgen_test_layout__bindgen_ty_21 ... ok
test bindgen_test_layout__bindgen_ty_23 ... ok
test bindgen_test_layout__bindgen_ty_24 ... ok
test bindgen_test_layout__bindgen_ty_26 ... ok
test bindgen_test_layout__bindgen_ty_27 ... ok
test bindgen_test_layout__bindgen_ty_28 ... ok
test bindgen_test_layout__bindgen_ty_29 ... ok
test bindgen_test_layout__bindgen_ty_30 ... ok
test bindgen_test_layout__bindgen_ty_31 ... ok
test bindgen_test_layout__bindgen_ty_33 ... ok
test bindgen_test_layout__bindgen_ty_34 ... ok
test bindgen_test_layout__bindgen_ty_35 ... ok
test bindgen_test_layout__bindgen_ty_36 ... ok
test bindgen_test_layout__bindgen_ty_37 ... ok
test bindgen_test_layout__bindgen_ty_38 ... ok
test bindgen_test_layout__bindgen_ty_39 ... ok
test bindgen_test_layout__bindgen_ty_4 ... ok
test bindgen_test_layout__bindgen_ty_41 ... ok
test bindgen_test_layout__bindgen_ty_44 ... ok
test bindgen_test_layout__bindgen_ty_45 ... ok
test bindgen_test_layout__bindgen_ty_51 ... ok
test bindgen_test_layout__bindgen_ty_52 ... ok
test bindgen_test_layout__bindgen_ty_53 ... ok
test bindgen_test_layout__bindgen_ty_54 ... ok
test bindgen_test_layout__bindgen_ty_67 ... ok
test bindgen_test_layout__bindgen_ty_68 ... ok
test bindgen_test_layout__bindgen_ty_69 ... ok
test bindgen_test_layout__bindgen_ty_69__bindgen_ty_1 ... ok
test bindgen_test_layout__bindgen_ty_7 ... ok
test bindgen_test_layout__bindgen_ty_70 ... ok
test bindgen_test_layout__bindgen_ty_71 ... ok
test bindgen_test_layout__bindgen_ty_72 ... ok
test bindgen_test_layout__bindgen_ty_73 ... ok
test bindgen_test_layout__bindgen_ty_74 ... ok
test bindgen_test_layout__bindgen_ty_74__bindgen_ty_1 ... ok
test bindgen_test_layout__bindgen_ty_75 ... ok
test bindgen_test_layout__bindgen_ty_76 ... ok
test bindgen_test_layout__bindgen_ty_77 ... ok
test bindgen_test_layout__bindgen_ty_77__bindgen_ty_1 ... ok
test bindgen_test_layout__bindgen_ty_78 ... ok
test bindgen_test_layout__bindgen_ty_79 ... ok
test bindgen_test_layout__bindgen_ty_8 ... ok
test bindgen_test_layout__bindgen_ty_80 ... ok
test bindgen_test_layout__bindgen_ty_80__bindgen_ty_1 ... ok
test bindgen_test_layout__bindgen_ty_82 ... ok
test bindgen_test_layout__bindgen_ty_83 ... ok
test bindgen_test_layout__bindgen_ty_84 ... ok
test bindgen_test_layout__bindgen_ty_85 ... ok
test bindgen_test_layout__bindgen_ty_85__bindgen_ty_1 ... ok
test bindgen_test_layout__bindgen_ty_86 ... ok
test bindgen_test_layout__bindgen_ty_87 ... ok
test bindgen_test_layout__bindgen_ty_88 ... ok
test bindgen_test_layout__bindgen_ty_88__bindgen_ty_1 ... ok
test bindgen_test_layout__bindgen_ty_89 ... ok
test bindgen_test_layout__bindgen_ty_9 ... ok
test bindgen_test_layout__bindgen_ty_90 ... ok
test bindgen_test_layout__bindgen_ty_90__bindgen_ty_1 ... ok
test bindgen_test_layout__bindgen_ty_91 ... ok
test bindgen_test_layout__bindgen_ty_92 ... ok
test bindgen_test_layout__bindgen_ty_93 ... ok
test bindgen_test_layout__bindgen_ty_94 ... ok
test bindgen_test_layout__bindgen_ty_95 ... ok
test bindgen_test_layout__bindgen_ty_96 ... ok
test bindgen_test_layout__bindgen_ty_97 ... ok
test bindgen_test_layout__bindgen_ty_98 ... ok
test bindgen_test_layout__bindgen_ty_99 ... ok
test bindgen_test_layout__bindgen_ty_99__bindgen_ty_1 ... ok
test bindgen_test_layout__opaque_pthread_attr_t ... ok
test bindgen_test_layout__opaque_pthread_cond_t ... ok
test bindgen_test_layout__opaque_pthread_condattr_t ... ok
test bindgen_test_layout__opaque_pthread_mutex_t ... ok
test bindgen_test_layout__opaque_pthread_mutexattr_t ... ok
test bindgen_test_layout__opaque_pthread_once_t ... ok
test bindgen_test_layout__opaque_pthread_rwlock_t ... ok
test bindgen_test_layout__opaque_pthread_rwlockattr_t ... ok
test bindgen_test_layout__opaque_pthread_t ... ok
test bindgen_test_layout_fd_set ... ok
test bindgen_test_layout_lcb_CMDBASE ... ok
test bindgen_test_layout_lcb_CONTIGBUF ... ok
test bindgen_test_layout_lcb_FRAGBUF ... ok
test bindgen_test_layout_lcb_KEYBUF ... ok
test bindgen_test_layout_lcb_MULTICMD_CTX_st ... ok
test bindgen_test_layout_lcb_MUTATION_TOKEN ... ok
test bindgen_test_layout_lcb_VALBUF ... ok
test bindgen_test_layout_lcb_VALBUF__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_arithmetic_cmd_st ... ok
test bindgen_test_layout_lcb_arithmetic_cmd_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_bsdprocs_st ... ok
test bindgen_test_layout_lcb_buf_info ... ok
test bindgen_test_layout_lcb_cached_config_st ... ok
test bindgen_test_layout_lcb_cntl_iops_info_st ... ok
test bindgen_test_layout_lcb_cntl_iops_info_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_cntl_iops_info_st__bindgen_ty_1__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_cntl_rdballocfactory ... ok
test bindgen_test_layout_lcb_cntl_server_st ... ok
test bindgen_test_layout_lcb_cntl_server_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_cntl_server_st__bindgen_ty_1__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_cntl_server_st__bindgen_ty_1__bindgen_ty_1__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_cntl_server_st__bindgen_ty_1__bindgen_ty_2 ... ok
test bindgen_test_layout_lcb_cntl_server_st__bindgen_ty_1__bindgen_ty_2__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_cntl_vbinfo_st ... ok
test bindgen_test_layout_lcb_cntl_vbinfo_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_cntl_vbinfo_st__bindgen_ty_1__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_create_io_ops_st ... ok
test bindgen_test_layout_lcb_create_io_ops_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_create_st ... ok
test bindgen_test_layout_lcb_create_st0 ... ok
test bindgen_test_layout_lcb_create_st1 ... ok
test bindgen_test_layout_lcb_create_st2 ... ok
test bindgen_test_layout_lcb_create_st3 ... ok
test bindgen_test_layout_lcb_create_st_lcb_CRST_u ... ok
test bindgen_test_layout_lcb_durability_cmd_st ... ok
test bindgen_test_layout_lcb_durability_cmd_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_durability_opts_st ... ok
test bindgen_test_layout_lcb_durability_opts_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_durability_resp_st ... ok
test bindgen_test_layout_lcb_durability_resp_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_evprocs_st ... ok
test bindgen_test_layout_lcb_flush_cmd_st ... ok
test bindgen_test_layout_lcb_flush_cmd_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_flush_cmd_st__bindgen_ty_1__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_flush_resp_st ... ok
test bindgen_test_layout_lcb_flush_resp_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_flush_resp_st__bindgen_ty_1__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_get_cmd_st ... ok
test bindgen_test_layout_lcb_get_cmd_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_get_replica_cmd_st ... ok
test bindgen_test_layout_lcb_get_replica_cmd_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_http_cmd_st ... ok
test bindgen_test_layout_lcb_http_cmd_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_io_opt_st ... ok
test bindgen_test_layout_lcb_io_opt_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_io_opt_st__bindgen_ty_1__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_io_writebuf_st ... ok
test bindgen_test_layout_lcb_iops2_st ... ok
test bindgen_test_layout_lcb_iops3_st ... ok
test bindgen_test_layout_lcb_iops_completion_st ... ok
test bindgen_test_layout_lcb_iops_evented_st ... ok
test bindgen_test_layout_lcb_iovec_st ... ok
test bindgen_test_layout_lcb_logprocs_st ... ok
test bindgen_test_layout_lcb_logprocs_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_logprocs_st__bindgen_ty_1__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_loopprocs_st ... ok
test bindgen_test_layout_lcb_memcached_st ... ok
test bindgen_test_layout_lcb_nameinfo_st ... ok
test bindgen_test_layout_lcb_nameinfo_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_nameinfo_st__bindgen_ty_2 ... ok
test bindgen_test_layout_lcb_observe_cmd_st ... ok
test bindgen_test_layout_lcb_observe_cmd_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_remove_cmd_st ... ok
test bindgen_test_layout_lcb_remove_cmd_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_server_stat_resp_st ... ok
test bindgen_test_layout_lcb_server_stat_resp_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_server_stats_cmd_st ... ok
test bindgen_test_layout_lcb_server_stats_cmd_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_server_version_cmd_st ... ok
test bindgen_test_layout_lcb_server_version_cmd_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_server_version_cmd_st__bindgen_ty_1__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_server_version_resp_st ... ok
test bindgen_test_layout_lcb_server_version_resp_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_server_version_resp_st__bindgen_ty_1__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_sockdata_st ... ok
test bindgen_test_layout_lcb_store_cmd_st ... ok
test bindgen_test_layout_lcb_store_cmd_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_timerprocs_st ... ok
test bindgen_test_layout_lcb_unlock_cmd_st ... ok
test bindgen_test_layout_lcb_unlock_cmd_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_verbosity_cmd_st ... ok
test bindgen_test_layout_lcb_verbosity_cmd_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_verbosity_resp_st ... ok
test bindgen_test_layout_lcb_verbosity_resp_st__bindgen_ty_1 ... ok
test bindgen_test_layout_lcb_verbosity_resp_st__bindgen_ty_1__bindgen_ty_1 ... ok
test bindgen_test_layout_timespec ... ok
test bindgen_test_layout_tm ... ok

test result: ok. 182 passed; 0 failed; 0 ignored; 0 measured

   Doc-tests couchbase-sys
     Running `rustdoc --test /Users/daschl/code/rust/couchbase-rs/couchbase-sys/src/lib.rs --crate-name couchbase_sys -L dependency=/Users/daschl/code/rust/couchbase-rs/target/debug/deps -L native=/usr/local/Cellar/libcouchbase/2.6.4/lib --extern couchbase_sys=/Users/daschl/code/rust/couchbase-rs/target/debug/deps/libcouchbase_sys-7eb681bf3d39aa71.rlib`

running 2 tests
test lcb_cntl_0 ... FAILED
test lcb_get3_0 ... FAILED

failures:

---- lcb_cntl_0 stdout ----
	error: incorrect close delimiter: `}`
 --> <anon>:6:1
  |
6 | }
  | ^
  |
note: unclosed delimiter
 --> <anon>:3:56
  |
3 |  will still compile with an older libcouchbase version (though
  |                                                        ^

error: expected one of `!`, `.`, `::`, `;`, `?`, `{`, `}`, or an operator, found `the`
 --> <anon>:2:8
  |
2 |  Using the actual value may be useful in ensuring your application
  |        ^^^

error[E0425]: unresolved name `Using`
 --> <anon>:2:2
  |
2 |  Using the actual value may be useful in ensuring your application
  |  ^^^^^ unresolved name

error: aborting due to previous error(s)

thread 'lcb_cntl_0' panicked at 'Box<Any>', /Users/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-mac/build/src/librustc/session/mod.rs:201
note: Run with `RUST_BACKTRACE=1` for a backtrace.

---- lcb_get3_0 stdout ----
	error: expected one of `!`, `)`, `,`, `.`, `::`, `?`, `{`, or an operator, found `->`
 --> <anon>:2:9
  |
2 | if (resp->rc != LCB_SUCCESS) {
  |         ^^

error: aborting due to previous error(s)

thread 'lcb_get3_0' panicked at 'Box<Any>', /Users/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-mac/build/src/librustc/session/mod.rs:201


failures:
    lcb_cntl_0
    lcb_get3_0

test result: FAILED. 0 passed; 2 failed; 0 ignored; 0 measured

error: test failed

So my question is: do those doc tests even make sense? If so, I'm happy to help getting it to work, if not maybe there is an option to add no_run to those to avoid the test suite failing?

Thanks!

[emilio]

So as you said, these doc tests doesn't make any sense, given they're C code... Is there any kind of compiler annotation that we could use to tell rustc "don't run doctests in this module"?

Easiest short-term solution is adding an option to avoid generating comments, but that's... not ideal.

I can imagine how could we process comments so no_run is added, but that solution would be a bit more flaky I think :/.

[daschl]

I don't think there is something we can put in rust code directly which would ignore the doc tests of a module (would be ideal), for now I think turning off doctests for the -sys crate works good enough:

[lib]
doctest = false

But of course this also rules out any doctests which one could add on top of the auto-generated ones, so I think a "code fix" is needed at some point.

[emilio]

I'm going to try to patch rustdoc upstream to support: #[doc(test(ignore))] On testable items. This will solve the issue nicely I think.

…

On Wed, Jan 04, 2017 at 04:00:35AM -0800, Michael Nitschinger wrote: I don't think there is something we can put in rust code directly which would ignore the doc tests of a module (would be ideal), for now I think turning off doctests for the -sys crate works good enough: ```toml [lib] doctest = false ``` But of course this also rules out any doctests which one could add on top of the auto-generated ones, so I think a "code fix" is needed at some point. -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: #378 (comment)

[daschl]

That would be great - how would that work in this autogenerated context? Would libbindgen add those options or would it be possible to do that on the module where we "inject" the code?

[emilio]

You could either:

#[doc(test(ignore))]
mod bindings {
    include!("/path/to/autogenerated/bindings.rs")
}

or use the --raw-line API to add it to the top level:

$ bindgen --raw-line "#![doc(test(ignore))]"

or

builder.raw_line("#![doc(test(ignore))]");

[daschl]

Sounds great 👍 looking forward to it!

[emilio]

rust-lang/rust#38825