(stdout not captured) (stderr not captured) (stdout not captured) (stderr not captured) thread 'bootstrap::server::tests::test_persistent_sled_agent_request_schema' (398955) panicked at sled-agent/src/bootstrap/server.rs:822:9: assertion failed: string doesn't match the contents of file: "../schema/start-sled-agent-request.json" see diffset above set EXPECTORATE=overwrite if these changes are intentional stack backtrace: 0: __rustc::rust_begin_unwind at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:689:5 1: core::panicking::panic_fmt at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:80:14 2: expectorate::assert_contents::<&str> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/expectorate-1.2.0/src/lib.rs:67:9 3: test_persistent_sled_agent_request_schema at ./src/bootstrap/server.rs:822:9 4: omicron_sled_agent::bootstrap::server::tests::test_persistent_sled_agent_request_schema::{closure#0} at ./src/bootstrap/server.rs:820:51 5: <omicron_sled_agent::bootstrap::server::tests::test_persistent_sled_agent_request_schema::{closure#0} as core::ops::function::FnOnce<()>>::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 6: core::ops::function::FnOnce::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. thread 'bootstrap::server::tests::test_persistent_sled_agent_request_schema' (399604) panicked at sled-agent/src/bootstrap/server.rs:822:9: assertion failed: string doesn't match the contents of file: "../schema/start-sled-agent-request.json" see diffset above set EXPECTORATE=overwrite if these changes are intentional stack backtrace: 0: __rustc::rust_begin_unwind at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:689:5 1: core::panicking::panic_fmt at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:80:14 2: expectorate::assert_contents::<&str> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/expectorate-1.2.0/src/lib.rs:67:9 3: test_persistent_sled_agent_request_schema at ./src/bootstrap/server.rs:822:9 4: omicron_sled_agent::bootstrap::server::tests::test_persistent_sled_agent_request_schema::{closure#0} at ./src/bootstrap/server.rs:820:51 5: <omicron_sled_agent::bootstrap::server::tests::test_persistent_sled_agent_request_schema::{closure#0} as core::ops::function::FnOnce<()>>::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 6: core::ops::function::FnOnce::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. running 1 test @@ -49,10 +49,19 @@ "net": { "$ref": "#/definitions/Ipv6Net" } } }, + "RackUuid": { + "type": "string", + "format": "uuid", + "x-rust-type": { + "crate": "omicron-uuid-kinds", + "path": "omicron_uuid_kinds::RackUuid", + "version": "*" + } + }, "SledUuid": { "type": "string", "format": "uuid", "x-rust-type": { "crate": "omicron-uuid-kinds", @@ -83,12 +92,15 @@ "description": "Is this node an LRTQ learner node?\n\nWe only put the node into learner mode if `use_trust_quorum` is also true.", "type": "boolean" }, "rack_id": { "description": "Uuid of the rack to which this sled agent belongs.", - "type": "string", - "format": "uuid" + "allOf": [ + { + "$ref": "#/definitions/RackUuid" + } + ] }, "subnet": { "description": "Portion of the IP space to be managed by the Sled Agent.", "allOf": [ { test bootstrap::server::tests::test_persistent_sled_agent_request_schema ... FAILED failures: failures: bootstrap::server::tests::test_persistent_sled_agent_request_schema test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 57 filtered out; finished in 0.13s thread 'bootstrap::server::tests::test_persistent_sled_agent_request_schema' (399604) panicked at sled-agent/src/bootstrap/server.rs:822:9: assertion failed: string doesn't match the contents of file: "../schema/start-sled-agent-request.json" see diffset above set EXPECTORATE=overwrite if these changes are intentional stack backtrace: 0: __rustc::rust_begin_unwind at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:689:5 1: core::panicking::panic_fmt at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:80:14 2: expectorate::assert_contents::<&str> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/expectorate-1.2.0/src/lib.rs:67:9 3: test_persistent_sled_agent_request_schema at ./src/bootstrap/server.rs:822:9 4: omicron_sled_agent::bootstrap::server::tests::test_persistent_sled_agent_request_schema::{closure#0} at ./src/bootstrap/server.rs:820:51 5: <omicron_sled_agent::bootstrap::server::tests::test_persistent_sled_agent_request_schema::{closure#0} as core::ops::function::FnOnce<()>>::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 6: core::ops::function::FnOnce::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. thread 'bootstrap::server::tests::test_persistent_sled_agent_request_schema' (399624) panicked at sled-agent/src/bootstrap/server.rs:822:9: assertion failed: string doesn't match the contents of file: "../schema/start-sled-agent-request.json" see diffset above set EXPECTORATE=overwrite if these changes are intentional stack backtrace: 0: __rustc::rust_begin_unwind at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:689:5 1: core::panicking::panic_fmt at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:80:14 2: expectorate::assert_contents::<&str> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/expectorate-1.2.0/src/lib.rs:67:9 3: test_persistent_sled_agent_request_schema at ./src/bootstrap/server.rs:822:9 4: omicron_sled_agent::bootstrap::server::tests::test_persistent_sled_agent_request_schema::{closure#0} at ./src/bootstrap/server.rs:820:51 5: <omicron_sled_agent::bootstrap::server::tests::test_persistent_sled_agent_request_schema::{closure#0} as core::ops::function::FnOnce<()>>::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 6: core::ops::function::FnOnce::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. running 1 test @@ -49,10 +49,19 @@ "net": { "$ref": "#/definitions/Ipv6Net" } } }, + "RackUuid": { + "type": "string", + "format": "uuid", + "x-rust-type": { + "crate": "omicron-uuid-kinds", + "path": "omicron_uuid_kinds::RackUuid", + "version": "*" + } + }, "SledUuid": { "type": "string", "format": "uuid", "x-rust-type": { "crate": "omicron-uuid-kinds", @@ -83,12 +92,15 @@ "description": "Is this node an LRTQ learner node?\n\nWe only put the node into learner mode if `use_trust_quorum` is also true.", "type": "boolean" }, "rack_id": { "description": "Uuid of the rack to which this sled agent belongs.", - "type": "string", - "format": "uuid" + "allOf": [ + { + "$ref": "#/definitions/RackUuid" + } + ] }, "subnet": { "description": "Portion of the IP space to be managed by the Sled Agent.", "allOf": [ { test bootstrap::server::tests::test_persistent_sled_agent_request_schema ... FAILED failures: failures: bootstrap::server::tests::test_persistent_sled_agent_request_schema test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 57 filtered out; finished in 0.13s thread 'bootstrap::server::tests::test_persistent_sled_agent_request_schema' (399624) panicked at sled-agent/src/bootstrap/server.rs:822:9: assertion failed: string doesn't match the contents of file: "../schema/start-sled-agent-request.json" see diffset above set EXPECTORATE=overwrite if these changes are intentional stack backtrace: 0: __rustc::rust_begin_unwind at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:689:5 1: core::panicking::panic_fmt at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:80:14 2: expectorate::assert_contents::<&str> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/expectorate-1.2.0/src/lib.rs:67:9 3: test_persistent_sled_agent_request_schema at ./src/bootstrap/server.rs:822:9 4: omicron_sled_agent::bootstrap::server::tests::test_persistent_sled_agent_request_schema::{closure#0} at ./src/bootstrap/server.rs:820:51 5: <omicron_sled_agent::bootstrap::server::tests::test_persistent_sled_agent_request_schema::{closure#0} as core::ops::function::FnOnce<()>>::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 6: core::ops::function::FnOnce::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. running 1 test @@ -49,10 +49,19 @@ "net": { "$ref": "#/definitions/Ipv6Net" } } }, + "RackUuid": { + "type": "string", + "format": "uuid", + "x-rust-type": { + "crate": "omicron-uuid-kinds", + "path": "omicron_uuid_kinds::RackUuid", + "version": "*" + } + }, "SledUuid": { "type": "string", "format": "uuid", "x-rust-type": { "crate": "omicron-uuid-kinds", @@ -83,12 +92,15 @@ "description": "Is this node an LRTQ learner node?\n\nWe only put the node into learner mode if `use_trust_quorum` is also true.", "type": "boolean" }, "rack_id": { "description": "Uuid of the rack to which this sled agent belongs.", - "type": "string", - "format": "uuid" + "allOf": [ + { + "$ref": "#/definitions/RackUuid" + } + ] }, "subnet": { "description": "Portion of the IP space to be managed by the Sled Agent.", "allOf": [ { test bootstrap::server::tests::test_persistent_sled_agent_request_schema ... FAILED failures: failures: bootstrap::server::tests::test_persistent_sled_agent_request_schema test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 57 filtered out; finished in 1.20s thread 'bootstrap::server::tests::test_persistent_sled_agent_request_schema' (398955) panicked at sled-agent/src/bootstrap/server.rs:822:9: assertion failed: string doesn't match the contents of file: "../schema/start-sled-agent-request.json" see diffset above set EXPECTORATE=overwrite if these changes are intentional stack backtrace: 0: __rustc::rust_begin_unwind at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:689:5 1: core::panicking::panic_fmt at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:80:14 2: expectorate::assert_contents::<&str> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/expectorate-1.2.0/src/lib.rs:67:9 3: test_persistent_sled_agent_request_schema at ./src/bootstrap/server.rs:822:9 4: omicron_sled_agent::bootstrap::server::tests::test_persistent_sled_agent_request_schema::{closure#0} at ./src/bootstrap/server.rs:820:51 5: <omicron_sled_agent::bootstrap::server::tests::test_persistent_sled_agent_request_schema::{closure#0} as core::ops::function::FnOnce<()>>::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 6: core::ops::function::FnOnce::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. thread 'test_omdb_success_cases' (397231) panicked at dev-tools/omdb/tests/test_all_output.rs:451:5: assertion failed: string doesn't match the contents of file: "tests/successes.out" see diffset above set EXPECTORATE=overwrite if these changes are intentional stack backtrace: 0: __rustc::rust_begin_unwind at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:689:5 1: core::panicking::panic_fmt at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:80:14 2: expectorate::assert_contents::<&str> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/expectorate-1.2.0/src/lib.rs:67:9 3: {async_block#0} at ./tests/test_all_output.rs:451:5 4: poll<&mut dyn core::future::future::Future<Output=()>> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/future/future.rs:133:9 5: poll<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/future/future.rs:133:9 6: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:778:70 7: with_budget<core::task::poll::Poll<()>, tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure#0}::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/task/coop/mod.rs:167:5 8: budget<core::task::poll::Poll<()>, tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure#0}::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/task/coop/mod.rs:133:5 9: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:778:25 10: <tokio::runtime::scheduler::current_thread::Context>::enter::<core::task::poll::Poll<()>, <tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}::{closure#0}> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:451:19 11: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:777:44 12: <tokio::runtime::scheduler::current_thread::CoreGuard>::enter::<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0} at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:865:68 13: <tokio::runtime::context::scoped::Scoped<tokio::runtime::scheduler::Context>>::set::<<tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>)> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context/scoped.rs:40:9 14: tokio::runtime::context::set_scheduler::<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}>::{closure#0} at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context.rs:181:38 15: try_with<tokio::runtime::context::Context, tokio::runtime::context::set_scheduler::{closure_env#0}<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>), tokio::runtime::scheduler::current_thread::{impl#9}::enter::{closure_env#0}<tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>, core::option::Option<()>>>, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>)> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/local.rs:513:12 16: <std::thread::local::LocalKey<tokio::runtime::context::Context>>::with::<tokio::runtime::context::set_scheduler<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}>::{closure#0}, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>)> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/local.rs:477:20 17: tokio::runtime::context::set_scheduler::<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context.rs:181:17 18: <tokio::runtime::scheduler::current_thread::CoreGuard>::enter::<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:865:27 19: <tokio::runtime::scheduler::current_thread::CoreGuard>::block_on::<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:765:24 20: {closure#0}<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:205:33 21: tokio::runtime::context::runtime::enter_runtime::<<tokio::runtime::scheduler::current_thread::CurrentThread>::block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>::{closure#0}, ()> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context/runtime.rs:65:16 22: block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:193:9 23: <tokio::runtime::runtime::Runtime>::block_on_inner::<core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/runtime.rs:371:52 24: block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/runtime.rs:345:18 25: test_omdb_success_cases at ./tests/test_all_output.rs:609:31 26: test_all_output::test_omdb_success_cases::{closure#0} at ./tests/test_all_output.rs:136:35 27: <test_all_output::test_omdb_success_cases::{closure#0} as core::ops::function::FnOnce<()>>::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 28: core::ops::function::FnOnce::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. WARN: dropped CockroachInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: temporary directory leaked: "/var/tmp/omicron_tmp/.tmp7HXJfb" If you would like to access the database for debugging, run the following: # Run the database cargo xtask db-dev run --no-populate --store-dir "/var/tmp/omicron_tmp/.tmp7HXJfb/data" # Access the database. Note the port may change if you run multiple databases. cockroach sql --host=localhost:32221 --insecure WARN: dropped ClickHouse process without cleaning it up first (there may still be a child process running (PID 397662) and a temporary directory leaked, /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.397230.1-clickhouse-Q4iMVM) WARN: dropped DendriteInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: dendrite temporary directory leaked: /var/tmp/omicron_tmp/.tmpc16QbK WARN: dropped DendriteInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: dendrite temporary directory leaked: /var/tmp/omicron_tmp/.tmpwpN53q WARN: dropped MgdInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: mgd temporary directory leaked: /var/tmp/omicron_tmp/.tmpLLeThm WARN: dropped MgdInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: mgd temporary directory leaked: /var/tmp/omicron_tmp/.tmpfoTmPx WARN: dropped DdmInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: ddmd temporary directory leaked: /var/tmp/omicron_tmp/.tmpKvShAk WARN: dropped DdmInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: ddmd temporary directory leaked: /var/tmp/omicron_tmp/.tmp7lhIwh thread 'test_omdb_success_cases' (417528) panicked at dev-tools/omdb/tests/test_all_output.rs:451:5: assertion failed: string doesn't match the contents of file: "tests/successes.out" see diffset above set EXPECTORATE=overwrite if these changes are intentional stack backtrace: 0: __rustc::rust_begin_unwind at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:689:5 1: core::panicking::panic_fmt at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:80:14 2: expectorate::assert_contents::<&str> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/expectorate-1.2.0/src/lib.rs:67:9 3: {async_block#0} at ./tests/test_all_output.rs:451:5 4: poll<&mut dyn core::future::future::Future<Output=()>> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/future/future.rs:133:9 5: poll<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/future/future.rs:133:9 6: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:778:70 7: with_budget<core::task::poll::Poll<()>, tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure#0}::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/task/coop/mod.rs:167:5 8: budget<core::task::poll::Poll<()>, tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure#0}::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/task/coop/mod.rs:133:5 9: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:778:25 10: <tokio::runtime::scheduler::current_thread::Context>::enter::<core::task::poll::Poll<()>, <tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}::{closure#0}> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:451:19 11: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:777:44 12: <tokio::runtime::scheduler::current_thread::CoreGuard>::enter::<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0} at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:865:68 13: <tokio::runtime::context::scoped::Scoped<tokio::runtime::scheduler::Context>>::set::<<tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>)> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context/scoped.rs:40:9 14: tokio::runtime::context::set_scheduler::<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}>::{closure#0} at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context.rs:181:38 15: try_with<tokio::runtime::context::Context, tokio::runtime::context::set_scheduler::{closure_env#0}<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>), tokio::runtime::scheduler::current_thread::{impl#9}::enter::{closure_env#0}<tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>, core::option::Option<()>>>, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>)> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/local.rs:513:12 16: <std::thread::local::LocalKey<tokio::runtime::context::Context>>::with::<tokio::runtime::context::set_scheduler<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}>::{closure#0}, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>)> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/local.rs:477:20 17: tokio::runtime::context::set_scheduler::<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context.rs:181:17 18: <tokio::runtime::scheduler::current_thread::CoreGuard>::enter::<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:865:27 19: <tokio::runtime::scheduler::current_thread::CoreGuard>::block_on::<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:765:24 20: {closure#0}<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:205:33 21: tokio::runtime::context::runtime::enter_runtime::<<tokio::runtime::scheduler::current_thread::CurrentThread>::block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>::{closure#0}, ()> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context/runtime.rs:65:16 22: block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:193:9 23: <tokio::runtime::runtime::Runtime>::block_on_inner::<core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/runtime.rs:371:52 24: block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/runtime.rs:345:18 25: test_omdb_success_cases at ./tests/test_all_output.rs:609:31 26: test_all_output::test_omdb_success_cases::{closure#0} at ./tests/test_all_output.rs:136:35 27: <test_all_output::test_omdb_success_cases::{closure#0} as core::ops::function::FnOnce<()>>::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 28: core::ops::function::FnOnce::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. WARN: dropped CockroachInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: temporary directory leaked: "/var/tmp/omicron_tmp/.tmpaiO0g9" If you would like to access the database for debugging, run the following: # Run the database cargo xtask db-dev run --no-populate --store-dir "/var/tmp/omicron_tmp/.tmpaiO0g9/data" # Access the database. Note the port may change if you run multiple databases. cockroach sql --host=localhost:32221 --insecure WARN: dropped ClickHouse process without cleaning it up first (there may still be a child process running (PID 419010) and a temporary directory leaked, /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.417460.1-clickhouse-Xa0Apo) WARN: dropped DendriteInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: dendrite temporary directory leaked: /var/tmp/omicron_tmp/.tmpiMc2Xh WARN: dropped DendriteInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: dendrite temporary directory leaked: /var/tmp/omicron_tmp/.tmpVuWCrR WARN: dropped MgdInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: mgd temporary directory leaked: /var/tmp/omicron_tmp/.tmpnMyaJk WARN: dropped MgdInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: mgd temporary directory leaked: /var/tmp/omicron_tmp/.tmpvi2I9H WARN: dropped DdmInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: ddmd temporary directory leaked: /var/tmp/omicron_tmp/.tmphGxmux WARN: dropped DdmInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: ddmd temporary directory leaked: /var/tmp/omicron_tmp/.tmpVsTTT6 running 1 test running commands with args: ["db", "db-metadata", "ls-nexus"] running commands with args: ["--destructive", "db", "db-metadata", "force-mark-nexus-quiesced", "--skip-confirmation", "913233fe-92a8-4635-9572-183f495429c4"] running commands with args: ["db", "disks", "list"] running commands with args: ["db", "dns", "show"] running commands with args: ["db", "dns", "diff", "external", "2"] running commands with args: ["db", "dns", "names", "external", "2"] running commands with args: ["db", "instances"] running commands with args: ["db", "sleds"] running commands with args: ["db", "sleds", "-F", "discretionary"] running commands with args: ["mgs", "inventory"] running commands with args: ["nexus", "background-tasks", "doc"] running commands with args: ["nexus", "background-tasks", "show", "--no-executing-info"] running commands with args: ["nexus", "background-tasks", "show", "saga_recovery", "--no-executing-info"] running commands with args: ["nexus", "background-tasks", "show", "blueprint_loader", "blueprint_executor", "--no-executing-info"] running commands with args: ["nexus", "background-tasks", "show", "dns_internal", "--no-executing-info"] running commands with args: ["nexus", "background-tasks", "show", "dns_external", "--no-executing-info"] running commands with args: ["nexus", "background-tasks", "show", "all", "--no-executing-info"] running commands with args: ["nexus", "sagas", "list"] running commands with args: ["--destructive", "nexus", "sagas", "demo-create"] running commands with args: ["nexus", "sagas", "list"] running commands with args: ["--destructive", "nexus", "background-tasks", "activate", "inventory_collection"] running commands with args: ["nexus", "blueprints", "list"] running commands with args: ["nexus", "blueprints", "show", "26db196a-b4eb-42e5-b5e3-9603ee511976"] running commands with args: ["nexus", "blueprints", "show", "current-target"] running commands with args: ["nexus", "blueprints", "diff", "26db196a-b4eb-42e5-b5e3-9603ee511976", "current-target"] running commands with args: ["nexus", "blueprints", "diff", "26db196a-b4eb-42e5-b5e3-9603ee511976"] running commands with args: ["nexus", "reconfigurator-config", "show", "current"] running commands with args: ["nexus", "update-status"] running commands with args: ["nexus", "update-status", "--details"] running commands with args: ["-w", "nexus", "reconfigurator-config", "set", "--planner-enabled", "true"] running commands with args: ["-w", "nexus", "reconfigurator-config", "set", "--disruption-policy", "migrate-or-terminate"] running commands with args: ["nexus", "reconfigurator-config", "show", "current"] running commands with args: ["reconfigurator", "export", "/var/tmp/omicron_tmp/.tmpRaqgmU/reconfigurator-export.out"] running commands with args: ["db", "whatis", "001de000-5110-4000-8000-000000000000", "001de000-05e4-4000-8000-000000004007"] running commands with args: ["--destructive", "db", "db-metadata", "force-mark-nexus-quiesced", "--skip-confirmation", "--skip-blueprint-validation", "913233fe-92a8-4635-9572-183f495429c4"] @@ -1013,11 +1013,11 @@ task: "switch_port_config_manager" configured period: every <REDACTED_DURATION>s last completed activation: <REDACTED ITERATIONS>, triggered by <TRIGGERED_BY_REDACTED> started at <REDACTED_TIMESTAMP> (<REDACTED DURATION>s ago) and ran for <REDACTED DURATION>ms -warning: unknown background task: "switch_port_config_manager" (don't know how to interpret details: Object {}) +warning: unknown background task: "switch_port_config_manager" (don't know how to interpret details: Object {"incomplete_bootstore_configs": Array []}) task: "trust_quorum_manager" configured period: every <REDACTED_DURATION>m last completed activation: <REDACTED ITERATIONS>, triggered by <TRIGGERED_BY_REDACTED> started at <REDACTED_TIMESTAMP> (<REDACTED DURATION>s ago) and ran for <REDACTED DURATION>ms @@ -1712,11 +1712,11 @@ task: "switch_port_config_manager" configured period: every <REDACTED_DURATION>s last completed activation: <REDACTED ITERATIONS>, triggered by <TRIGGERED_BY_REDACTED> started at <REDACTED_TIMESTAMP> (<REDACTED DURATION>s ago) and ran for <REDACTED DURATION>ms -warning: unknown background task: "switch_port_config_manager" (don't know how to interpret details: Object {}) +warning: unknown background task: "switch_port_config_manager" (don't know how to interpret details: Object {"incomplete_bootstore_configs": Array []}) task: "trust_quorum_manager" configured period: every <REDACTED_DURATION>m last completed activation: <REDACTED ITERATIONS>, triggered by <TRIGGERED_BY_REDACTED> started at <REDACTED_TIMESTAMP> (<REDACTED DURATION>s ago) and ran for <REDACTED DURATION>ms test test_omdb_success_cases ... FAILED failures: failures: test_omdb_success_cases test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 2 filtered out; finished in 17.91s log file: /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.417460.0.log note: configured to log to "/var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.417460.0.log" DB URL: postgresql://root@[::1]:46281/omicron?sslmode=disable DB address: [::1]:46281 log file: /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.417460.2.log note: configured to log to "/var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.417460.2.log" log file: /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.417460.3.log note: configured to log to "/var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.417460.3.log" thread 'test_omdb_success_cases' (417528) panicked at dev-tools/omdb/tests/test_all_output.rs:451:5: assertion failed: string doesn't match the contents of file: "tests/successes.out" see diffset above set EXPECTORATE=overwrite if these changes are intentional stack backtrace: 0: __rustc::rust_begin_unwind at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:689:5 1: core::panicking::panic_fmt at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:80:14 2: expectorate::assert_contents::<&str> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/expectorate-1.2.0/src/lib.rs:67:9 3: {async_block#0} at ./tests/test_all_output.rs:451:5 4: poll<&mut dyn core::future::future::Future<Output=()>> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/future/future.rs:133:9 5: poll<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/future/future.rs:133:9 6: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:778:70 7: with_budget<core::task::poll::Poll<()>, tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure#0}::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/task/coop/mod.rs:167:5 8: budget<core::task::poll::Poll<()>, tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure#0}::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/task/coop/mod.rs:133:5 9: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:778:25 10: <tokio::runtime::scheduler::current_thread::Context>::enter::<core::task::poll::Poll<()>, <tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}::{closure#0}> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:451:19 11: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:777:44 12: <tokio::runtime::scheduler::current_thread::CoreGuard>::enter::<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0} at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:865:68 13: <tokio::runtime::context::scoped::Scoped<tokio::runtime::scheduler::Context>>::set::<<tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>)> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context/scoped.rs:40:9 14: tokio::runtime::context::set_scheduler::<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}>::{closure#0} at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context.rs:181:38 15: try_with<tokio::runtime::context::Context, tokio::runtime::context::set_scheduler::{closure_env#0}<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>), tokio::runtime::scheduler::current_thread::{impl#9}::enter::{closure_env#0}<tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>, core::option::Option<()>>>, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>)> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/local.rs:513:12 16: <std::thread::local::LocalKey<tokio::runtime::context::Context>>::with::<tokio::runtime::context::set_scheduler<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}>::{closure#0}, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>)> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/local.rs:477:20 17: tokio::runtime::context::set_scheduler::<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context.rs:181:17 18: <tokio::runtime::scheduler::current_thread::CoreGuard>::enter::<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:865:27 19: <tokio::runtime::scheduler::current_thread::CoreGuard>::block_on::<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:765:24 20: {closure#0}<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:205:33 21: tokio::runtime::context::runtime::enter_runtime::<<tokio::runtime::scheduler::current_thread::CurrentThread>::block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>::{closure#0}, ()> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context/runtime.rs:65:16 22: block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:193:9 23: <tokio::runtime::runtime::Runtime>::block_on_inner::<core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/runtime.rs:371:52 24: block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/runtime.rs:345:18 25: test_omdb_success_cases at ./tests/test_all_output.rs:609:31 26: test_all_output::test_omdb_success_cases::{closure#0} at ./tests/test_all_output.rs:136:35 27: <test_all_output::test_omdb_success_cases::{closure#0} as core::ops::function::FnOnce<()>>::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 28: core::ops::function::FnOnce::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. WARN: dropped CockroachInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: temporary directory leaked: "/var/tmp/omicron_tmp/.tmpaiO0g9" If you would like to access the database for debugging, run the following: # Run the database cargo xtask db-dev run --no-populate --store-dir "/var/tmp/omicron_tmp/.tmpaiO0g9/data" # Access the database. Note the port may change if you run multiple databases. cockroach sql --host=localhost:32221 --insecure WARN: dropped ClickHouse process without cleaning it up first (there may still be a child process running (PID 419010) and a temporary directory leaked, /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.417460.1-clickhouse-Xa0Apo) WARN: dropped DendriteInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: dendrite temporary directory leaked: /var/tmp/omicron_tmp/.tmpiMc2Xh WARN: dropped DendriteInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: dendrite temporary directory leaked: /var/tmp/omicron_tmp/.tmpVuWCrR WARN: dropped MgdInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: mgd temporary directory leaked: /var/tmp/omicron_tmp/.tmpnMyaJk WARN: dropped MgdInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: mgd temporary directory leaked: /var/tmp/omicron_tmp/.tmpvi2I9H WARN: dropped DdmInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: ddmd temporary directory leaked: /var/tmp/omicron_tmp/.tmphGxmux WARN: dropped DdmInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: ddmd temporary directory leaked: /var/tmp/omicron_tmp/.tmpVsTTT6 thread 'test_omdb_success_cases' (424676) panicked at dev-tools/omdb/tests/test_all_output.rs:451:5: assertion failed: string doesn't match the contents of file: "tests/successes.out" see diffset above set EXPECTORATE=overwrite if these changes are intentional stack backtrace: 0: __rustc::rust_begin_unwind at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:689:5 1: core::panicking::panic_fmt at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:80:14 2: expectorate::assert_contents::<&str> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/expectorate-1.2.0/src/lib.rs:67:9 3: {async_block#0} at ./tests/test_all_output.rs:451:5 4: poll<&mut dyn core::future::future::Future<Output=()>> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/future/future.rs:133:9 5: poll<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/future/future.rs:133:9 6: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:778:70 7: with_budget<core::task::poll::Poll<()>, tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure#0}::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/task/coop/mod.rs:167:5 8: budget<core::task::poll::Poll<()>, tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure#0}::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/task/coop/mod.rs:133:5 9: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:778:25 10: <tokio::runtime::scheduler::current_thread::Context>::enter::<core::task::poll::Poll<()>, <tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}::{closure#0}> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:451:19 11: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:777:44 12: <tokio::runtime::scheduler::current_thread::CoreGuard>::enter::<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0} at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:865:68 13: <tokio::runtime::context::scoped::Scoped<tokio::runtime::scheduler::Context>>::set::<<tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>)> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context/scoped.rs:40:9 14: tokio::runtime::context::set_scheduler::<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}>::{closure#0} at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context.rs:181:38 15: try_with<tokio::runtime::context::Context, tokio::runtime::context::set_scheduler::{closure_env#0}<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>), tokio::runtime::scheduler::current_thread::{impl#9}::enter::{closure_env#0}<tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>, core::option::Option<()>>>, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>)> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/local.rs:513:12 16: <std::thread::local::LocalKey<tokio::runtime::context::Context>>::with::<tokio::runtime::context::set_scheduler<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}>::{closure#0}, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>)> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/local.rs:477:20 17: tokio::runtime::context::set_scheduler::<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context.rs:181:17 18: <tokio::runtime::scheduler::current_thread::CoreGuard>::enter::<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:865:27 19: <tokio::runtime::scheduler::current_thread::CoreGuard>::block_on::<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:765:24 20: {closure#0}<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:205:33 21: tokio::runtime::context::runtime::enter_runtime::<<tokio::runtime::scheduler::current_thread::CurrentThread>::block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>::{closure#0}, ()> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context/runtime.rs:65:16 22: block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:193:9 23: <tokio::runtime::runtime::Runtime>::block_on_inner::<core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/runtime.rs:371:52 24: block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/runtime.rs:345:18 25: test_omdb_success_cases at ./tests/test_all_output.rs:609:31 26: test_all_output::test_omdb_success_cases::{closure#0} at ./tests/test_all_output.rs:136:35 27: <test_all_output::test_omdb_success_cases::{closure#0} as core::ops::function::FnOnce<()>>::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 28: core::ops::function::FnOnce::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. WARN: dropped CockroachInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: temporary directory leaked: "/var/tmp/omicron_tmp/.tmpRaakCX" If you would like to access the database for debugging, run the following: # Run the database cargo xtask db-dev run --no-populate --store-dir "/var/tmp/omicron_tmp/.tmpRaakCX/data" # Access the database. Note the port may change if you run multiple databases. cockroach sql --host=localhost:32221 --insecure WARN: dropped ClickHouse process without cleaning it up first (there may still be a child process running (PID 424729) and a temporary directory leaked, /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.424675.1-clickhouse-Z3yXCV) WARN: dropped DendriteInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: dendrite temporary directory leaked: /var/tmp/omicron_tmp/.tmpSz1oDE WARN: dropped DendriteInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: dendrite temporary directory leaked: /var/tmp/omicron_tmp/.tmpzwDeDv WARN: dropped MgdInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: mgd temporary directory leaked: /var/tmp/omicron_tmp/.tmpb5lGB3 WARN: dropped MgdInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: mgd temporary directory leaked: /var/tmp/omicron_tmp/.tmpXv86Uw WARN: dropped DdmInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: ddmd temporary directory leaked: /var/tmp/omicron_tmp/.tmpxsJzVF WARN: dropped DdmInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: ddmd temporary directory leaked: /var/tmp/omicron_tmp/.tmpKy9vYY running 1 test running commands with args: ["db", "db-metadata", "ls-nexus"] running commands with args: ["--destructive", "db", "db-metadata", "force-mark-nexus-quiesced", "--skip-confirmation", "913233fe-92a8-4635-9572-183f495429c4"] running commands with args: ["db", "disks", "list"] running commands with args: ["db", "dns", "show"] running commands with args: ["db", "dns", "diff", "external", "2"] running commands with args: ["db", "dns", "names", "external", "2"] running commands with args: ["db", "instances"] running commands with args: ["db", "sleds"] running commands with args: ["db", "sleds", "-F", "discretionary"] running commands with args: ["mgs", "inventory"] running commands with args: ["nexus", "background-tasks", "doc"] running commands with args: ["nexus", "background-tasks", "show", "--no-executing-info"] running commands with args: ["nexus", "background-tasks", "show", "saga_recovery", "--no-executing-info"] running commands with args: ["nexus", "background-tasks", "show", "blueprint_loader", "blueprint_executor", "--no-executing-info"] running commands with args: ["nexus", "background-tasks", "show", "dns_internal", "--no-executing-info"] running commands with args: ["nexus", "background-tasks", "show", "dns_external", "--no-executing-info"] running commands with args: ["nexus", "background-tasks", "show", "all", "--no-executing-info"] running commands with args: ["nexus", "sagas", "list"] running commands with args: ["--destructive", "nexus", "sagas", "demo-create"] running commands with args: ["nexus", "sagas", "list"] running commands with args: ["--destructive", "nexus", "background-tasks", "activate", "inventory_collection"] running commands with args: ["nexus", "blueprints", "list"] running commands with args: ["nexus", "blueprints", "show", "fb35c14f-3134-42b3-a655-e8a0a885f17d"] running commands with args: ["nexus", "blueprints", "show", "current-target"] running commands with args: ["nexus", "blueprints", "diff", "fb35c14f-3134-42b3-a655-e8a0a885f17d", "current-target"] running commands with args: ["nexus", "blueprints", "diff", "fb35c14f-3134-42b3-a655-e8a0a885f17d"] running commands with args: ["nexus", "reconfigurator-config", "show", "current"] running commands with args: ["nexus", "update-status"] running commands with args: ["nexus", "update-status", "--details"] running commands with args: ["-w", "nexus", "reconfigurator-config", "set", "--planner-enabled", "true"] running commands with args: ["-w", "nexus", "reconfigurator-config", "set", "--disruption-policy", "migrate-or-terminate"] running commands with args: ["nexus", "reconfigurator-config", "show", "current"] running commands with args: ["reconfigurator", "export", "/var/tmp/omicron_tmp/.tmp0TX1Va/reconfigurator-export.out"] running commands with args: ["db", "whatis", "001de000-5110-4000-8000-000000000000", "001de000-05e4-4000-8000-000000004007"] running commands with args: ["--destructive", "db", "db-metadata", "force-mark-nexus-quiesced", "--skip-confirmation", "--skip-blueprint-validation", "913233fe-92a8-4635-9572-183f495429c4"] @@ -1013,11 +1013,11 @@ task: "switch_port_config_manager" configured period: every <REDACTED_DURATION>s last completed activation: <REDACTED ITERATIONS>, triggered by <TRIGGERED_BY_REDACTED> started at <REDACTED_TIMESTAMP> (<REDACTED DURATION>s ago) and ran for <REDACTED DURATION>ms -warning: unknown background task: "switch_port_config_manager" (don't know how to interpret details: Object {}) +warning: unknown background task: "switch_port_config_manager" (don't know how to interpret details: Object {"incomplete_bootstore_configs": Array []}) task: "trust_quorum_manager" configured period: every <REDACTED_DURATION>m last completed activation: <REDACTED ITERATIONS>, triggered by <TRIGGERED_BY_REDACTED> started at <REDACTED_TIMESTAMP> (<REDACTED DURATION>s ago) and ran for <REDACTED DURATION>ms @@ -1712,11 +1712,11 @@ task: "switch_port_config_manager" configured period: every <REDACTED_DURATION>s last completed activation: <REDACTED ITERATIONS>, triggered by <TRIGGERED_BY_REDACTED> started at <REDACTED_TIMESTAMP> (<REDACTED DURATION>s ago) and ran for <REDACTED DURATION>ms -warning: unknown background task: "switch_port_config_manager" (don't know how to interpret details: Object {}) +warning: unknown background task: "switch_port_config_manager" (don't know how to interpret details: Object {"incomplete_bootstore_configs": Array []}) task: "trust_quorum_manager" configured period: every <REDACTED_DURATION>m last completed activation: <REDACTED ITERATIONS>, triggered by <TRIGGERED_BY_REDACTED> started at <REDACTED_TIMESTAMP> (<REDACTED DURATION>s ago) and ran for <REDACTED DURATION>ms test test_omdb_success_cases ... FAILED failures: failures: test_omdb_success_cases test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 2 filtered out; finished in 16.93s log file: /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.424675.0.log note: configured to log to "/var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.424675.0.log" DB URL: postgresql://root@[::1]:43347/omicron?sslmode=disable DB address: [::1]:43347 log file: /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.424675.2.log note: configured to log to "/var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.424675.2.log" log file: /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.424675.3.log note: configured to log to "/var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.424675.3.log" thread 'test_omdb_success_cases' (424676) panicked at dev-tools/omdb/tests/test_all_output.rs:451:5: assertion failed: string doesn't match the contents of file: "tests/successes.out" see diffset above set EXPECTORATE=overwrite if these changes are intentional stack backtrace: 0: __rustc::rust_begin_unwind at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:689:5 1: core::panicking::panic_fmt at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:80:14 2: expectorate::assert_contents::<&str> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/expectorate-1.2.0/src/lib.rs:67:9 3: {async_block#0} at ./tests/test_all_output.rs:451:5 4: poll<&mut dyn core::future::future::Future<Output=()>> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/future/future.rs:133:9 5: poll<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/future/future.rs:133:9 6: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:778:70 7: with_budget<core::task::poll::Poll<()>, tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure#0}::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/task/coop/mod.rs:167:5 8: budget<core::task::poll::Poll<()>, tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure#0}::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/task/coop/mod.rs:133:5 9: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:778:25 10: <tokio::runtime::scheduler::current_thread::Context>::enter::<core::task::poll::Poll<()>, <tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}::{closure#0}> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:451:19 11: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:777:44 12: <tokio::runtime::scheduler::current_thread::CoreGuard>::enter::<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0} at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:865:68 13: <tokio::runtime::context::scoped::Scoped<tokio::runtime::scheduler::Context>>::set::<<tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>)> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context/scoped.rs:40:9 14: tokio::runtime::context::set_scheduler::<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}>::{closure#0} at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context.rs:181:38 15: try_with<tokio::runtime::context::Context, tokio::runtime::context::set_scheduler::{closure_env#0}<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>), tokio::runtime::scheduler::current_thread::{impl#9}::enter::{closure_env#0}<tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>, core::option::Option<()>>>, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>)> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/local.rs:513:12 16: <std::thread::local::LocalKey<tokio::runtime::context::Context>>::with::<tokio::runtime::context::set_scheduler<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}>::{closure#0}, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>)> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/local.rs:477:20 17: tokio::runtime::context::set_scheduler::<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context.rs:181:17 18: <tokio::runtime::scheduler::current_thread::CoreGuard>::enter::<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:865:27 19: <tokio::runtime::scheduler::current_thread::CoreGuard>::block_on::<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:765:24 20: {closure#0}<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:205:33 21: tokio::runtime::context::runtime::enter_runtime::<<tokio::runtime::scheduler::current_thread::CurrentThread>::block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>::{closure#0}, ()> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context/runtime.rs:65:16 22: block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:193:9 23: <tokio::runtime::runtime::Runtime>::block_on_inner::<core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/runtime.rs:371:52 24: block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/runtime.rs:345:18 25: test_omdb_success_cases at ./tests/test_all_output.rs:609:31 26: test_all_output::test_omdb_success_cases::{closure#0} at ./tests/test_all_output.rs:136:35 27: <test_all_output::test_omdb_success_cases::{closure#0} as core::ops::function::FnOnce<()>>::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 28: core::ops::function::FnOnce::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. WARN: dropped CockroachInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: temporary directory leaked: "/var/tmp/omicron_tmp/.tmpRaakCX" If you would like to access the database for debugging, run the following: # Run the database cargo xtask db-dev run --no-populate --store-dir "/var/tmp/omicron_tmp/.tmpRaakCX/data" # Access the database. Note the port may change if you run multiple databases. cockroach sql --host=localhost:32221 --insecure WARN: dropped ClickHouse process without cleaning it up first (there may still be a child process running (PID 424729) and a temporary directory leaked, /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.424675.1-clickhouse-Z3yXCV) WARN: dropped DendriteInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: dendrite temporary directory leaked: /var/tmp/omicron_tmp/.tmpSz1oDE WARN: dropped DendriteInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: dendrite temporary directory leaked: /var/tmp/omicron_tmp/.tmpzwDeDv WARN: dropped MgdInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: mgd temporary directory leaked: /var/tmp/omicron_tmp/.tmpb5lGB3 WARN: dropped MgdInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: mgd temporary directory leaked: /var/tmp/omicron_tmp/.tmpXv86Uw WARN: dropped DdmInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: ddmd temporary directory leaked: /var/tmp/omicron_tmp/.tmpxsJzVF WARN: dropped DdmInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: ddmd temporary directory leaked: /var/tmp/omicron_tmp/.tmpKy9vYY running 1 test running commands with args: ["db", "db-metadata", "ls-nexus"] running commands with args: ["--destructive", "db", "db-metadata", "force-mark-nexus-quiesced", "--skip-confirmation", "913233fe-92a8-4635-9572-183f495429c4"] running commands with args: ["db", "disks", "list"] running commands with args: ["db", "dns", "show"] running commands with args: ["db", "dns", "diff", "external", "2"] running commands with args: ["db", "dns", "names", "external", "2"] running commands with args: ["db", "instances"] running commands with args: ["db", "sleds"] running commands with args: ["db", "sleds", "-F", "discretionary"] running commands with args: ["mgs", "inventory"] running commands with args: ["nexus", "background-tasks", "doc"] running commands with args: ["nexus", "background-tasks", "show", "--no-executing-info"] running commands with args: ["nexus", "background-tasks", "show", "saga_recovery", "--no-executing-info"] running commands with args: ["nexus", "background-tasks", "show", "blueprint_loader", "blueprint_executor", "--no-executing-info"] running commands with args: ["nexus", "background-tasks", "show", "dns_internal", "--no-executing-info"] running commands with args: ["nexus", "background-tasks", "show", "dns_external", "--no-executing-info"] running commands with args: ["nexus", "background-tasks", "show", "all", "--no-executing-info"] running commands with args: ["nexus", "sagas", "list"] running commands with args: ["--destructive", "nexus", "sagas", "demo-create"] running commands with args: ["nexus", "sagas", "list"] running commands with args: ["--destructive", "nexus", "background-tasks", "activate", "inventory_collection"] running commands with args: ["nexus", "blueprints", "list"] running commands with args: ["nexus", "blueprints", "show", "7578cc6b-d135-46c1-8fe1-4d22afa30441"] running commands with args: ["nexus", "blueprints", "show", "current-target"] running commands with args: ["nexus", "blueprints", "diff", "7578cc6b-d135-46c1-8fe1-4d22afa30441", "current-target"] running commands with args: ["nexus", "blueprints", "diff", "7578cc6b-d135-46c1-8fe1-4d22afa30441"] running commands with args: ["nexus", "reconfigurator-config", "show", "current"] running commands with args: ["nexus", "update-status"] running commands with args: ["nexus", "update-status", "--details"] running commands with args: ["-w", "nexus", "reconfigurator-config", "set", "--planner-enabled", "true"] running commands with args: ["-w", "nexus", "reconfigurator-config", "set", "--disruption-policy", "migrate-or-terminate"] running commands with args: ["nexus", "reconfigurator-config", "show", "current"] running commands with args: ["reconfigurator", "export", "/var/tmp/omicron_tmp/.tmpVPwUdL/reconfigurator-export.out"] running commands with args: ["db", "whatis", "001de000-5110-4000-8000-000000000000", "001de000-05e4-4000-8000-000000004007"] running commands with args: ["--destructive", "db", "db-metadata", "force-mark-nexus-quiesced", "--skip-confirmation", "--skip-blueprint-validation", "913233fe-92a8-4635-9572-183f495429c4"] @@ -1013,11 +1013,11 @@ task: "switch_port_config_manager" configured period: every <REDACTED_DURATION>s last completed activation: <REDACTED ITERATIONS>, triggered by <TRIGGERED_BY_REDACTED> started at <REDACTED_TIMESTAMP> (<REDACTED DURATION>s ago) and ran for <REDACTED DURATION>ms -warning: unknown background task: "switch_port_config_manager" (don't know how to interpret details: Object {}) +warning: unknown background task: "switch_port_config_manager" (don't know how to interpret details: Object {"incomplete_bootstore_configs": Array []}) task: "trust_quorum_manager" configured period: every <REDACTED_DURATION>m last completed activation: <REDACTED ITERATIONS>, triggered by <TRIGGERED_BY_REDACTED> started at <REDACTED_TIMESTAMP> (<REDACTED DURATION>s ago) and ran for <REDACTED DURATION>ms @@ -1712,11 +1712,11 @@ task: "switch_port_config_manager" configured period: every <REDACTED_DURATION>s last completed activation: <REDACTED ITERATIONS>, triggered by <TRIGGERED_BY_REDACTED> started at <REDACTED_TIMESTAMP> (<REDACTED DURATION>s ago) and ran for <REDACTED DURATION>ms -warning: unknown background task: "switch_port_config_manager" (don't know how to interpret details: Object {}) +warning: unknown background task: "switch_port_config_manager" (don't know how to interpret details: Object {"incomplete_bootstore_configs": Array []}) task: "trust_quorum_manager" configured period: every <REDACTED_DURATION>m last completed activation: <REDACTED ITERATIONS>, triggered by <TRIGGERED_BY_REDACTED> started at <REDACTED_TIMESTAMP> (<REDACTED DURATION>s ago) and ran for <REDACTED DURATION>ms test test_omdb_success_cases ... FAILED failures: failures: test_omdb_success_cases test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 2 filtered out; finished in 19.64s log file: /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.397230.0.log note: configured to log to "/var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.397230.0.log" DB URL: postgresql://root@[::1]:39275/omicron?sslmode=disable DB address: [::1]:39275 log file: /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.397230.2.log note: configured to log to "/var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.397230.2.log" log file: /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.397230.3.log note: configured to log to "/var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.397230.3.log" thread 'test_omdb_success_cases' (397231) panicked at dev-tools/omdb/tests/test_all_output.rs:451:5: assertion failed: string doesn't match the contents of file: "tests/successes.out" see diffset above set EXPECTORATE=overwrite if these changes are intentional stack backtrace: 0: __rustc::rust_begin_unwind at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:689:5 1: core::panicking::panic_fmt at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:80:14 2: expectorate::assert_contents::<&str> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/expectorate-1.2.0/src/lib.rs:67:9 3: {async_block#0} at ./tests/test_all_output.rs:451:5 4: poll<&mut dyn core::future::future::Future<Output=()>> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/future/future.rs:133:9 5: poll<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/future/future.rs:133:9 6: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:778:70 7: with_budget<core::task::poll::Poll<()>, tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure#0}::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/task/coop/mod.rs:167:5 8: budget<core::task::poll::Poll<()>, tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure#0}::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/task/coop/mod.rs:133:5 9: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:778:25 10: <tokio::runtime::scheduler::current_thread::Context>::enter::<core::task::poll::Poll<()>, <tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}::{closure#0}> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:451:19 11: {closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:777:44 12: <tokio::runtime::scheduler::current_thread::CoreGuard>::enter::<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0} at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:865:68 13: <tokio::runtime::context::scoped::Scoped<tokio::runtime::scheduler::Context>>::set::<<tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>)> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context/scoped.rs:40:9 14: tokio::runtime::context::set_scheduler::<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}>::{closure#0} at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context.rs:181:38 15: try_with<tokio::runtime::context::Context, tokio::runtime::context::set_scheduler::{closure_env#0}<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>), tokio::runtime::scheduler::current_thread::{impl#9}::enter::{closure_env#0}<tokio::runtime::scheduler::current_thread::{impl#9}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>, core::option::Option<()>>>, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>)> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/local.rs:513:12 16: <std::thread::local::LocalKey<tokio::runtime::context::Context>>::with::<tokio::runtime::context::set_scheduler<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}>::{closure#0}, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>)> at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/local.rs:477:20 17: tokio::runtime::context::set_scheduler::<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core>, core::option::Option<()>), <tokio::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>>::{closure#0}> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context.rs:181:17 18: <tokio::runtime::scheduler::current_thread::CoreGuard>::enter::<<tokio::runtime::scheduler::current_thread::CoreGuard>::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>>::{closure#0}, core::option::Option<()>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:865:27 19: <tokio::runtime::scheduler::current_thread::CoreGuard>::block_on::<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:765:24 20: {closure#0}<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:205:33 21: tokio::runtime::context::runtime::enter_runtime::<<tokio::runtime::scheduler::current_thread::CurrentThread>::block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>>::{closure#0}, ()> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context/runtime.rs:65:16 22: block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/current_thread/mod.rs:193:9 23: <tokio::runtime::runtime::Runtime>::block_on_inner::<core::pin::Pin<&mut dyn core::future::future::Future<Output = ()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/runtime.rs:371:52 24: block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/runtime.rs:345:18 25: test_omdb_success_cases at ./tests/test_all_output.rs:609:31 26: test_all_output::test_omdb_success_cases::{closure#0} at ./tests/test_all_output.rs:136:35 27: <test_all_output::test_omdb_success_cases::{closure#0} as core::ops::function::FnOnce<()>>::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 28: core::ops::function::FnOnce::call_once at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. WARN: dropped CockroachInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: temporary directory leaked: "/var/tmp/omicron_tmp/.tmp7HXJfb" If you would like to access the database for debugging, run the following: # Run the database cargo xtask db-dev run --no-populate --store-dir "/var/tmp/omicron_tmp/.tmp7HXJfb/data" # Access the database. Note the port may change if you run multiple databases. cockroach sql --host=localhost:32221 --insecure WARN: dropped ClickHouse process without cleaning it up first (there may still be a child process running (PID 397662) and a temporary directory leaked, /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.397230.1-clickhouse-Q4iMVM) WARN: dropped DendriteInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: dendrite temporary directory leaked: /var/tmp/omicron_tmp/.tmpc16QbK WARN: dropped DendriteInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: dendrite temporary directory leaked: /var/tmp/omicron_tmp/.tmpwpN53q WARN: dropped MgdInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: mgd temporary directory leaked: /var/tmp/omicron_tmp/.tmpLLeThm WARN: dropped MgdInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: mgd temporary directory leaked: /var/tmp/omicron_tmp/.tmpfoTmPx WARN: dropped DdmInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: ddmd temporary directory leaked: /var/tmp/omicron_tmp/.tmpKvShAk WARN: dropped DdmInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked) WARN: ddmd temporary directory leaked: /var/tmp/omicron_tmp/.tmp7lhIwh