Max level shown:
12026-05-11T21:00:16.222ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::c]:63494
22026-05-11T21:00:16.222ZINFOcrucible-agent (datafile): region 8c7e86ac-9d1d-4827-9202-ef5598e3bdce state: Requested
32026-05-11T21:00:16.223ZINFOcrucible-agent (dropshot): request completed latency_us = 770 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63494 req_id = 17c5eaa1-fbe2-46a2-b75c-a2cc4adb8d07 response_code = 200 uri = /crucible/0/regions
42026-05-11T21:00:16.223ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-05-11T21:00:16.232ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_d61e2d73-0b4c-4c1f-9eb8-242200796615/crucible/regions/8c7e86ac-9d1d-4827-9202-ef5598e3bdce
62026-05-11T21:00:16.232ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_d61e2d73-0b4c-4c1f-9eb8-242200796615/crucible/regions/8c7e86ac-9d1d-4827-9202-ef5598e3bdce
72026-05-11T21:00:16.287ZINFOcrucible-agent (worker): creating region Region { id: RegionId("8c7e86ac-9d1d-4827-9202-ef5598e3bdce"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/8c7e86ac-9d1d-4827-9202-ef5598e3bdce" region = 8c7e86ac-9d1d-4827-9202-ef5598e3bdce
82026-05-11T21:00:16.287ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "8c7e86ac-9d1d-4827-9202-ef5598e3bdce" "--data" "/data/regions/8c7e86ac-9d1d-4827-9202-ef5598e3bdce" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 8c7e86ac-9d1d-4827-9202-ef5598e3bdce
92026-05-11T21:00:16.397ZINFOcrucible-agent (worker): region files created ok region = 8c7e86ac-9d1d-4827-9202-ef5598e3bdce
102026-05-11T21:00:16.397ZINFOcrucible-agent (datafile): region 8c7e86ac-9d1d-4827-9202-ef5598e3bdce state: Requested -> Created
112026-05-11T21:00:16.397ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-05-11T21:00:16.397ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-8c7e86ac-9d1d-4827-9202-ef5598e3bdce
132026-05-11T21:00:16.406ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-8c7e86ac-9d1d-4827-9202-ef5598e3bdce
142026-05-11T21:00:16.428ZINFOcrucible-agent (worker): creating config property group
152026-05-11T21:00:16.432ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-8c7e86ac-9d1d-4827-9202-ef5598e3bdce
162026-05-11T21:00:16.432ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8c7e86ac-9d1d-4827-9202-ef5598e3bdce
172026-05-11T21:00:16.432ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-05-11T21:00:16.432ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
192026-05-11T21:00:16.432ZINFOcrucible-agent (worker): commit
202026-05-11T21:00:16.436ZINFOcrucible-agent (worker): ok!
212026-05-11T21:00:16.445ZINFOcrucible-agent (dropshot): request completed latency_us = 53 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63494 req_id = b6910e2a-c2b2-4604-9c92-aefdd0aaacd3 response_code = 200 uri = /crucible/0/regions
222026-05-11T21:00:16.454ZINFOcrucible-agent (worker): SMF ok!
232026-05-11T21:00:18.248ZINFOcrucible-agent (dropshot): request completed latency_us = 76 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:63494 req_id = a9f8d44d-bd41-421a-8c1a-fb8ef78237b5 response_code = 200 uri = /crucible/0/regions/b497c08b-4949-4940-8586-31a0d512f8fa
242026-05-11T21:00:18.251ZINFOcrucible-agent (datafile): path is "/data/regions/b497c08b-4949-4940-8586-31a0d512f8fa"
252026-05-11T21:00:18.257ZINFOcrucible-agent (datafile): dataset is oxp_d61e2d73-0b4c-4c1f-9eb8-242200796615/crucible/regions/b497c08b-4949-4940-8586-31a0d512f8fa
262026-05-11T21:00:18.264ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d61e2d73-0b4c-4c1f-9eb8-242200796615/crucible/regions/b497c08b-4949-4940-8586-31a0d512f8fa@87c100df-da85-4f64-a3bd-043597571c99
272026-05-11T21:00:18.264ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d61e2d73-0b4c-4c1f-9eb8-242200796615/crucible/regions/b497c08b-4949-4940-8586-31a0d512f8fa", "87c100df-da85-4f64-a3bd-043597571c99"]
282026-05-11T21:00:18.272ZINFOcrucible-agent (dropshot): request completed latency_us = 21165 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:63494 req_id = c599f737-1ce1-47aa-a0b0-8caca37cf07e response_code = 200 uri = /crucible/0/regions/b497c08b-4949-4940-8586-31a0d512f8fa/snapshots/87c100df-da85-4f64-a3bd-043597571c99
292026-05-11T21:00:18.275ZINFOcrucible-agent (datafile): path is "/data/regions/b497c08b-4949-4940-8586-31a0d512f8fa"
302026-05-11T21:00:18.281ZINFOcrucible-agent (datafile): dataset is oxp_d61e2d73-0b4c-4c1f-9eb8-242200796615/crucible/regions/b497c08b-4949-4940-8586-31a0d512f8fa
312026-05-11T21:00:18.287ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d61e2d73-0b4c-4c1f-9eb8-242200796615/crucible/regions/b497c08b-4949-4940-8586-31a0d512f8fa@87c100df-da85-4f64-a3bd-043597571c99
322026-05-11T21:00:18.287ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d61e2d73-0b4c-4c1f-9eb8-242200796615/crucible/regions/b497c08b-4949-4940-8586-31a0d512f8fa", "87c100df-da85-4f64-a3bd-043597571c99"]
332026-05-11T21:00:18.296ZINFOcrucible-agent (datafile): requesting running snapshot b497c08b-4949-4940-8586-31a0d512f8fa-87c100df-da85-4f64-a3bd-043597571c99 state: Requested
342026-05-11T21:00:18.296ZINFOcrucible-agent (dropshot): request completed latency_us = 20788 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63494 req_id = 00aa6ab1-d523-42c9-a066-45ae9325cf15 response_code = 200 uri = /crucible/0/regions/b497c08b-4949-4940-8586-31a0d512f8fa/snapshots/87c100df-da85-4f64-a3bd-043597571c99/run
352026-05-11T21:00:18.296ZINFOcrucible-agent (worker): applying SMF actions for region b497c08b-4949-4940-8586-31a0d512f8fa running snapshot 87c100df-da85-4f64-a3bd-043597571c99 (state Requested)...
362026-05-11T21:00:18.300ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-b497c08b-4949-4940-8586-31a0d512f8fa-87c100df-da85-4f64-a3bd-043597571c99
372026-05-11T21:00:18.306ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-b497c08b-4949-4940-8586-31a0d512f8fa-87c100df-da85-4f64-a3bd-043597571c99
382026-05-11T21:00:18.317ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-b497c08b-4949-4940-8586-31a0d512f8fa-87c100df-da85-4f64-a3bd-043597571c99
392026-05-11T21:00:18.317ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b497c08b-4949-4940-8586-31a0d512f8fa/.zfs/snapshot/87c100df-da85-4f64-a3bd-043597571c99
402026-05-11T21:00:18.317ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-05-11T21:00:18.317ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-05-11T21:00:18.317ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
432026-05-11T21:00:18.317ZINFOcrucible-agent (worker): commit
442026-05-11T21:00:18.321ZINFOcrucible-agent (worker): ok!
452026-05-11T21:00:18.325ZINFOcrucible-agent (worker): SMF ok!
462026-05-11T21:00:18.325ZINFOcrucible-agent (datafile): region b497c08b-4949-4940-8586-31a0d512f8fa running snapshot 87c100df-da85-4f64-a3bd-043597571c99 state: Requested -> Created
472026-05-11T21:00:18.653ZINFOcrucible-agent (datafile): path is "/data/regions/b497c08b-4949-4940-8586-31a0d512f8fa"
482026-05-11T21:00:18.660ZINFOcrucible-agent (datafile): dataset is oxp_d61e2d73-0b4c-4c1f-9eb8-242200796615/crucible/regions/b497c08b-4949-4940-8586-31a0d512f8fa
492026-05-11T21:00:18.666ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d61e2d73-0b4c-4c1f-9eb8-242200796615/crucible/regions/b497c08b-4949-4940-8586-31a0d512f8fa@87c100df-da85-4f64-a3bd-043597571c99
502026-05-11T21:00:18.666ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d61e2d73-0b4c-4c1f-9eb8-242200796615/crucible/regions/b497c08b-4949-4940-8586-31a0d512f8fa", "87c100df-da85-4f64-a3bd-043597571c99"]
512026-05-11T21:00:18.673ZINFOcrucible-agent (dropshot): request completed latency_us = 20076 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63494 req_id = d4fceb48-39cf-441f-9950-b97b8b0514f1 response_code = 200 uri = /crucible/0/regions/b497c08b-4949-4940-8586-31a0d512f8fa/snapshots/87c100df-da85-4f64-a3bd-043597571c99/run