Max level shown:
12026-03-14T00:54:23.904ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::c]:63193
22026-03-14T00:54:23.904ZINFOcrucible-agent (datafile): region d1cc0eea-1145-4627-8a64-65fba13fbaf5 state: Requested
32026-03-14T00:54:23.905ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-03-14T00:54:23.905ZINFOcrucible-agent (dropshot): request completed latency_us = 1409 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63193 req_id = 653c1adb-e783-4ccf-9058-7a6343c218df response_code = 200 uri = /crucible/0/regions
52026-03-14T00:54:23.915ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_1eee919a-3b9e-425a-946f-52dd27d1736d/crucible/regions/d1cc0eea-1145-4627-8a64-65fba13fbaf5
62026-03-14T00:54:23.915ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_1eee919a-3b9e-425a-946f-52dd27d1736d/crucible/regions/d1cc0eea-1145-4627-8a64-65fba13fbaf5
72026-03-14T00:54:23.965ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d1cc0eea-1145-4627-8a64-65fba13fbaf5"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d1cc0eea-1145-4627-8a64-65fba13fbaf5" region = d1cc0eea-1145-4627-8a64-65fba13fbaf5
82026-03-14T00:54:23.965ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d1cc0eea-1145-4627-8a64-65fba13fbaf5" "--data" "/data/regions/d1cc0eea-1145-4627-8a64-65fba13fbaf5" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = d1cc0eea-1145-4627-8a64-65fba13fbaf5
92026-03-14T00:54:24.042ZINFOcrucible-agent (worker): region files created ok region = d1cc0eea-1145-4627-8a64-65fba13fbaf5
102026-03-14T00:54:24.042ZINFOcrucible-agent (datafile): region d1cc0eea-1145-4627-8a64-65fba13fbaf5 state: Requested -> Created
112026-03-14T00:54:24.043ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-03-14T00:54:24.043ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d1cc0eea-1145-4627-8a64-65fba13fbaf5
132026-03-14T00:54:24.054ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d1cc0eea-1145-4627-8a64-65fba13fbaf5
142026-03-14T00:54:24.076ZINFOcrucible-agent (worker): creating config property group
152026-03-14T00:54:24.079ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d1cc0eea-1145-4627-8a64-65fba13fbaf5
162026-03-14T00:54:24.079ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d1cc0eea-1145-4627-8a64-65fba13fbaf5
172026-03-14T00:54:24.079ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
182026-03-14T00:54:24.080ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
192026-03-14T00:54:24.080ZINFOcrucible-agent (worker): commit
202026-03-14T00:54:24.083ZINFOcrucible-agent (worker): ok!
212026-03-14T00:54:24.088ZINFOcrucible-agent (worker): SMF ok!
222026-03-14T00:54:24.196ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63193 req_id = 9db8a5bd-ac7b-4969-bb34-87567848fcd0 response_code = 200 uri = /crucible/0/regions
232026-03-14T00:54:30.865ZINFOcrucible-agent (datafile): region aa6f5a84-27c7-4c3b-ba73-ac449a72efb8 state: Requested
242026-03-14T00:54:30.865ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
252026-03-14T00:54:30.865ZINFOcrucible-agent (dropshot): request completed latency_us = 245 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63193 req_id = 3ce5b910-7761-4f19-ad12-534b874be45b response_code = 200 uri = /crucible/0/regions
262026-03-14T00:54:30.871ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_1eee919a-3b9e-425a-946f-52dd27d1736d/crucible/regions/aa6f5a84-27c7-4c3b-ba73-ac449a72efb8
272026-03-14T00:54:30.871ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_1eee919a-3b9e-425a-946f-52dd27d1736d/crucible/regions/aa6f5a84-27c7-4c3b-ba73-ac449a72efb8
282026-03-14T00:54:30.922ZINFOcrucible-agent (worker): creating region Region { id: RegionId("aa6f5a84-27c7-4c3b-ba73-ac449a72efb8"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/aa6f5a84-27c7-4c3b-ba73-ac449a72efb8" region = aa6f5a84-27c7-4c3b-ba73-ac449a72efb8
292026-03-14T00:54:30.922ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "aa6f5a84-27c7-4c3b-ba73-ac449a72efb8" "--data" "/data/regions/aa6f5a84-27c7-4c3b-ba73-ac449a72efb8" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = aa6f5a84-27c7-4c3b-ba73-ac449a72efb8
302026-03-14T00:54:31.001ZINFOcrucible-agent (worker): region files created ok region = aa6f5a84-27c7-4c3b-ba73-ac449a72efb8
312026-03-14T00:54:31.001ZINFOcrucible-agent (datafile): region aa6f5a84-27c7-4c3b-ba73-ac449a72efb8 state: Requested -> Created
322026-03-14T00:54:31.001ZINFOcrucible-agent (worker): applying SMF actions post create...
332026-03-14T00:54:31.002ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-aa6f5a84-27c7-4c3b-ba73-ac449a72efb8
342026-03-14T00:54:31.010ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-aa6f5a84-27c7-4c3b-ba73-ac449a72efb8
352026-03-14T00:54:31.017ZINFOcrucible-agent (worker): creating config property group
362026-03-14T00:54:31.020ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-aa6f5a84-27c7-4c3b-ba73-ac449a72efb8
372026-03-14T00:54:31.020ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/aa6f5a84-27c7-4c3b-ba73-ac449a72efb8
382026-03-14T00:54:31.020ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
392026-03-14T00:54:31.020ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
402026-03-14T00:54:31.020ZINFOcrucible-agent (worker): commit
412026-03-14T00:54:31.024ZINFOcrucible-agent (worker): ok!
422026-03-14T00:54:31.033ZINFOcrucible-agent (worker): SMF ok!
432026-03-14T00:54:31.163ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63193 req_id = 5b1d4edc-c7bb-4f28-9b66-ee9b58d9357e response_code = 200 uri = /crucible/0/regions
442026-03-14T00:54:57.042ZINFOcrucible-agent (dropshot): request completed latency_us = 56 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:63193 req_id = 04807f31-5d07-4451-bb23-503c8b79a667 response_code = 200 uri = /crucible/0/regions/aa6f5a84-27c7-4c3b-ba73-ac449a72efb8
452026-03-14T00:54:57.045ZINFOcrucible-agent (datafile): path is "/data/regions/aa6f5a84-27c7-4c3b-ba73-ac449a72efb8"
462026-03-14T00:54:57.057ZINFOcrucible-agent (datafile): dataset is oxp_1eee919a-3b9e-425a-946f-52dd27d1736d/crucible/regions/aa6f5a84-27c7-4c3b-ba73-ac449a72efb8
472026-03-14T00:54:57.064ZINFOcrucible-agent (datafile): region aa6f5a84-27c7-4c3b-ba73-ac449a72efb8 state: Created -> Tombstoned
482026-03-14T00:54:57.065ZINFOcrucible-agent (worker): applying SMF actions before removal...
492026-03-14T00:54:57.065ZINFOcrucible-agent (dropshot): request completed latency_us = 19726 local_addr = [fd00:1122:3344:101::14]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:63193 req_id = fa331d45-231b-4f3e-99f6-e07fa8790985 response_code = 204 uri = /crucible/0/regions/aa6f5a84-27c7-4c3b-ba73-ac449a72efb8
502026-03-14T00:54:57.066ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-aa6f5a84-27c7-4c3b-ba73-ac449a72efb8 (instance states: (Some(Online), None))
512026-03-14T00:54:57.068ZINFOcrucible-agent (dropshot): request completed latency_us = 33 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:63193 req_id = f8f229b0-23cf-4d1c-940c-36930ff6cde5 response_code = 200 uri = /crucible/0/regions/aa6f5a84-27c7-4c3b-ba73-ac449a72efb8
522026-03-14T00:54:57.074ZINFOcrucible-agent (worker): SMF ok!
532026-03-14T00:54:57.083ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_1eee919a-3b9e-425a-946f-52dd27d1736d/crucible/regions/aa6f5a84-27c7-4c3b-ba73-ac449a72efb8" region = aa6f5a84-27c7-4c3b-ba73-ac449a72efb8
542026-03-14T00:54:57.140ZINFOcrucible-agent (datafile): region aa6f5a84-27c7-4c3b-ba73-ac449a72efb8 state: Tombstoned -> Destroyed
552026-03-14T00:54:57.190ZINFOcrucible-agent (dropshot): request completed latency_us = 52 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:63193 req_id = ba9dd45d-5a33-4083-8128-229a3c370f6b response_code = 200 uri = /crucible/0/regions/aa6f5a84-27c7-4c3b-ba73-ac449a72efb8