Max level shown:
12026-05-06T00:36:05.957ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:51946
22026-05-06T00:36:05.957ZINFOcrucible-agent (datafile): region 84835a19-0f07-44b7-bde2-cbb0d96f1ed4 state: Requested
32026-05-06T00:36:05.958ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-05-06T00:36:05.958ZINFOcrucible-agent (dropshot): request completed latency_us = 1190 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:51946 req_id = 7d1b334d-207a-413b-8ca9-db5a504e0659 response_code = 200 uri = /crucible/0/regions
52026-05-06T00:36:05.968ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_24cb58f8-57b3-4998-9110-b31c5f0c795b/crucible/regions/84835a19-0f07-44b7-bde2-cbb0d96f1ed4
62026-05-06T00:36:05.968ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_24cb58f8-57b3-4998-9110-b31c5f0c795b/crucible/regions/84835a19-0f07-44b7-bde2-cbb0d96f1ed4
72026-05-06T00:36:06.031ZINFOcrucible-agent (worker): creating region Region { id: RegionId("84835a19-0f07-44b7-bde2-cbb0d96f1ed4"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/84835a19-0f07-44b7-bde2-cbb0d96f1ed4" region = 84835a19-0f07-44b7-bde2-cbb0d96f1ed4
82026-05-06T00:36:06.031ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "84835a19-0f07-44b7-bde2-cbb0d96f1ed4" "--data" "/data/regions/84835a19-0f07-44b7-bde2-cbb0d96f1ed4" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 84835a19-0f07-44b7-bde2-cbb0d96f1ed4
92026-05-06T00:36:06.118ZINFOcrucible-agent (worker): region files created ok region = 84835a19-0f07-44b7-bde2-cbb0d96f1ed4
102026-05-06T00:36:06.118ZINFOcrucible-agent (datafile): region 84835a19-0f07-44b7-bde2-cbb0d96f1ed4 state: Requested -> Created
112026-05-06T00:36:06.118ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-05-06T00:36:06.143ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-84835a19-0f07-44b7-bde2-cbb0d96f1ed4
132026-05-06T00:36:06.149ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-84835a19-0f07-44b7-bde2-cbb0d96f1ed4
142026-05-06T00:36:06.160ZINFOcrucible-agent (worker): creating config property group
152026-05-06T00:36:06.163ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-84835a19-0f07-44b7-bde2-cbb0d96f1ed4
162026-05-06T00:36:06.164ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/84835a19-0f07-44b7-bde2-cbb0d96f1ed4
172026-05-06T00:36:06.164ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-05-06T00:36:06.164ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
192026-05-06T00:36:06.164ZINFOcrucible-agent (worker): commit
202026-05-06T00:36:06.167ZINFOcrucible-agent (worker): ok!
212026-05-06T00:36:06.172ZINFOcrucible-agent (worker): SMF ok!
222026-05-06T00:36:06.180ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:51946 req_id = eaef6ed7-2acd-4cd7-bea8-27045eac7083 response_code = 200 uri = /crucible/0/regions
232026-05-06T00:36:06.605ZINFOcrucible-agent (dropshot): request completed latency_us = 76 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:51946 req_id = b3e71378-4e07-4aa0-83be-10a72217c6f5 response_code = 200 uri = /crucible/0/regions/322c850f-903b-4f35-bcd6-8a1013463262
242026-05-06T00:36:06.611ZINFOcrucible-agent (datafile): path is "/data/regions/322c850f-903b-4f35-bcd6-8a1013463262"
252026-05-06T00:36:06.618ZINFOcrucible-agent (datafile): dataset is oxp_24cb58f8-57b3-4998-9110-b31c5f0c795b/crucible/regions/322c850f-903b-4f35-bcd6-8a1013463262
262026-05-06T00:36:06.624ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_24cb58f8-57b3-4998-9110-b31c5f0c795b/crucible/regions/322c850f-903b-4f35-bcd6-8a1013463262@dd9d27d6-2533-4c8d-a630-eb806b33667d
272026-05-06T00:36:06.624ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_24cb58f8-57b3-4998-9110-b31c5f0c795b/crucible/regions/322c850f-903b-4f35-bcd6-8a1013463262", "dd9d27d6-2533-4c8d-a630-eb806b33667d"]
282026-05-06T00:36:06.633ZINFOcrucible-agent (dropshot): request completed latency_us = 21712 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:51946 req_id = 11048da1-bbe9-4895-9c6e-acdb3e8f6010 response_code = 200 uri = /crucible/0/regions/322c850f-903b-4f35-bcd6-8a1013463262/snapshots/dd9d27d6-2533-4c8d-a630-eb806b33667d
292026-05-06T00:36:06.636ZINFOcrucible-agent (datafile): path is "/data/regions/322c850f-903b-4f35-bcd6-8a1013463262"
302026-05-06T00:36:06.642ZINFOcrucible-agent (datafile): dataset is oxp_24cb58f8-57b3-4998-9110-b31c5f0c795b/crucible/regions/322c850f-903b-4f35-bcd6-8a1013463262
312026-05-06T00:36:06.648ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_24cb58f8-57b3-4998-9110-b31c5f0c795b/crucible/regions/322c850f-903b-4f35-bcd6-8a1013463262@dd9d27d6-2533-4c8d-a630-eb806b33667d
322026-05-06T00:36:06.648ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_24cb58f8-57b3-4998-9110-b31c5f0c795b/crucible/regions/322c850f-903b-4f35-bcd6-8a1013463262", "dd9d27d6-2533-4c8d-a630-eb806b33667d"]
332026-05-06T00:36:06.656ZINFOcrucible-agent (datafile): requesting running snapshot 322c850f-903b-4f35-bcd6-8a1013463262-dd9d27d6-2533-4c8d-a630-eb806b33667d state: Requested
342026-05-06T00:36:06.656ZINFOcrucible-agent (worker): applying SMF actions for region 322c850f-903b-4f35-bcd6-8a1013463262 running snapshot dd9d27d6-2533-4c8d-a630-eb806b33667d (state Requested)...
352026-05-06T00:36:06.656ZINFOcrucible-agent (dropshot): request completed latency_us = 20003 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:51946 req_id = f965eded-baef-477b-a126-519eeeb7391d response_code = 200 uri = /crucible/0/regions/322c850f-903b-4f35-bcd6-8a1013463262/snapshots/dd9d27d6-2533-4c8d-a630-eb806b33667d/run
362026-05-06T00:36:06.661ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-322c850f-903b-4f35-bcd6-8a1013463262-dd9d27d6-2533-4c8d-a630-eb806b33667d
372026-05-06T00:36:06.668ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-322c850f-903b-4f35-bcd6-8a1013463262-dd9d27d6-2533-4c8d-a630-eb806b33667d
382026-05-06T00:36:06.678ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-322c850f-903b-4f35-bcd6-8a1013463262-dd9d27d6-2533-4c8d-a630-eb806b33667d
392026-05-06T00:36:06.678ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/322c850f-903b-4f35-bcd6-8a1013463262/.zfs/snapshot/dd9d27d6-2533-4c8d-a630-eb806b33667d
402026-05-06T00:36:06.678ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-05-06T00:36:06.678ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-05-06T00:36:06.678ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
432026-05-06T00:36:06.679ZINFOcrucible-agent (worker): commit
442026-05-06T00:36:06.682ZINFOcrucible-agent (worker): ok!
452026-05-06T00:36:06.686ZINFOcrucible-agent (worker): SMF ok!
462026-05-06T00:36:06.686ZINFOcrucible-agent (datafile): region 322c850f-903b-4f35-bcd6-8a1013463262 running snapshot dd9d27d6-2533-4c8d-a630-eb806b33667d state: Requested -> Created
472026-05-06T00:36:06.905ZINFOcrucible-agent (datafile): path is "/data/regions/322c850f-903b-4f35-bcd6-8a1013463262"
482026-05-06T00:36:06.910ZINFOcrucible-agent (datafile): dataset is oxp_24cb58f8-57b3-4998-9110-b31c5f0c795b/crucible/regions/322c850f-903b-4f35-bcd6-8a1013463262
492026-05-06T00:36:06.916ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_24cb58f8-57b3-4998-9110-b31c5f0c795b/crucible/regions/322c850f-903b-4f35-bcd6-8a1013463262@dd9d27d6-2533-4c8d-a630-eb806b33667d
502026-05-06T00:36:06.916ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_24cb58f8-57b3-4998-9110-b31c5f0c795b/crucible/regions/322c850f-903b-4f35-bcd6-8a1013463262", "dd9d27d6-2533-4c8d-a630-eb806b33667d"]
512026-05-06T00:36:06.922ZINFOcrucible-agent (dropshot): request completed latency_us = 17516 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:51946 req_id = e6cef47a-2a6a-40ff-a565-8ade8d02733a response_code = 200 uri = /crucible/0/regions/322c850f-903b-4f35-bcd6-8a1013463262/snapshots/dd9d27d6-2533-4c8d-a630-eb806b33667d/run