Max level shown:
12026-05-22T22:07:43.202ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::b]:43412
22026-05-22T22:07:43.202ZINFOcrucible-agent (datafile): region c05ba3d2-973e-41e5-9a15-f61cea476e10 state: Requested
32026-05-22T22:07:43.203ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-05-22T22:07:43.203ZINFOcrucible-agent (dropshot): request completed latency_us = 718 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43412 req_id = 681bc923-cefe-4b3b-be09-b7d0ea2e37fc response_code = 200 uri = /crucible/0/regions
52026-05-22T22:07:43.211ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_d208cecb-d446-4037-8ac2-3b49d6cccdc3/crucible/regions/c05ba3d2-973e-41e5-9a15-f61cea476e10
62026-05-22T22:07:43.211ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_d208cecb-d446-4037-8ac2-3b49d6cccdc3/crucible/regions/c05ba3d2-973e-41e5-9a15-f61cea476e10
72026-05-22T22:07:43.258ZINFOcrucible-agent (worker): creating region Region { id: RegionId("c05ba3d2-973e-41e5-9a15-f61cea476e10"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/c05ba3d2-973e-41e5-9a15-f61cea476e10" region = c05ba3d2-973e-41e5-9a15-f61cea476e10
82026-05-22T22:07:43.258ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "c05ba3d2-973e-41e5-9a15-f61cea476e10" "--data" "/data/regions/c05ba3d2-973e-41e5-9a15-f61cea476e10" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = c05ba3d2-973e-41e5-9a15-f61cea476e10
92026-05-22T22:07:43.314ZINFOcrucible-agent (worker): region files created ok region = c05ba3d2-973e-41e5-9a15-f61cea476e10
102026-05-22T22:07:43.314ZINFOcrucible-agent (datafile): region c05ba3d2-973e-41e5-9a15-f61cea476e10 state: Requested -> Created
112026-05-22T22:07:43.314ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-05-22T22:07:43.329ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-c05ba3d2-973e-41e5-9a15-f61cea476e10
132026-05-22T22:07:43.332ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-c05ba3d2-973e-41e5-9a15-f61cea476e10
142026-05-22T22:07:43.338ZINFOcrucible-agent (worker): creating config property group
152026-05-22T22:07:43.340ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-c05ba3d2-973e-41e5-9a15-f61cea476e10
162026-05-22T22:07:43.340ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/c05ba3d2-973e-41e5-9a15-f61cea476e10
172026-05-22T22:07:43.340ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-05-22T22:07:43.340ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
192026-05-22T22:07:43.340ZINFOcrucible-agent (worker): commit
202026-05-22T22:07:43.342ZINFOcrucible-agent (worker): ok!
212026-05-22T22:07:43.344ZINFOcrucible-agent (worker): SMF ok!
222026-05-22T22:07:43.416ZINFOcrucible-agent (dropshot): request completed latency_us = 58 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43412 req_id = 5a3726ec-a946-4ddf-80af-a3db8fb2f244 response_code = 200 uri = /crucible/0/regions
232026-05-22T22:07:44.327ZINFOcrucible-agent (dropshot): request completed latency_us = 67 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:43412 req_id = b0de0da5-8f52-494d-bf56-46bfc69b4617 response_code = 200 uri = /crucible/0/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b
242026-05-22T22:07:44.331ZINFOcrucible-agent (datafile): path is "/data/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b"
252026-05-22T22:07:44.337ZINFOcrucible-agent (datafile): dataset is oxp_d208cecb-d446-4037-8ac2-3b49d6cccdc3/crucible/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b
262026-05-22T22:07:44.344ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d208cecb-d446-4037-8ac2-3b49d6cccdc3/crucible/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b@2454829e-c135-4d72-8911-45ad9cc85f35
272026-05-22T22:07:44.344ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d208cecb-d446-4037-8ac2-3b49d6cccdc3/crucible/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b", "2454829e-c135-4d72-8911-45ad9cc85f35"]
282026-05-22T22:07:44.351ZINFOcrucible-agent (dropshot): request completed latency_us = 20519 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:43412 req_id = 508e91b6-a60e-45c9-8f04-7bb258dcbff3 response_code = 200 uri = /crucible/0/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b/snapshots/2454829e-c135-4d72-8911-45ad9cc85f35
292026-05-22T22:07:44.355ZINFOcrucible-agent (datafile): path is "/data/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b"
302026-05-22T22:07:44.360ZINFOcrucible-agent (datafile): dataset is oxp_d208cecb-d446-4037-8ac2-3b49d6cccdc3/crucible/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b
312026-05-22T22:07:44.366ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d208cecb-d446-4037-8ac2-3b49d6cccdc3/crucible/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b@2454829e-c135-4d72-8911-45ad9cc85f35
322026-05-22T22:07:44.366ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d208cecb-d446-4037-8ac2-3b49d6cccdc3/crucible/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b", "2454829e-c135-4d72-8911-45ad9cc85f35"]
332026-05-22T22:07:44.373ZINFOcrucible-agent (datafile): requesting running snapshot 1c5f2a5a-87e6-4868-9876-69557fb3b00b-2454829e-c135-4d72-8911-45ad9cc85f35 state: Requested
342026-05-22T22:07:44.373ZINFOcrucible-agent (worker): applying SMF actions for region 1c5f2a5a-87e6-4868-9876-69557fb3b00b running snapshot 2454829e-c135-4d72-8911-45ad9cc85f35 (state Requested)...
352026-05-22T22:07:44.373ZINFOcrucible-agent (dropshot): request completed latency_us = 18204 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43412 req_id = 50189e36-abf2-4867-8c60-fec4c243160e response_code = 200 uri = /crucible/0/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b/snapshots/2454829e-c135-4d72-8911-45ad9cc85f35/run
362026-05-22T22:07:44.378ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-1c5f2a5a-87e6-4868-9876-69557fb3b00b-2454829e-c135-4d72-8911-45ad9cc85f35
372026-05-22T22:07:44.387ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-1c5f2a5a-87e6-4868-9876-69557fb3b00b-2454829e-c135-4d72-8911-45ad9cc85f35
382026-05-22T22:07:44.392ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-1c5f2a5a-87e6-4868-9876-69557fb3b00b-2454829e-c135-4d72-8911-45ad9cc85f35
392026-05-22T22:07:44.393ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b/.zfs/snapshot/2454829e-c135-4d72-8911-45ad9cc85f35
402026-05-22T22:07:44.393ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-05-22T22:07:44.393ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-05-22T22:07:44.393ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
432026-05-22T22:07:44.393ZINFOcrucible-agent (worker): commit
442026-05-22T22:07:44.395ZINFOcrucible-agent (worker): ok!
452026-05-22T22:07:44.398ZINFOcrucible-agent (worker): SMF ok!
462026-05-22T22:07:44.398ZINFOcrucible-agent (datafile): region 1c5f2a5a-87e6-4868-9876-69557fb3b00b running snapshot 2454829e-c135-4d72-8911-45ad9cc85f35 state: Requested -> Created
472026-05-22T22:07:44.690ZINFOcrucible-agent (datafile): path is "/data/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b"
482026-05-22T22:07:44.696ZINFOcrucible-agent (datafile): dataset is oxp_d208cecb-d446-4037-8ac2-3b49d6cccdc3/crucible/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b
492026-05-22T22:07:44.702ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_d208cecb-d446-4037-8ac2-3b49d6cccdc3/crucible/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b@2454829e-c135-4d72-8911-45ad9cc85f35
502026-05-22T22:07:44.702ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_d208cecb-d446-4037-8ac2-3b49d6cccdc3/crucible/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b", "2454829e-c135-4d72-8911-45ad9cc85f35"]
512026-05-22T22:07:44.709ZINFOcrucible-agent (dropshot): request completed latency_us = 19169 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:43412 req_id = ba135349-4c94-4f1e-96bc-e497c562b0ba response_code = 200 uri = /crucible/0/regions/1c5f2a5a-87e6-4868-9876-69557fb3b00b/snapshots/2454829e-c135-4d72-8911-45ad9cc85f35/run