Max level shown:
12025-12-24T05:54:27.519ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::b]:46645
22025-12-24T05:54:27.520ZINFOcrucible-agent (datafile): region 05b4d890-85ec-4d93-b935-560599f00d01 state: Requested
32025-12-24T05:54:27.521ZINFOcrucible-agent (dropshot): request completed latency_us = 1431 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:46645 req_id = fcf88a69-2900-42a2-b2c4-afafb417d670 response_code = 200 uri = /crucible/0/regions
42025-12-24T05:54:27.521ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52025-12-24T05:54:27.528ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_b5c1611c-99f5-4a85-acc0-6baa14c5138e/crucible/regions/05b4d890-85ec-4d93-b935-560599f00d01
62025-12-24T05:54:27.528ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_b5c1611c-99f5-4a85-acc0-6baa14c5138e/crucible/regions/05b4d890-85ec-4d93-b935-560599f00d01
72025-12-24T05:54:27.580ZINFOcrucible-agent (worker): creating region Region { id: RegionId("05b4d890-85ec-4d93-b935-560599f00d01"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/05b4d890-85ec-4d93-b935-560599f00d01" region = 05b4d890-85ec-4d93-b935-560599f00d01
82025-12-24T05:54:27.580ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "05b4d890-85ec-4d93-b935-560599f00d01" "--data" "/data/regions/05b4d890-85ec-4d93-b935-560599f00d01" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 05b4d890-85ec-4d93-b935-560599f00d01
92025-12-24T05:54:27.670ZINFOcrucible-agent (worker): region files created ok region = 05b4d890-85ec-4d93-b935-560599f00d01
102025-12-24T05:54:27.670ZINFOcrucible-agent (datafile): region 05b4d890-85ec-4d93-b935-560599f00d01 state: Requested -> Created
112025-12-24T05:54:27.670ZINFOcrucible-agent (worker): applying SMF actions post create...
122025-12-24T05:54:27.670ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-05b4d890-85ec-4d93-b935-560599f00d01
132025-12-24T05:54:27.682ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-05b4d890-85ec-4d93-b935-560599f00d01
142025-12-24T05:54:27.709ZINFOcrucible-agent (worker): creating config property group
152025-12-24T05:54:27.713ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-05b4d890-85ec-4d93-b935-560599f00d01
162025-12-24T05:54:27.713ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/05b4d890-85ec-4d93-b935-560599f00d01
172025-12-24T05:54:27.713ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182025-12-24T05:54:27.713ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
192025-12-24T05:54:27.713ZINFOcrucible-agent (worker): commit
202025-12-24T05:54:27.718ZINFOcrucible-agent (worker): ok!
212025-12-24T05:54:27.739ZINFOcrucible-agent (worker): SMF ok!
222025-12-24T05:54:27.804ZINFOcrucible-agent (dropshot): request completed latency_us = 56 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:46645 req_id = 73a7e989-0bd2-483d-912b-b805d1d7072d response_code = 200 uri = /crucible/0/regions
232025-12-24T05:54:29.604ZINFOcrucible-agent (dropshot): request completed latency_us = 68 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:46645 req_id = 0ac9f230-17ff-4c78-a679-004f2279a327 response_code = 200 uri = /crucible/0/regions/94824d0a-b98d-4927-b397-b7110859a07b
242025-12-24T05:54:29.607ZINFOcrucible-agent (datafile): path is "/data/regions/94824d0a-b98d-4927-b397-b7110859a07b"
252025-12-24T05:54:29.614ZINFOcrucible-agent (datafile): dataset is oxp_b5c1611c-99f5-4a85-acc0-6baa14c5138e/crucible/regions/94824d0a-b98d-4927-b397-b7110859a07b
262025-12-24T05:54:29.620ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b5c1611c-99f5-4a85-acc0-6baa14c5138e/crucible/regions/94824d0a-b98d-4927-b397-b7110859a07b@c8a59639-5d7e-4a5a-b19d-b92aa18f6ead
272025-12-24T05:54:29.620ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b5c1611c-99f5-4a85-acc0-6baa14c5138e/crucible/regions/94824d0a-b98d-4927-b397-b7110859a07b", "c8a59639-5d7e-4a5a-b19d-b92aa18f6ead"]
282025-12-24T05:54:29.628ZINFOcrucible-agent (dropshot): request completed latency_us = 20576 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:46645 req_id = d7267604-315f-417d-a019-438f7000d6f3 response_code = 200 uri = /crucible/0/regions/94824d0a-b98d-4927-b397-b7110859a07b/snapshots/c8a59639-5d7e-4a5a-b19d-b92aa18f6ead
292025-12-24T05:54:29.631ZINFOcrucible-agent (datafile): path is "/data/regions/94824d0a-b98d-4927-b397-b7110859a07b"
302025-12-24T05:54:29.638ZINFOcrucible-agent (datafile): dataset is oxp_b5c1611c-99f5-4a85-acc0-6baa14c5138e/crucible/regions/94824d0a-b98d-4927-b397-b7110859a07b
312025-12-24T05:54:29.644ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b5c1611c-99f5-4a85-acc0-6baa14c5138e/crucible/regions/94824d0a-b98d-4927-b397-b7110859a07b@c8a59639-5d7e-4a5a-b19d-b92aa18f6ead
322025-12-24T05:54:29.644ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b5c1611c-99f5-4a85-acc0-6baa14c5138e/crucible/regions/94824d0a-b98d-4927-b397-b7110859a07b", "c8a59639-5d7e-4a5a-b19d-b92aa18f6ead"]
332025-12-24T05:54:29.651ZINFOcrucible-agent (datafile): requesting running snapshot 94824d0a-b98d-4927-b397-b7110859a07b-c8a59639-5d7e-4a5a-b19d-b92aa18f6ead state: Requested
342025-12-24T05:54:29.651ZINFOcrucible-agent (dropshot): request completed latency_us = 19754 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:46645 req_id = 41d9eef5-2118-4cda-82a3-9c9fd91b16d1 response_code = 200 uri = /crucible/0/regions/94824d0a-b98d-4927-b397-b7110859a07b/snapshots/c8a59639-5d7e-4a5a-b19d-b92aa18f6ead/run
352025-12-24T05:54:29.651ZINFOcrucible-agent (worker): applying SMF actions for region 94824d0a-b98d-4927-b397-b7110859a07b running snapshot c8a59639-5d7e-4a5a-b19d-b92aa18f6ead (state Requested)...
362025-12-24T05:54:29.656ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-94824d0a-b98d-4927-b397-b7110859a07b-c8a59639-5d7e-4a5a-b19d-b92aa18f6ead
372025-12-24T05:54:29.664ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-94824d0a-b98d-4927-b397-b7110859a07b-c8a59639-5d7e-4a5a-b19d-b92aa18f6ead
382025-12-24T05:54:29.675ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-94824d0a-b98d-4927-b397-b7110859a07b-c8a59639-5d7e-4a5a-b19d-b92aa18f6ead
392025-12-24T05:54:29.675ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/94824d0a-b98d-4927-b397-b7110859a07b/.zfs/snapshot/c8a59639-5d7e-4a5a-b19d-b92aa18f6ead
402025-12-24T05:54:29.675ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412025-12-24T05:54:29.675ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422025-12-24T05:54:29.675ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
432025-12-24T05:54:29.675ZINFOcrucible-agent (worker): commit
442025-12-24T05:54:29.679ZINFOcrucible-agent (worker): ok!
452025-12-24T05:54:29.684ZINFOcrucible-agent (worker): SMF ok!
462025-12-24T05:54:29.684ZINFOcrucible-agent (datafile): region 94824d0a-b98d-4927-b397-b7110859a07b running snapshot c8a59639-5d7e-4a5a-b19d-b92aa18f6ead state: Requested -> Created
472025-12-24T05:54:30.022ZINFOcrucible-agent (datafile): path is "/data/regions/94824d0a-b98d-4927-b397-b7110859a07b"
482025-12-24T05:54:30.032ZINFOcrucible-agent (datafile): dataset is oxp_b5c1611c-99f5-4a85-acc0-6baa14c5138e/crucible/regions/94824d0a-b98d-4927-b397-b7110859a07b
492025-12-24T05:54:30.042ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b5c1611c-99f5-4a85-acc0-6baa14c5138e/crucible/regions/94824d0a-b98d-4927-b397-b7110859a07b@c8a59639-5d7e-4a5a-b19d-b92aa18f6ead
502025-12-24T05:54:30.042ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b5c1611c-99f5-4a85-acc0-6baa14c5138e/crucible/regions/94824d0a-b98d-4927-b397-b7110859a07b", "c8a59639-5d7e-4a5a-b19d-b92aa18f6ead"]
512025-12-24T05:54:30.050ZINFOcrucible-agent (dropshot): request completed latency_us = 27921 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:46645 req_id = 9da72538-a7bd-4946-b0a0-dc25a97cc116 response_code = 200 uri = /crucible/0/regions/94824d0a-b98d-4927-b397-b7110859a07b/snapshots/c8a59639-5d7e-4a5a-b19d-b92aa18f6ead/run