Max level shown:
12026-01-13T00:49:31.455ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::b]:63914
22026-01-13T00:49:31.455ZINFOcrucible-agent (datafile): region 5319e0a4-2f2f-4623-b060-7553dab7a46d state: Requested
32026-01-13T00:49:31.456ZINFOcrucible-agent (dropshot): request completed latency_us = 943 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:63914 req_id = 513ae444-aa9b-463e-b78e-f03180b5aaf4 response_code = 200 uri = /crucible/0/regions
42026-01-13T00:49:31.456ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-01-13T00:49:31.465ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_f7b7f4ed-1a6c-4468-b428-af24f29483d9/crucible/regions/5319e0a4-2f2f-4623-b060-7553dab7a46d
62026-01-13T00:49:31.465ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_f7b7f4ed-1a6c-4468-b428-af24f29483d9/crucible/regions/5319e0a4-2f2f-4623-b060-7553dab7a46d
72026-01-13T00:49:31.527ZINFOcrucible-agent (worker): creating region Region { id: RegionId("5319e0a4-2f2f-4623-b060-7553dab7a46d"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/5319e0a4-2f2f-4623-b060-7553dab7a46d" region = 5319e0a4-2f2f-4623-b060-7553dab7a46d
82026-01-13T00:49:31.527ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "5319e0a4-2f2f-4623-b060-7553dab7a46d" "--data" "/data/regions/5319e0a4-2f2f-4623-b060-7553dab7a46d" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 5319e0a4-2f2f-4623-b060-7553dab7a46d
92026-01-13T00:49:31.615ZINFOcrucible-agent (worker): region files created ok region = 5319e0a4-2f2f-4623-b060-7553dab7a46d
102026-01-13T00:49:31.615ZINFOcrucible-agent (datafile): region 5319e0a4-2f2f-4623-b060-7553dab7a46d state: Requested -> Created
112026-01-13T00:49:31.615ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-01-13T00:49:31.636ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-5319e0a4-2f2f-4623-b060-7553dab7a46d
132026-01-13T00:49:31.642ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-5319e0a4-2f2f-4623-b060-7553dab7a46d
142026-01-13T00:49:31.648ZINFOcrucible-agent (worker): creating config property group
152026-01-13T00:49:31.651ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-5319e0a4-2f2f-4623-b060-7553dab7a46d
162026-01-13T00:49:31.651ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/5319e0a4-2f2f-4623-b060-7553dab7a46d
172026-01-13T00:49:31.651ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-01-13T00:49:31.651ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
192026-01-13T00:49:31.651ZINFOcrucible-agent (worker): commit
202026-01-13T00:49:31.655ZINFOcrucible-agent (worker): ok!
212026-01-13T00:49:31.659ZINFOcrucible-agent (worker): SMF ok!
222026-01-13T00:49:31.674ZINFOcrucible-agent (dropshot): request completed latency_us = 61 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:63914 req_id = e01cb6f7-f5d0-43c1-843f-14a626a6f49a response_code = 200 uri = /crucible/0/regions
232026-01-13T00:49:32.686ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:63914 req_id = e7beb8a5-db91-4ad8-9476-c542809222e8 response_code = 200 uri = /crucible/0/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b
242026-01-13T00:49:32.690ZINFOcrucible-agent (datafile): path is "/data/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b"
252026-01-13T00:49:32.697ZINFOcrucible-agent (datafile): dataset is oxp_f7b7f4ed-1a6c-4468-b428-af24f29483d9/crucible/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b
262026-01-13T00:49:32.703ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f7b7f4ed-1a6c-4468-b428-af24f29483d9/crucible/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b@d9e2d10b-bdd5-468d-8211-52723d9f0c73
272026-01-13T00:49:32.703ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f7b7f4ed-1a6c-4468-b428-af24f29483d9/crucible/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b", "d9e2d10b-bdd5-468d-8211-52723d9f0c73"]
282026-01-13T00:49:32.711ZINFOcrucible-agent (dropshot): request completed latency_us = 21083 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:63914 req_id = 12c38c06-1da8-4032-8d95-105fbe21ea46 response_code = 200 uri = /crucible/0/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b/snapshots/d9e2d10b-bdd5-468d-8211-52723d9f0c73
292026-01-13T00:49:32.714ZINFOcrucible-agent (datafile): path is "/data/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b"
302026-01-13T00:49:32.719ZINFOcrucible-agent (datafile): dataset is oxp_f7b7f4ed-1a6c-4468-b428-af24f29483d9/crucible/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b
312026-01-13T00:49:32.725ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f7b7f4ed-1a6c-4468-b428-af24f29483d9/crucible/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b@d9e2d10b-bdd5-468d-8211-52723d9f0c73
322026-01-13T00:49:32.725ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f7b7f4ed-1a6c-4468-b428-af24f29483d9/crucible/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b", "d9e2d10b-bdd5-468d-8211-52723d9f0c73"]
332026-01-13T00:49:32.733ZINFOcrucible-agent (datafile): requesting running snapshot 17438d8f-0d2d-4a59-8c90-a19631ceba0b-d9e2d10b-bdd5-468d-8211-52723d9f0c73 state: Requested
342026-01-13T00:49:32.733ZINFOcrucible-agent (dropshot): request completed latency_us = 18830 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:63914 req_id = 919e61b5-ac54-4fbe-8f41-fc4797a70225 response_code = 200 uri = /crucible/0/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b/snapshots/d9e2d10b-bdd5-468d-8211-52723d9f0c73/run
352026-01-13T00:49:32.733ZINFOcrucible-agent (worker): applying SMF actions for region 17438d8f-0d2d-4a59-8c90-a19631ceba0b running snapshot d9e2d10b-bdd5-468d-8211-52723d9f0c73 (state Requested)...
362026-01-13T00:49:32.737ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-17438d8f-0d2d-4a59-8c90-a19631ceba0b-d9e2d10b-bdd5-468d-8211-52723d9f0c73
372026-01-13T00:49:32.745ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-17438d8f-0d2d-4a59-8c90-a19631ceba0b-d9e2d10b-bdd5-468d-8211-52723d9f0c73
382026-01-13T00:49:32.756ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-17438d8f-0d2d-4a59-8c90-a19631ceba0b-d9e2d10b-bdd5-468d-8211-52723d9f0c73
392026-01-13T00:49:32.756ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b/.zfs/snapshot/d9e2d10b-bdd5-468d-8211-52723d9f0c73
402026-01-13T00:49:32.756ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-01-13T00:49:32.756ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-01-13T00:49:32.756ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
432026-01-13T00:49:32.756ZINFOcrucible-agent (worker): commit
442026-01-13T00:49:32.760ZINFOcrucible-agent (worker): ok!
452026-01-13T00:49:32.764ZINFOcrucible-agent (worker): SMF ok!
462026-01-13T00:49:32.764ZINFOcrucible-agent (datafile): region 17438d8f-0d2d-4a59-8c90-a19631ceba0b running snapshot d9e2d10b-bdd5-468d-8211-52723d9f0c73 state: Requested -> Created
472026-01-13T00:49:33.027ZINFOcrucible-agent (datafile): path is "/data/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b"
482026-01-13T00:49:33.034ZINFOcrucible-agent (datafile): dataset is oxp_f7b7f4ed-1a6c-4468-b428-af24f29483d9/crucible/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b
492026-01-13T00:49:33.040ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_f7b7f4ed-1a6c-4468-b428-af24f29483d9/crucible/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b@d9e2d10b-bdd5-468d-8211-52723d9f0c73
502026-01-13T00:49:33.040ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_f7b7f4ed-1a6c-4468-b428-af24f29483d9/crucible/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b", "d9e2d10b-bdd5-468d-8211-52723d9f0c73"]
512026-01-13T00:49:33.047ZINFOcrucible-agent (dropshot): request completed latency_us = 20173 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:63914 req_id = 9c8ae38a-065b-4bc2-80f0-a3ae1f3c548e response_code = 200 uri = /crucible/0/regions/17438d8f-0d2d-4a59-8c90-a19631ceba0b/snapshots/d9e2d10b-bdd5-468d-8211-52723d9f0c73/run