Max level shown:
12026-03-17T00:00:40.169ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::b]:34525
22026-03-17T00:00:40.169ZINFOcrucible-agent (datafile): region abda1616-1e3c-4df4-9011-8ff685987e96 state: Requested
32026-03-17T00:00:40.169ZINFOcrucible-agent (dropshot): request completed latency_us = 692 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:34525 req_id = c5afff63-a1cd-49fc-928b-04c61308f303 response_code = 200 uri = /crucible/0/regions
42026-03-17T00:00:40.169ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-03-17T00:00:40.177ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_ffc24c46-2e65-4754-8308-75c016d91387/crucible/regions/abda1616-1e3c-4df4-9011-8ff685987e96
62026-03-17T00:00:40.177ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_ffc24c46-2e65-4754-8308-75c016d91387/crucible/regions/abda1616-1e3c-4df4-9011-8ff685987e96
72026-03-17T00:00:40.226ZINFOcrucible-agent (worker): creating region Region { id: RegionId("abda1616-1e3c-4df4-9011-8ff685987e96"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/abda1616-1e3c-4df4-9011-8ff685987e96" region = abda1616-1e3c-4df4-9011-8ff685987e96
82026-03-17T00:00:40.226ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "abda1616-1e3c-4df4-9011-8ff685987e96" "--data" "/data/regions/abda1616-1e3c-4df4-9011-8ff685987e96" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = abda1616-1e3c-4df4-9011-8ff685987e96
92026-03-17T00:00:40.345ZINFOcrucible-agent (worker): region files created ok region = abda1616-1e3c-4df4-9011-8ff685987e96
102026-03-17T00:00:40.345ZINFOcrucible-agent (datafile): region abda1616-1e3c-4df4-9011-8ff685987e96 state: Requested -> Created
112026-03-17T00:00:40.345ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-03-17T00:00:40.365ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-abda1616-1e3c-4df4-9011-8ff685987e96
132026-03-17T00:00:40.372ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-abda1616-1e3c-4df4-9011-8ff685987e96
142026-03-17T00:00:40.383ZINFOcrucible-agent (worker): creating config property group
152026-03-17T00:00:40.386ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-abda1616-1e3c-4df4-9011-8ff685987e96
162026-03-17T00:00:40.386ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/abda1616-1e3c-4df4-9011-8ff685987e96
172026-03-17T00:00:40.386ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-03-17T00:00:40.386ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
192026-03-17T00:00:40.386ZINFOcrucible-agent (worker): commit
202026-03-17T00:00:40.390ZINFOcrucible-agent (worker): ok!
212026-03-17T00:00:40.395ZINFOcrucible-agent (worker): SMF ok!
222026-03-17T00:00:40.419ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:34525 req_id = 1b468419-756c-4003-a1bc-6fa50fe20ac4 response_code = 200 uri = /crucible/0/regions
232026-03-17T00:00:41.706ZINFOcrucible-agent (dropshot): request completed latency_us = 73 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:34525 req_id = a3de44b7-429b-40af-89f5-1e0fd0fb5b0b response_code = 200 uri = /crucible/0/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2
242026-03-17T00:00:41.709ZINFOcrucible-agent (datafile): path is "/data/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2"
252026-03-17T00:00:41.715ZINFOcrucible-agent (datafile): dataset is oxp_ffc24c46-2e65-4754-8308-75c016d91387/crucible/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2
262026-03-17T00:00:41.722ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ffc24c46-2e65-4754-8308-75c016d91387/crucible/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2@f1a02dc4-ab12-4949-8791-fda9f0d52fc6
272026-03-17T00:00:41.722ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ffc24c46-2e65-4754-8308-75c016d91387/crucible/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2", "f1a02dc4-ab12-4949-8791-fda9f0d52fc6"]
282026-03-17T00:00:41.731ZINFOcrucible-agent (dropshot): request completed latency_us = 21785 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:34525 req_id = dee94cf5-bca3-441d-b8bb-d7aade4325c2 response_code = 200 uri = /crucible/0/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2/snapshots/f1a02dc4-ab12-4949-8791-fda9f0d52fc6
292026-03-17T00:00:41.734ZINFOcrucible-agent (datafile): path is "/data/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2"
302026-03-17T00:00:41.740ZINFOcrucible-agent (datafile): dataset is oxp_ffc24c46-2e65-4754-8308-75c016d91387/crucible/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2
312026-03-17T00:00:41.744ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ffc24c46-2e65-4754-8308-75c016d91387/crucible/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2@f1a02dc4-ab12-4949-8791-fda9f0d52fc6
322026-03-17T00:00:41.744ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ffc24c46-2e65-4754-8308-75c016d91387/crucible/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2", "f1a02dc4-ab12-4949-8791-fda9f0d52fc6"]
332026-03-17T00:00:41.753ZINFOcrucible-agent (datafile): requesting running snapshot 3a067ee1-29d5-4835-b2ca-390e46056fe2-f1a02dc4-ab12-4949-8791-fda9f0d52fc6 state: Requested
342026-03-17T00:00:41.753ZINFOcrucible-agent (dropshot): request completed latency_us = 18513 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:34525 req_id = 7c69d52a-8e73-4f42-bba8-9dd4a633aaff response_code = 200 uri = /crucible/0/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2/snapshots/f1a02dc4-ab12-4949-8791-fda9f0d52fc6/run
352026-03-17T00:00:41.753ZINFOcrucible-agent (worker): applying SMF actions for region 3a067ee1-29d5-4835-b2ca-390e46056fe2 running snapshot f1a02dc4-ab12-4949-8791-fda9f0d52fc6 (state Requested)...
362026-03-17T00:00:41.757ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-3a067ee1-29d5-4835-b2ca-390e46056fe2-f1a02dc4-ab12-4949-8791-fda9f0d52fc6
372026-03-17T00:00:41.769ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-3a067ee1-29d5-4835-b2ca-390e46056fe2-f1a02dc4-ab12-4949-8791-fda9f0d52fc6
382026-03-17T00:00:41.779ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-3a067ee1-29d5-4835-b2ca-390e46056fe2-f1a02dc4-ab12-4949-8791-fda9f0d52fc6
392026-03-17T00:00:41.779ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2/.zfs/snapshot/f1a02dc4-ab12-4949-8791-fda9f0d52fc6
402026-03-17T00:00:41.779ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-03-17T00:00:41.779ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-03-17T00:00:41.779ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
432026-03-17T00:00:41.779ZINFOcrucible-agent (worker): commit
442026-03-17T00:00:41.782ZINFOcrucible-agent (worker): ok!
452026-03-17T00:00:41.787ZINFOcrucible-agent (worker): SMF ok!
462026-03-17T00:00:41.787ZINFOcrucible-agent (datafile): region 3a067ee1-29d5-4835-b2ca-390e46056fe2 running snapshot f1a02dc4-ab12-4949-8791-fda9f0d52fc6 state: Requested -> Created
472026-03-17T00:00:41.992ZINFOcrucible-agent (datafile): path is "/data/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2"
482026-03-17T00:00:41.998ZINFOcrucible-agent (datafile): dataset is oxp_ffc24c46-2e65-4754-8308-75c016d91387/crucible/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2
492026-03-17T00:00:42.004ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ffc24c46-2e65-4754-8308-75c016d91387/crucible/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2@f1a02dc4-ab12-4949-8791-fda9f0d52fc6
502026-03-17T00:00:42.004ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ffc24c46-2e65-4754-8308-75c016d91387/crucible/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2", "f1a02dc4-ab12-4949-8791-fda9f0d52fc6"]
512026-03-17T00:00:42.010ZINFOcrucible-agent (dropshot): request completed latency_us = 18494 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:34525 req_id = b000147b-910b-453a-804d-4e679092c782 response_code = 200 uri = /crucible/0/regions/3a067ee1-29d5-4835-b2ca-390e46056fe2/snapshots/f1a02dc4-ab12-4949-8791-fda9f0d52fc6/run