Max level shown:
12026-08-25T03:18:46.234ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::c]:37462
22026-08-25T03:18:46.234ZINFOcrucible-agent (datafile): region a90775d9-bb3e-40f8-83a3-895ab547fa43 state: Requested
32026-08-25T03:18:46.235ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-08-25T03:18:46.235ZINFOcrucible-agent (dropshot): request completed latency_us = 670 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:37462 req_id = e6715d29-8402-47b3-aebe-d84cf2ba6a69 response_code = 200 uri = /crucible/0/regions
52026-08-25T03:18:46.242ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_5afd5416-5005-44e6-bc42-9944d71eae92/crucible/regions/a90775d9-bb3e-40f8-83a3-895ab547fa43
62026-08-25T03:18:46.242ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_5afd5416-5005-44e6-bc42-9944d71eae92/crucible/regions/a90775d9-bb3e-40f8-83a3-895ab547fa43
72026-08-25T03:18:46.309ZINFOcrucible-agent (worker): creating region Region { id: RegionId("a90775d9-bb3e-40f8-83a3-895ab547fa43"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/a90775d9-bb3e-40f8-83a3-895ab547fa43" region = a90775d9-bb3e-40f8-83a3-895ab547fa43
82026-08-25T03:18:46.309ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "a90775d9-bb3e-40f8-83a3-895ab547fa43" "--data" "/data/regions/a90775d9-bb3e-40f8-83a3-895ab547fa43" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = a90775d9-bb3e-40f8-83a3-895ab547fa43
92026-08-25T03:18:46.416ZINFOcrucible-agent (worker): region files created ok region = a90775d9-bb3e-40f8-83a3-895ab547fa43
102026-08-25T03:18:46.416ZINFOcrucible-agent (datafile): region a90775d9-bb3e-40f8-83a3-895ab547fa43 state: Requested -> Created
112026-08-25T03:18:46.416ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-08-25T03:18:46.423ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-a90775d9-bb3e-40f8-83a3-895ab547fa43
132026-08-25T03:18:46.438ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-a90775d9-bb3e-40f8-83a3-895ab547fa43
142026-08-25T03:18:46.452ZINFOcrucible-agent (worker): creating config property group
152026-08-25T03:18:46.457ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a90775d9-bb3e-40f8-83a3-895ab547fa43
162026-08-25T03:18:46.457ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a90775d9-bb3e-40f8-83a3-895ab547fa43
172026-08-25T03:18:46.457ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-08-25T03:18:46.458ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
192026-08-25T03:18:46.458ZINFOcrucible-agent (worker): commit
202026-08-25T03:18:46.463ZINFOcrucible-agent (worker): ok!
212026-08-25T03:18:46.471ZINFOcrucible-agent (worker): SMF ok!
222026-08-25T03:18:46.493ZINFOcrucible-agent (dropshot): request completed latency_us = 66 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:37462 req_id = ddabf650-d7f5-4f1b-b12c-5d5d106c2e9a response_code = 200 uri = /crucible/0/regions
232026-08-25T03:18:47.501ZINFOcrucible-agent (dropshot): request completed latency_us = 80 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:37462 req_id = bd779ac9-e82c-4eda-a1eb-218091e87f38 response_code = 200 uri = /crucible/0/regions/104e1528-151f-4053-a1a3-ea5864c7bad6
242026-08-25T03:18:47.506ZINFOcrucible-agent (datafile): path is "/data/regions/104e1528-151f-4053-a1a3-ea5864c7bad6"
252026-08-25T03:18:47.516ZINFOcrucible-agent (datafile): dataset is oxp_5afd5416-5005-44e6-bc42-9944d71eae92/crucible/regions/104e1528-151f-4053-a1a3-ea5864c7bad6
262026-08-25T03:18:47.526ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_5afd5416-5005-44e6-bc42-9944d71eae92/crucible/regions/104e1528-151f-4053-a1a3-ea5864c7bad6@6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf
272026-08-25T03:18:47.526ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_5afd5416-5005-44e6-bc42-9944d71eae92/crucible/regions/104e1528-151f-4053-a1a3-ea5864c7bad6", "6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf"]
282026-08-25T03:18:47.543ZINFOcrucible-agent (dropshot): request completed latency_us = 37423 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:37462 req_id = 62df7754-ed1e-4b61-9078-af0c6f41b9c4 response_code = 200 uri = /crucible/0/regions/104e1528-151f-4053-a1a3-ea5864c7bad6/snapshots/6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf
292026-08-25T03:18:47.559ZINFOcrucible-agent (datafile): path is "/data/regions/104e1528-151f-4053-a1a3-ea5864c7bad6"
302026-08-25T03:18:47.570ZINFOcrucible-agent (datafile): dataset is oxp_5afd5416-5005-44e6-bc42-9944d71eae92/crucible/regions/104e1528-151f-4053-a1a3-ea5864c7bad6
312026-08-25T03:18:47.579ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_5afd5416-5005-44e6-bc42-9944d71eae92/crucible/regions/104e1528-151f-4053-a1a3-ea5864c7bad6@6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf
322026-08-25T03:18:47.579ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_5afd5416-5005-44e6-bc42-9944d71eae92/crucible/regions/104e1528-151f-4053-a1a3-ea5864c7bad6", "6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf"]
332026-08-25T03:18:47.589ZINFOcrucible-agent (datafile): requesting running snapshot 104e1528-151f-4053-a1a3-ea5864c7bad6-6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf state: Requested
342026-08-25T03:18:47.595ZINFOcrucible-agent (worker): applying SMF actions for region 104e1528-151f-4053-a1a3-ea5864c7bad6 running snapshot 6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf (state Requested)...
352026-08-25T03:18:47.595ZINFOcrucible-agent (dropshot): request completed latency_us = 36131 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:37462 req_id = d8dbcee0-7207-4bed-b0e8-3adf08afd7a9 response_code = 200 uri = /crucible/0/regions/104e1528-151f-4053-a1a3-ea5864c7bad6/snapshots/6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf/run
362026-08-25T03:18:47.598ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-104e1528-151f-4053-a1a3-ea5864c7bad6-6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf
372026-08-25T03:18:47.602ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-104e1528-151f-4053-a1a3-ea5864c7bad6-6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf
382026-08-25T03:18:47.619ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-104e1528-151f-4053-a1a3-ea5864c7bad6-6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf
392026-08-25T03:18:47.620ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/104e1528-151f-4053-a1a3-ea5864c7bad6/.zfs/snapshot/6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf
402026-08-25T03:18:47.620ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-08-25T03:18:47.620ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-08-25T03:18:47.620ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432026-08-25T03:18:47.620ZINFOcrucible-agent (worker): commit
442026-08-25T03:18:47.632ZINFOcrucible-agent (worker): ok!
452026-08-25T03:18:47.637ZINFOcrucible-agent (worker): SMF ok!
462026-08-25T03:18:47.637ZINFOcrucible-agent (datafile): region 104e1528-151f-4053-a1a3-ea5864c7bad6 running snapshot 6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf state: Requested -> Created
472026-08-25T03:18:47.905ZINFOcrucible-agent (datafile): path is "/data/regions/104e1528-151f-4053-a1a3-ea5864c7bad6"
482026-08-25T03:18:47.912ZINFOcrucible-agent (datafile): dataset is oxp_5afd5416-5005-44e6-bc42-9944d71eae92/crucible/regions/104e1528-151f-4053-a1a3-ea5864c7bad6
492026-08-25T03:18:47.918ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_5afd5416-5005-44e6-bc42-9944d71eae92/crucible/regions/104e1528-151f-4053-a1a3-ea5864c7bad6@6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf
502026-08-25T03:18:47.918ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_5afd5416-5005-44e6-bc42-9944d71eae92/crucible/regions/104e1528-151f-4053-a1a3-ea5864c7bad6", "6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf"]
512026-08-25T03:18:47.923ZINFOcrucible-agent (dropshot): request completed latency_us = 17383 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:37462 req_id = 3a0654f9-1d63-4d68-adf9-9e01a3032262 response_code = 200 uri = /crucible/0/regions/104e1528-151f-4053-a1a3-ea5864c7bad6/snapshots/6289163c-f1b5-4ac4-a5b5-f9fb8deae4cf/run