Max level shown:
12026-01-19T03:38:38.196ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::b]:64990
22026-01-19T03:38:38.197ZINFOcrucible-agent (datafile): region 2f98c707-9468-422f-bab1-2564171c5701 state: Requested
32026-01-19T03:38:38.197ZINFOcrucible-agent (dropshot): request completed latency_us = 611 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:64990 req_id = 2ff8b124-d3cb-411e-8290-06fd7d1b7e64 response_code = 200 uri = /crucible/0/regions
42026-01-19T03:38:38.197ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-01-19T03:38:38.206ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_3b9e1cd4-daf5-4f8b-b4a7-3b22e595ba9b/crucible/regions/2f98c707-9468-422f-bab1-2564171c5701
62026-01-19T03:38:38.206ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_3b9e1cd4-daf5-4f8b-b4a7-3b22e595ba9b/crucible/regions/2f98c707-9468-422f-bab1-2564171c5701
72026-01-19T03:38:38.264ZINFOcrucible-agent (worker): creating region Region { id: RegionId("2f98c707-9468-422f-bab1-2564171c5701"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/2f98c707-9468-422f-bab1-2564171c5701" region = 2f98c707-9468-422f-bab1-2564171c5701
82026-01-19T03:38:38.264ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "2f98c707-9468-422f-bab1-2564171c5701" "--data" "/data/regions/2f98c707-9468-422f-bab1-2564171c5701" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 2f98c707-9468-422f-bab1-2564171c5701
92026-01-19T03:38:38.340ZINFOcrucible-agent (worker): region files created ok region = 2f98c707-9468-422f-bab1-2564171c5701
102026-01-19T03:38:38.340ZINFOcrucible-agent (datafile): region 2f98c707-9468-422f-bab1-2564171c5701 state: Requested -> Created
112026-01-19T03:38:38.340ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-01-19T03:38:38.341ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-2f98c707-9468-422f-bab1-2564171c5701
132026-01-19T03:38:38.361ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-2f98c707-9468-422f-bab1-2564171c5701
142026-01-19T03:38:38.378ZINFOcrucible-agent (worker): creating config property group
152026-01-19T03:38:38.382ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-2f98c707-9468-422f-bab1-2564171c5701
162026-01-19T03:38:38.383ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/2f98c707-9468-422f-bab1-2564171c5701
172026-01-19T03:38:38.383ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-01-19T03:38:38.383ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
192026-01-19T03:38:38.383ZINFOcrucible-agent (worker): commit
202026-01-19T03:38:38.388ZINFOcrucible-agent (worker): ok!
212026-01-19T03:38:38.403ZINFOcrucible-agent (worker): SMF ok!
222026-01-19T03:38:38.543ZINFOcrucible-agent (dropshot): request completed latency_us = 67 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:64990 req_id = 38beee49-17dd-47ff-9016-9d91eb6b2c79 response_code = 200 uri = /crucible/0/regions
232026-01-19T03:38:38.905ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:64990 req_id = 441a1034-eee1-40a9-8470-ccd91248d5d9 response_code = 200 uri = /crucible/0/regions/d0e26259-8219-4104-a5c5-f1272edfeb72
242026-01-19T03:38:38.909ZINFOcrucible-agent (datafile): path is "/data/regions/d0e26259-8219-4104-a5c5-f1272edfeb72"
252026-01-19T03:38:38.918ZINFOcrucible-agent (datafile): dataset is oxp_3b9e1cd4-daf5-4f8b-b4a7-3b22e595ba9b/crucible/regions/d0e26259-8219-4104-a5c5-f1272edfeb72
262026-01-19T03:38:38.928ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_3b9e1cd4-daf5-4f8b-b4a7-3b22e595ba9b/crucible/regions/d0e26259-8219-4104-a5c5-f1272edfeb72@e4237574-e912-44be-9a5c-defb92cbd7f6
272026-01-19T03:38:38.928ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_3b9e1cd4-daf5-4f8b-b4a7-3b22e595ba9b/crucible/regions/d0e26259-8219-4104-a5c5-f1272edfeb72", "e4237574-e912-44be-9a5c-defb92cbd7f6"]
282026-01-19T03:38:38.939ZINFOcrucible-agent (dropshot): request completed latency_us = 30058 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:64990 req_id = f33d3ea6-d65b-484e-a0cf-c53dc58a9313 response_code = 200 uri = /crucible/0/regions/d0e26259-8219-4104-a5c5-f1272edfeb72/snapshots/e4237574-e912-44be-9a5c-defb92cbd7f6
292026-01-19T03:38:38.952ZINFOcrucible-agent (datafile): path is "/data/regions/d0e26259-8219-4104-a5c5-f1272edfeb72"
302026-01-19T03:38:38.961ZINFOcrucible-agent (datafile): dataset is oxp_3b9e1cd4-daf5-4f8b-b4a7-3b22e595ba9b/crucible/regions/d0e26259-8219-4104-a5c5-f1272edfeb72
312026-01-19T03:38:38.969ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_3b9e1cd4-daf5-4f8b-b4a7-3b22e595ba9b/crucible/regions/d0e26259-8219-4104-a5c5-f1272edfeb72@e4237574-e912-44be-9a5c-defb92cbd7f6
322026-01-19T03:38:38.969ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_3b9e1cd4-daf5-4f8b-b4a7-3b22e595ba9b/crucible/regions/d0e26259-8219-4104-a5c5-f1272edfeb72", "e4237574-e912-44be-9a5c-defb92cbd7f6"]
332026-01-19T03:38:38.979ZINFOcrucible-agent (datafile): requesting running snapshot d0e26259-8219-4104-a5c5-f1272edfeb72-e4237574-e912-44be-9a5c-defb92cbd7f6 state: Requested
342026-01-19T03:38:38.979ZINFOcrucible-agent (dropshot): request completed latency_us = 26845 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:64990 req_id = 27d74685-2481-4b78-b189-2bd9d6831301 response_code = 200 uri = /crucible/0/regions/d0e26259-8219-4104-a5c5-f1272edfeb72/snapshots/e4237574-e912-44be-9a5c-defb92cbd7f6/run
352026-01-19T03:38:38.979ZINFOcrucible-agent (worker): applying SMF actions for region d0e26259-8219-4104-a5c5-f1272edfeb72 running snapshot e4237574-e912-44be-9a5c-defb92cbd7f6 (state Requested)...
362026-01-19T03:38:38.986ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-d0e26259-8219-4104-a5c5-f1272edfeb72-e4237574-e912-44be-9a5c-defb92cbd7f6
372026-01-19T03:38:38.989ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-d0e26259-8219-4104-a5c5-f1272edfeb72-e4237574-e912-44be-9a5c-defb92cbd7f6
382026-01-19T03:38:39.000ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-d0e26259-8219-4104-a5c5-f1272edfeb72-e4237574-e912-44be-9a5c-defb92cbd7f6
392026-01-19T03:38:39.000ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d0e26259-8219-4104-a5c5-f1272edfeb72/.zfs/snapshot/e4237574-e912-44be-9a5c-defb92cbd7f6
402026-01-19T03:38:39.000ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-01-19T03:38:39.000ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-01-19T03:38:39.000ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432026-01-19T03:38:39.000ZINFOcrucible-agent (worker): commit
442026-01-19T03:38:39.005ZINFOcrucible-agent (worker): ok!
452026-01-19T03:38:39.010ZINFOcrucible-agent (worker): SMF ok!
462026-01-19T03:38:39.010ZINFOcrucible-agent (datafile): region d0e26259-8219-4104-a5c5-f1272edfeb72 running snapshot e4237574-e912-44be-9a5c-defb92cbd7f6 state: Requested -> Created
472026-01-19T03:38:39.137ZINFOcrucible-agent (datafile): path is "/data/regions/d0e26259-8219-4104-a5c5-f1272edfeb72"
482026-01-19T03:38:39.143ZINFOcrucible-agent (datafile): dataset is oxp_3b9e1cd4-daf5-4f8b-b4a7-3b22e595ba9b/crucible/regions/d0e26259-8219-4104-a5c5-f1272edfeb72
492026-01-19T03:38:39.150ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_3b9e1cd4-daf5-4f8b-b4a7-3b22e595ba9b/crucible/regions/d0e26259-8219-4104-a5c5-f1272edfeb72@e4237574-e912-44be-9a5c-defb92cbd7f6
502026-01-19T03:38:39.150ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_3b9e1cd4-daf5-4f8b-b4a7-3b22e595ba9b/crucible/regions/d0e26259-8219-4104-a5c5-f1272edfeb72", "e4237574-e912-44be-9a5c-defb92cbd7f6"]
512026-01-19T03:38:39.157ZINFOcrucible-agent (dropshot): request completed latency_us = 20838 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:64990 req_id = c161fd0f-a4c9-48cc-bd96-a41032b53edd response_code = 200 uri = /crucible/0/regions/d0e26259-8219-4104-a5c5-f1272edfeb72/snapshots/e4237574-e912-44be-9a5c-defb92cbd7f6/run