Max level shown:
12026-05-27T00:07:40.945ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::b]:35216
22026-05-27T00:07:40.945ZINFOcrucible-agent (datafile): region f593705b-cb7e-4ae8-9f2a-71147c9c2976 state: Requested
32026-05-27T00:07:40.946ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-05-27T00:07:40.946ZINFOcrucible-agent (dropshot): request completed latency_us = 742 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:35216 req_id = 92501cee-c3bf-44c9-af37-013f54b1502d response_code = 200 uri = /crucible/0/regions
52026-05-27T00:07:40.955ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_571be214-bd36-4896-ac5d-5a6be1d32533/crucible/regions/f593705b-cb7e-4ae8-9f2a-71147c9c2976
62026-05-27T00:07:40.955ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_571be214-bd36-4896-ac5d-5a6be1d32533/crucible/regions/f593705b-cb7e-4ae8-9f2a-71147c9c2976
72026-05-27T00:07:40.998ZINFOcrucible-agent (worker): creating region Region { id: RegionId("f593705b-cb7e-4ae8-9f2a-71147c9c2976"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/f593705b-cb7e-4ae8-9f2a-71147c9c2976" region = f593705b-cb7e-4ae8-9f2a-71147c9c2976
82026-05-27T00:07:40.998ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "f593705b-cb7e-4ae8-9f2a-71147c9c2976" "--data" "/data/regions/f593705b-cb7e-4ae8-9f2a-71147c9c2976" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = f593705b-cb7e-4ae8-9f2a-71147c9c2976
92026-05-27T00:07:41.049ZINFOcrucible-agent (worker): region files created ok region = f593705b-cb7e-4ae8-9f2a-71147c9c2976
102026-05-27T00:07:41.049ZINFOcrucible-agent (datafile): region f593705b-cb7e-4ae8-9f2a-71147c9c2976 state: Requested -> Created
112026-05-27T00:07:41.049ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-05-27T00:07:41.056ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-f593705b-cb7e-4ae8-9f2a-71147c9c2976
132026-05-27T00:07:41.064ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-f593705b-cb7e-4ae8-9f2a-71147c9c2976
142026-05-27T00:07:41.070ZINFOcrucible-agent (worker): creating config property group
152026-05-27T00:07:41.071ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-f593705b-cb7e-4ae8-9f2a-71147c9c2976
162026-05-27T00:07:41.072ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/f593705b-cb7e-4ae8-9f2a-71147c9c2976
172026-05-27T00:07:41.072ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-05-27T00:07:41.072ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
192026-05-27T00:07:41.072ZINFOcrucible-agent (worker): commit
202026-05-27T00:07:41.073ZINFOcrucible-agent (worker): ok!
212026-05-27T00:07:41.076ZINFOcrucible-agent (worker): SMF ok!
222026-05-27T00:07:41.324ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:35216 req_id = 0f819a88-4f90-4d08-80b5-d4963cc1e68d response_code = 200 uri = /crucible/0/regions
232026-05-27T00:07:42.430ZINFOcrucible-agent (dropshot): request completed latency_us = 51 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:35216 req_id = b8a135f9-b525-451b-8225-ac712ae9eab9 response_code = 200 uri = /crucible/0/regions/00403839-ff09-499f-a317-70e7a964a050
242026-05-27T00:07:42.433ZINFOcrucible-agent (datafile): path is "/data/regions/00403839-ff09-499f-a317-70e7a964a050"
252026-05-27T00:07:42.439ZINFOcrucible-agent (datafile): dataset is oxp_571be214-bd36-4896-ac5d-5a6be1d32533/crucible/regions/00403839-ff09-499f-a317-70e7a964a050
262026-05-27T00:07:42.445ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_571be214-bd36-4896-ac5d-5a6be1d32533/crucible/regions/00403839-ff09-499f-a317-70e7a964a050@bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b
272026-05-27T00:07:42.445ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_571be214-bd36-4896-ac5d-5a6be1d32533/crucible/regions/00403839-ff09-499f-a317-70e7a964a050", "bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b"]
282026-05-27T00:07:42.454ZINFOcrucible-agent (dropshot): request completed latency_us = 21027 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:35216 req_id = c5b3d9d4-55e7-4dae-bcf5-c3faed26fe14 response_code = 200 uri = /crucible/0/regions/00403839-ff09-499f-a317-70e7a964a050/snapshots/bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b
292026-05-27T00:07:42.457ZINFOcrucible-agent (datafile): path is "/data/regions/00403839-ff09-499f-a317-70e7a964a050"
302026-05-27T00:07:42.463ZINFOcrucible-agent (datafile): dataset is oxp_571be214-bd36-4896-ac5d-5a6be1d32533/crucible/regions/00403839-ff09-499f-a317-70e7a964a050
312026-05-27T00:07:42.468ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_571be214-bd36-4896-ac5d-5a6be1d32533/crucible/regions/00403839-ff09-499f-a317-70e7a964a050@bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b
322026-05-27T00:07:42.468ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_571be214-bd36-4896-ac5d-5a6be1d32533/crucible/regions/00403839-ff09-499f-a317-70e7a964a050", "bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b"]
332026-05-27T00:07:42.475ZINFOcrucible-agent (datafile): requesting running snapshot 00403839-ff09-499f-a317-70e7a964a050-bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b state: Requested
342026-05-27T00:07:42.475ZINFOcrucible-agent (dropshot): request completed latency_us = 17381 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:35216 req_id = d78a101b-84c8-4bf8-b1af-99893170e00a response_code = 200 uri = /crucible/0/regions/00403839-ff09-499f-a317-70e7a964a050/snapshots/bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b/run
352026-05-27T00:07:42.475ZINFOcrucible-agent (worker): applying SMF actions for region 00403839-ff09-499f-a317-70e7a964a050 running snapshot bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b (state Requested)...
362026-05-27T00:07:42.480ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-00403839-ff09-499f-a317-70e7a964a050-bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b
372026-05-27T00:07:42.484ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-00403839-ff09-499f-a317-70e7a964a050-bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b
382026-05-27T00:07:42.488ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-00403839-ff09-499f-a317-70e7a964a050-bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b
392026-05-27T00:07:42.488ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/00403839-ff09-499f-a317-70e7a964a050/.zfs/snapshot/bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b
402026-05-27T00:07:42.488ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-05-27T00:07:42.488ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-05-27T00:07:42.488ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
432026-05-27T00:07:42.488ZINFOcrucible-agent (worker): commit
442026-05-27T00:07:42.490ZINFOcrucible-agent (worker): ok!
452026-05-27T00:07:42.494ZINFOcrucible-agent (worker): SMF ok!
462026-05-27T00:07:42.494ZINFOcrucible-agent (datafile): region 00403839-ff09-499f-a317-70e7a964a050 running snapshot bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b state: Requested -> Created
472026-05-27T00:07:42.854ZINFOcrucible-agent (datafile): path is "/data/regions/00403839-ff09-499f-a317-70e7a964a050"
482026-05-27T00:07:42.860ZINFOcrucible-agent (datafile): dataset is oxp_571be214-bd36-4896-ac5d-5a6be1d32533/crucible/regions/00403839-ff09-499f-a317-70e7a964a050
492026-05-27T00:07:42.867ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_571be214-bd36-4896-ac5d-5a6be1d32533/crucible/regions/00403839-ff09-499f-a317-70e7a964a050@bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b
502026-05-27T00:07:42.867ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_571be214-bd36-4896-ac5d-5a6be1d32533/crucible/regions/00403839-ff09-499f-a317-70e7a964a050", "bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b"]
512026-05-27T00:07:42.873ZINFOcrucible-agent (dropshot): request completed latency_us = 18793 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:35216 req_id = daf28fb9-3c6e-483f-960b-db2454fe132e response_code = 200 uri = /crucible/0/regions/00403839-ff09-499f-a317-70e7a964a050/snapshots/bb3cbf5d-6fc0-4fc2-88a4-1cd1a719dc3b/run