Max level shown:
12026-05-07T01:09:29.179ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:59975
22026-05-07T01:09:29.180ZINFOcrucible-agent (datafile): region 2ba9c7db-4cba-4be1-a83c-61583b50a542 state: Requested
32026-05-07T01:09:29.181ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-05-07T01:09:29.181ZINFOcrucible-agent (dropshot): request completed latency_us = 937 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:59975 req_id = 0dc9cafe-8371-4dd4-89b2-cda06716d069 response_code = 200 uri = /crucible/0/regions
52026-05-07T01:09:29.188ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_2b436a8c-f89b-4a1a-9685-2140cacc6192/crucible/regions/2ba9c7db-4cba-4be1-a83c-61583b50a542
62026-05-07T01:09:29.188ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_2b436a8c-f89b-4a1a-9685-2140cacc6192/crucible/regions/2ba9c7db-4cba-4be1-a83c-61583b50a542
72026-05-07T01:09:29.249ZINFOcrucible-agent (worker): creating region Region { id: RegionId("2ba9c7db-4cba-4be1-a83c-61583b50a542"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/2ba9c7db-4cba-4be1-a83c-61583b50a542" region = 2ba9c7db-4cba-4be1-a83c-61583b50a542
82026-05-07T01:09:29.249ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "2ba9c7db-4cba-4be1-a83c-61583b50a542" "--data" "/data/regions/2ba9c7db-4cba-4be1-a83c-61583b50a542" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 2ba9c7db-4cba-4be1-a83c-61583b50a542
92026-05-07T01:09:29.325ZINFOcrucible-agent (worker): region files created ok region = 2ba9c7db-4cba-4be1-a83c-61583b50a542
102026-05-07T01:09:29.325ZINFOcrucible-agent (datafile): region 2ba9c7db-4cba-4be1-a83c-61583b50a542 state: Requested -> Created
112026-05-07T01:09:29.325ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-05-07T01:09:29.325ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-2ba9c7db-4cba-4be1-a83c-61583b50a542
132026-05-07T01:09:29.335ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-2ba9c7db-4cba-4be1-a83c-61583b50a542
142026-05-07T01:09:29.357ZINFOcrucible-agent (worker): creating config property group
152026-05-07T01:09:29.361ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-2ba9c7db-4cba-4be1-a83c-61583b50a542
162026-05-07T01:09:29.361ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/2ba9c7db-4cba-4be1-a83c-61583b50a542
172026-05-07T01:09:29.361ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-05-07T01:09:29.361ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
192026-05-07T01:09:29.361ZINFOcrucible-agent (worker): commit
202026-05-07T01:09:29.365ZINFOcrucible-agent (worker): ok!
212026-05-07T01:09:29.380ZINFOcrucible-agent (worker): SMF ok!
222026-05-07T01:09:29.494ZINFOcrucible-agent (dropshot): request completed latency_us = 58 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:59975 req_id = aa8c5bd1-43c8-43de-b6ff-c294ac41e9df response_code = 200 uri = /crucible/0/regions
232026-05-07T01:09:31.581ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:59975 req_id = 6124aacc-bed6-451b-8b30-d60dc5be82e6 response_code = 200 uri = /crucible/0/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757
242026-05-07T01:09:31.586ZINFOcrucible-agent (datafile): path is "/data/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757"
252026-05-07T01:09:31.595ZINFOcrucible-agent (datafile): dataset is oxp_2b436a8c-f89b-4a1a-9685-2140cacc6192/crucible/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757
262026-05-07T01:09:31.601ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_2b436a8c-f89b-4a1a-9685-2140cacc6192/crucible/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757@05dc2b1a-27f0-400b-ac8b-abe73f6a3b29
272026-05-07T01:09:31.601ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_2b436a8c-f89b-4a1a-9685-2140cacc6192/crucible/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757", "05dc2b1a-27f0-400b-ac8b-abe73f6a3b29"]
282026-05-07T01:09:31.609ZINFOcrucible-agent (dropshot): request completed latency_us = 22376 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:59975 req_id = 39e9a3b5-4c83-499e-a727-3f6969cd6458 response_code = 200 uri = /crucible/0/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757/snapshots/05dc2b1a-27f0-400b-ac8b-abe73f6a3b29
292026-05-07T01:09:31.612ZINFOcrucible-agent (datafile): path is "/data/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757"
302026-05-07T01:09:31.618ZINFOcrucible-agent (datafile): dataset is oxp_2b436a8c-f89b-4a1a-9685-2140cacc6192/crucible/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757
312026-05-07T01:09:31.625ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_2b436a8c-f89b-4a1a-9685-2140cacc6192/crucible/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757@05dc2b1a-27f0-400b-ac8b-abe73f6a3b29
322026-05-07T01:09:31.625ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_2b436a8c-f89b-4a1a-9685-2140cacc6192/crucible/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757", "05dc2b1a-27f0-400b-ac8b-abe73f6a3b29"]
332026-05-07T01:09:31.635ZINFOcrucible-agent (datafile): requesting running snapshot 6afe1957-acb9-4f42-8e67-98b1e93cf757-05dc2b1a-27f0-400b-ac8b-abe73f6a3b29 state: Requested
342026-05-07T01:09:31.635ZINFOcrucible-agent (worker): applying SMF actions for region 6afe1957-acb9-4f42-8e67-98b1e93cf757 running snapshot 05dc2b1a-27f0-400b-ac8b-abe73f6a3b29 (state Requested)...
352026-05-07T01:09:31.635ZINFOcrucible-agent (dropshot): request completed latency_us = 22860 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:59975 req_id = ec583dc7-ead9-470a-aeb2-8e9ceefbc02d response_code = 200 uri = /crucible/0/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757/snapshots/05dc2b1a-27f0-400b-ac8b-abe73f6a3b29/run
362026-05-07T01:09:31.640ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-6afe1957-acb9-4f42-8e67-98b1e93cf757-05dc2b1a-27f0-400b-ac8b-abe73f6a3b29
372026-05-07T01:09:31.648ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-6afe1957-acb9-4f42-8e67-98b1e93cf757-05dc2b1a-27f0-400b-ac8b-abe73f6a3b29
382026-05-07T01:09:31.658ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-6afe1957-acb9-4f42-8e67-98b1e93cf757-05dc2b1a-27f0-400b-ac8b-abe73f6a3b29
392026-05-07T01:09:31.658ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757/.zfs/snapshot/05dc2b1a-27f0-400b-ac8b-abe73f6a3b29
402026-05-07T01:09:31.658ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-05-07T01:09:31.658ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-05-07T01:09:31.659ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
432026-05-07T01:09:31.659ZINFOcrucible-agent (worker): commit
442026-05-07T01:09:31.662ZINFOcrucible-agent (worker): ok!
452026-05-07T01:09:31.666ZINFOcrucible-agent (worker): SMF ok!
462026-05-07T01:09:31.666ZINFOcrucible-agent (datafile): region 6afe1957-acb9-4f42-8e67-98b1e93cf757 running snapshot 05dc2b1a-27f0-400b-ac8b-abe73f6a3b29 state: Requested -> Created
472026-05-07T01:09:31.866ZINFOcrucible-agent (datafile): path is "/data/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757"
482026-05-07T01:09:31.873ZINFOcrucible-agent (datafile): dataset is oxp_2b436a8c-f89b-4a1a-9685-2140cacc6192/crucible/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757
492026-05-07T01:09:31.881ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_2b436a8c-f89b-4a1a-9685-2140cacc6192/crucible/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757@05dc2b1a-27f0-400b-ac8b-abe73f6a3b29
502026-05-07T01:09:31.881ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_2b436a8c-f89b-4a1a-9685-2140cacc6192/crucible/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757", "05dc2b1a-27f0-400b-ac8b-abe73f6a3b29"]
512026-05-07T01:09:31.887ZINFOcrucible-agent (dropshot): request completed latency_us = 21420 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:59975 req_id = ca186e6f-33f5-4818-b81a-1fe4bc4c6dff response_code = 200 uri = /crucible/0/regions/6afe1957-acb9-4f42-8e67-98b1e93cf757/snapshots/05dc2b1a-27f0-400b-ac8b-abe73f6a3b29/run