Max level shown:
12026-03-10T19:17:33.337ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::a]:46016
22026-03-10T19:17:33.337ZINFOcrucible-agent (datafile): region a4acc60f-7c8a-4031-9a5a-b5bae2a74a9d state: Requested
32026-03-10T19:17:33.338ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-03-10T19:17:33.338ZINFOcrucible-agent (dropshot): request completed latency_us = 678 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:46016 req_id = 5029d147-14a2-433c-800f-7cf799937f96 response_code = 200 uri = /crucible/0/regions
52026-03-10T19:17:33.347ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_7ce71eac-5bcc-40ea-87e0-cb474c2cc418/crucible/regions/a4acc60f-7c8a-4031-9a5a-b5bae2a74a9d
62026-03-10T19:17:33.348ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_7ce71eac-5bcc-40ea-87e0-cb474c2cc418/crucible/regions/a4acc60f-7c8a-4031-9a5a-b5bae2a74a9d
72026-03-10T19:17:33.393ZINFOcrucible-agent (worker): creating region Region { id: RegionId("a4acc60f-7c8a-4031-9a5a-b5bae2a74a9d"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/a4acc60f-7c8a-4031-9a5a-b5bae2a74a9d" region = a4acc60f-7c8a-4031-9a5a-b5bae2a74a9d
82026-03-10T19:17:33.393ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "a4acc60f-7c8a-4031-9a5a-b5bae2a74a9d" "--data" "/data/regions/a4acc60f-7c8a-4031-9a5a-b5bae2a74a9d" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = a4acc60f-7c8a-4031-9a5a-b5bae2a74a9d
92026-03-10T19:17:33.491ZINFOcrucible-agent (worker): region files created ok region = a4acc60f-7c8a-4031-9a5a-b5bae2a74a9d
102026-03-10T19:17:33.491ZINFOcrucible-agent (datafile): region a4acc60f-7c8a-4031-9a5a-b5bae2a74a9d state: Requested -> Created
112026-03-10T19:17:33.491ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-03-10T19:17:33.492ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-a4acc60f-7c8a-4031-9a5a-b5bae2a74a9d
132026-03-10T19:17:33.504ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-a4acc60f-7c8a-4031-9a5a-b5bae2a74a9d
142026-03-10T19:17:33.534ZINFOcrucible-agent (worker): creating config property group
152026-03-10T19:17:33.537ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a4acc60f-7c8a-4031-9a5a-b5bae2a74a9d
162026-03-10T19:17:33.537ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a4acc60f-7c8a-4031-9a5a-b5bae2a74a9d
172026-03-10T19:17:33.537ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-03-10T19:17:33.537ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
192026-03-10T19:17:33.537ZINFOcrucible-agent (worker): commit
202026-03-10T19:17:33.541ZINFOcrucible-agent (worker): ok!
212026-03-10T19:17:33.557ZINFOcrucible-agent (worker): SMF ok!
222026-03-10T19:17:33.695ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:46016 req_id = a9f68a9d-b3b8-45b1-a571-c9c3c10246ff response_code = 200 uri = /crucible/0/regions
232026-03-10T19:17:34.441ZINFOcrucible-agent (dropshot): request completed latency_us = 77 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:46016 req_id = 5cacaf07-fca2-40f4-8c36-ecdbd3fed688 response_code = 200 uri = /crucible/0/regions/f429acb7-3577-4c57-a266-4f0a86d85046
242026-03-10T19:17:34.446ZINFOcrucible-agent (datafile): path is "/data/regions/f429acb7-3577-4c57-a266-4f0a86d85046"
252026-03-10T19:17:34.453ZINFOcrucible-agent (datafile): dataset is oxp_7ce71eac-5bcc-40ea-87e0-cb474c2cc418/crucible/regions/f429acb7-3577-4c57-a266-4f0a86d85046
262026-03-10T19:17:34.460ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_7ce71eac-5bcc-40ea-87e0-cb474c2cc418/crucible/regions/f429acb7-3577-4c57-a266-4f0a86d85046@f323e763-8baf-411c-938f-ae305748a3b2
272026-03-10T19:17:34.460ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_7ce71eac-5bcc-40ea-87e0-cb474c2cc418/crucible/regions/f429acb7-3577-4c57-a266-4f0a86d85046", "f323e763-8baf-411c-938f-ae305748a3b2"]
282026-03-10T19:17:34.468ZINFOcrucible-agent (dropshot): request completed latency_us = 22250 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:46016 req_id = 7c4c0e29-6ff1-43df-b5a1-3a2d6aab8cd1 response_code = 200 uri = /crucible/0/regions/f429acb7-3577-4c57-a266-4f0a86d85046/snapshots/f323e763-8baf-411c-938f-ae305748a3b2
292026-03-10T19:17:34.471ZINFOcrucible-agent (datafile): path is "/data/regions/f429acb7-3577-4c57-a266-4f0a86d85046"
302026-03-10T19:17:34.477ZINFOcrucible-agent (datafile): dataset is oxp_7ce71eac-5bcc-40ea-87e0-cb474c2cc418/crucible/regions/f429acb7-3577-4c57-a266-4f0a86d85046
312026-03-10T19:17:34.484ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_7ce71eac-5bcc-40ea-87e0-cb474c2cc418/crucible/regions/f429acb7-3577-4c57-a266-4f0a86d85046@f323e763-8baf-411c-938f-ae305748a3b2
322026-03-10T19:17:34.484ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_7ce71eac-5bcc-40ea-87e0-cb474c2cc418/crucible/regions/f429acb7-3577-4c57-a266-4f0a86d85046", "f323e763-8baf-411c-938f-ae305748a3b2"]
332026-03-10T19:17:34.490ZINFOcrucible-agent (datafile): requesting running snapshot f429acb7-3577-4c57-a266-4f0a86d85046-f323e763-8baf-411c-938f-ae305748a3b2 state: Requested
342026-03-10T19:17:34.491ZINFOcrucible-agent (worker): applying SMF actions for region f429acb7-3577-4c57-a266-4f0a86d85046 running snapshot f323e763-8baf-411c-938f-ae305748a3b2 (state Requested)...
352026-03-10T19:17:34.491ZINFOcrucible-agent (dropshot): request completed latency_us = 19836 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:46016 req_id = 1036bd20-d2d0-413e-a87b-dcea56d229ec response_code = 200 uri = /crucible/0/regions/f429acb7-3577-4c57-a266-4f0a86d85046/snapshots/f323e763-8baf-411c-938f-ae305748a3b2/run
362026-03-10T19:17:34.495ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-f429acb7-3577-4c57-a266-4f0a86d85046-f323e763-8baf-411c-938f-ae305748a3b2
372026-03-10T19:17:34.509ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-f429acb7-3577-4c57-a266-4f0a86d85046-f323e763-8baf-411c-938f-ae305748a3b2
382026-03-10T19:17:34.520ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-f429acb7-3577-4c57-a266-4f0a86d85046-f323e763-8baf-411c-938f-ae305748a3b2
392026-03-10T19:17:34.520ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/f429acb7-3577-4c57-a266-4f0a86d85046/.zfs/snapshot/f323e763-8baf-411c-938f-ae305748a3b2
402026-03-10T19:17:34.520ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-03-10T19:17:34.520ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-03-10T19:17:34.520ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
432026-03-10T19:17:34.520ZINFOcrucible-agent (worker): commit
442026-03-10T19:17:34.524ZINFOcrucible-agent (worker): ok!
452026-03-10T19:17:34.529ZINFOcrucible-agent (worker): SMF ok!
462026-03-10T19:17:34.529ZINFOcrucible-agent (datafile): region f429acb7-3577-4c57-a266-4f0a86d85046 running snapshot f323e763-8baf-411c-938f-ae305748a3b2 state: Requested -> Created
472026-03-10T19:17:34.851ZINFOcrucible-agent (datafile): path is "/data/regions/f429acb7-3577-4c57-a266-4f0a86d85046"
482026-03-10T19:17:34.857ZINFOcrucible-agent (datafile): dataset is oxp_7ce71eac-5bcc-40ea-87e0-cb474c2cc418/crucible/regions/f429acb7-3577-4c57-a266-4f0a86d85046
492026-03-10T19:17:34.863ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_7ce71eac-5bcc-40ea-87e0-cb474c2cc418/crucible/regions/f429acb7-3577-4c57-a266-4f0a86d85046@f323e763-8baf-411c-938f-ae305748a3b2
502026-03-10T19:17:34.863ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_7ce71eac-5bcc-40ea-87e0-cb474c2cc418/crucible/regions/f429acb7-3577-4c57-a266-4f0a86d85046", "f323e763-8baf-411c-938f-ae305748a3b2"]
512026-03-10T19:17:34.870ZINFOcrucible-agent (dropshot): request completed latency_us = 19033 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:46016 req_id = f7c7faa6-583a-45db-8014-08e7725982a4 response_code = 200 uri = /crucible/0/regions/f429acb7-3577-4c57-a266-4f0a86d85046/snapshots/f323e763-8baf-411c-938f-ae305748a3b2/run