Max level shown:
12025-12-24T00:30:45.373ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::c]:63832
22025-12-24T00:30:45.400ZINFOcrucible-agent (datafile): region a4834b7e-2bfe-4f38-85c9-1c2e01e132b7 state: Requested
32025-12-24T00:30:45.403ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42025-12-24T00:30:45.403ZINFOcrucible-agent (dropshot): request completed latency_us = 14164 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63832 req_id = d578e6c1-c4ba-4d7f-97bb-4bcc4720f162 response_code = 200 uri = /crucible/0/regions
52025-12-24T00:30:45.414ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_6c840405-0e0d-4e68-b9ce-2541c8e9c372/crucible/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7
62025-12-24T00:30:45.414ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_6c840405-0e0d-4e68-b9ce-2541c8e9c372/crucible/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7
72025-12-24T00:30:45.492ZINFOcrucible-agent (worker): creating region Region { id: RegionId("a4834b7e-2bfe-4f38-85c9-1c2e01e132b7"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7" region = a4834b7e-2bfe-4f38-85c9-1c2e01e132b7
82025-12-24T00:30:45.494ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "a4834b7e-2bfe-4f38-85c9-1c2e01e132b7" "--data" "/data/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = a4834b7e-2bfe-4f38-85c9-1c2e01e132b7
92025-12-24T00:30:45.616ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63832 req_id = 1b51a437-6208-47a5-990f-3575bc1aded4 response_code = 200 uri = /crucible/0/regions
102025-12-24T00:30:45.880ZINFOcrucible-agent (worker): region files created ok region = a4834b7e-2bfe-4f38-85c9-1c2e01e132b7
112025-12-24T00:30:45.880ZINFOcrucible-agent (datafile): region a4834b7e-2bfe-4f38-85c9-1c2e01e132b7 state: Requested -> Created
122025-12-24T00:30:45.880ZINFOcrucible-agent (worker): applying SMF actions post create...
132025-12-24T00:30:45.880ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-a4834b7e-2bfe-4f38-85c9-1c2e01e132b7
142025-12-24T00:30:45.886ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-a4834b7e-2bfe-4f38-85c9-1c2e01e132b7
152025-12-24T00:30:45.903ZINFOcrucible-agent (worker): creating config property group
162025-12-24T00:30:45.906ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-a4834b7e-2bfe-4f38-85c9-1c2e01e132b7
172025-12-24T00:30:45.907ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7
182025-12-24T00:30:45.907ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
192025-12-24T00:30:45.907ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
202025-12-24T00:30:45.907ZINFOcrucible-agent (worker): commit
212025-12-24T00:30:45.911ZINFOcrucible-agent (worker): ok!
222025-12-24T00:30:45.915ZINFOcrucible-agent (worker): SMF ok!
232025-12-24T00:30:45.986ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:63832 req_id = c411441b-832c-4648-86ad-538b202eb967 response_code = 200 uri = /crucible/0/regions
242025-12-24T00:41:39.118ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::16]:32345 remote_addr = [fd00:1122:3344:101::c]:38796
252025-12-24T00:41:39.119ZINFOcrucible-agent (dropshot): request completed latency_us = 53 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:38796 req_id = d60d9bfa-518e-49b4-bccd-4c1441b4b9b4 response_code = 200 uri = /crucible/0/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7
262025-12-24T00:41:39.122ZINFOcrucible-agent (datafile): path is "/data/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7"
272025-12-24T00:41:39.128ZINFOcrucible-agent (datafile): dataset is oxp_6c840405-0e0d-4e68-b9ce-2541c8e9c372/crucible/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7
282025-12-24T00:41:39.135ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_6c840405-0e0d-4e68-b9ce-2541c8e9c372/crucible/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7@476c70b1-54cf-43f2-9d2a-064da4fde985
292025-12-24T00:41:39.135ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_6c840405-0e0d-4e68-b9ce-2541c8e9c372/crucible/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7", "476c70b1-54cf-43f2-9d2a-064da4fde985"]
302025-12-24T00:41:39.145ZINFOcrucible-agent (dropshot): request completed latency_us = 22835 local_addr = [fd00:1122:3344:101::16]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:38796 req_id = eb577f7d-0735-4a2e-ba8a-c40eca79ac3e response_code = 200 uri = /crucible/0/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7/snapshots/476c70b1-54cf-43f2-9d2a-064da4fde985
312025-12-24T00:41:39.148ZINFOcrucible-agent (datafile): path is "/data/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7"
322025-12-24T00:41:39.155ZINFOcrucible-agent (datafile): dataset is oxp_6c840405-0e0d-4e68-b9ce-2541c8e9c372/crucible/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7
332025-12-24T00:41:39.161ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_6c840405-0e0d-4e68-b9ce-2541c8e9c372/crucible/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7@476c70b1-54cf-43f2-9d2a-064da4fde985
342025-12-24T00:41:39.161ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_6c840405-0e0d-4e68-b9ce-2541c8e9c372/crucible/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7", "476c70b1-54cf-43f2-9d2a-064da4fde985"]
352025-12-24T00:41:39.169ZINFOcrucible-agent (datafile): requesting running snapshot a4834b7e-2bfe-4f38-85c9-1c2e01e132b7-476c70b1-54cf-43f2-9d2a-064da4fde985 state: Requested
362025-12-24T00:41:39.169ZINFOcrucible-agent (dropshot): request completed latency_us = 21145 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:38796 req_id = 69d22613-6c69-4610-95b4-b1a62c41c7d1 response_code = 200 uri = /crucible/0/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7/snapshots/476c70b1-54cf-43f2-9d2a-064da4fde985/run
372025-12-24T00:41:39.170ZINFOcrucible-agent (worker): applying SMF actions for region a4834b7e-2bfe-4f38-85c9-1c2e01e132b7 running snapshot 476c70b1-54cf-43f2-9d2a-064da4fde985 (state Requested)...
382025-12-24T00:41:39.191ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-a4834b7e-2bfe-4f38-85c9-1c2e01e132b7-476c70b1-54cf-43f2-9d2a-064da4fde985
392025-12-24T00:41:39.202ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-a4834b7e-2bfe-4f38-85c9-1c2e01e132b7-476c70b1-54cf-43f2-9d2a-064da4fde985
402025-12-24T00:41:39.214ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-a4834b7e-2bfe-4f38-85c9-1c2e01e132b7-476c70b1-54cf-43f2-9d2a-064da4fde985
412025-12-24T00:41:39.215ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7/.zfs/snapshot/476c70b1-54cf-43f2-9d2a-064da4fde985
422025-12-24T00:41:39.215ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
432025-12-24T00:41:39.215ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
442025-12-24T00:41:39.215ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::16
452025-12-24T00:41:39.215ZINFOcrucible-agent (worker): commit
462025-12-24T00:41:39.218ZINFOcrucible-agent (worker): ok!
472025-12-24T00:41:39.223ZINFOcrucible-agent (worker): SMF ok!
482025-12-24T00:41:39.223ZINFOcrucible-agent (datafile): region a4834b7e-2bfe-4f38-85c9-1c2e01e132b7 running snapshot 476c70b1-54cf-43f2-9d2a-064da4fde985 state: Requested -> Created
492025-12-24T00:41:39.356ZINFOcrucible-agent (datafile): path is "/data/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7"
502025-12-24T00:41:39.363ZINFOcrucible-agent (datafile): dataset is oxp_6c840405-0e0d-4e68-b9ce-2541c8e9c372/crucible/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7
512025-12-24T00:41:39.369ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_6c840405-0e0d-4e68-b9ce-2541c8e9c372/crucible/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7@476c70b1-54cf-43f2-9d2a-064da4fde985
522025-12-24T00:41:39.369ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_6c840405-0e0d-4e68-b9ce-2541c8e9c372/crucible/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7", "476c70b1-54cf-43f2-9d2a-064da4fde985"]
532025-12-24T00:41:39.376ZINFOcrucible-agent (dropshot): request completed latency_us = 19224 local_addr = [fd00:1122:3344:101::16]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:38796 req_id = 0aed7ac2-3136-4eb2-9242-a959f2c93df6 response_code = 200 uri = /crucible/0/regions/a4834b7e-2bfe-4f38-85c9-1c2e01e132b7/snapshots/476c70b1-54cf-43f2-9d2a-064da4fde985/run