Max level shown:
12026-02-03T00:46:04.104ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::15]:32345 remote_addr = [fd00:1122:3344:101::b]:51601
22026-02-03T00:46:04.104ZINFOcrucible-agent (datafile): region 7988560f-0bf0-462d-9747-b33867c2e97f state: Requested
32026-02-03T00:46:04.105ZINFOcrucible-agent (dropshot): request completed latency_us = 691 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:51601 req_id = 1fd13696-87ed-45d2-8f2b-9d8093612ee2 response_code = 200 uri = /crucible/0/regions
42026-02-03T00:46:04.105ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
52026-02-03T00:46:04.114ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_b67d20c2-4625-46d9-9718-418d8e03e712/crucible/regions/7988560f-0bf0-462d-9747-b33867c2e97f
62026-02-03T00:46:04.114ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_b67d20c2-4625-46d9-9718-418d8e03e712/crucible/regions/7988560f-0bf0-462d-9747-b33867c2e97f
72026-02-03T00:46:04.185ZINFOcrucible-agent (worker): creating region Region { id: RegionId("7988560f-0bf0-462d-9747-b33867c2e97f"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/7988560f-0bf0-462d-9747-b33867c2e97f" region = 7988560f-0bf0-462d-9747-b33867c2e97f
82026-02-03T00:46:04.185ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "7988560f-0bf0-462d-9747-b33867c2e97f" "--data" "/data/regions/7988560f-0bf0-462d-9747-b33867c2e97f" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 7988560f-0bf0-462d-9747-b33867c2e97f
92026-02-03T00:46:04.272ZINFOcrucible-agent (worker): region files created ok region = 7988560f-0bf0-462d-9747-b33867c2e97f
102026-02-03T00:46:04.272ZINFOcrucible-agent (datafile): region 7988560f-0bf0-462d-9747-b33867c2e97f state: Requested -> Created
112026-02-03T00:46:04.273ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-02-03T00:46:04.293ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-7988560f-0bf0-462d-9747-b33867c2e97f
132026-02-03T00:46:04.300ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-7988560f-0bf0-462d-9747-b33867c2e97f
142026-02-03T00:46:04.312ZINFOcrucible-agent (worker): creating config property group
152026-02-03T00:46:04.317ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-7988560f-0bf0-462d-9747-b33867c2e97f
162026-02-03T00:46:04.317ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/7988560f-0bf0-462d-9747-b33867c2e97f
172026-02-03T00:46:04.317ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-02-03T00:46:04.317ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
192026-02-03T00:46:04.317ZINFOcrucible-agent (worker): commit
202026-02-03T00:46:04.323ZINFOcrucible-agent (worker): ok!
212026-02-03T00:46:04.331ZINFOcrucible-agent (worker): SMF ok!
222026-02-03T00:46:04.414ZINFOcrucible-agent (dropshot): request completed latency_us = 56 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:51601 req_id = 3988e8f1-c79b-4b8a-8630-a84afbc3425c response_code = 200 uri = /crucible/0/regions
232026-02-03T00:46:05.173ZINFOcrucible-agent (dropshot): request completed latency_us = 66 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:51601 req_id = 683579f2-8d37-4e13-b58c-5fa4bdbae700 response_code = 200 uri = /crucible/0/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd
242026-02-03T00:46:05.176ZINFOcrucible-agent (datafile): path is "/data/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd"
252026-02-03T00:46:05.182ZINFOcrucible-agent (datafile): dataset is oxp_b67d20c2-4625-46d9-9718-418d8e03e712/crucible/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd
262026-02-03T00:46:05.189ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b67d20c2-4625-46d9-9718-418d8e03e712/crucible/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd@0d17ac9e-fef3-4883-b302-e3b15e1ba84d
272026-02-03T00:46:05.189ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b67d20c2-4625-46d9-9718-418d8e03e712/crucible/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd", "0d17ac9e-fef3-4883-b302-e3b15e1ba84d"]
282026-02-03T00:46:05.198ZINFOcrucible-agent (dropshot): request completed latency_us = 21527 local_addr = [fd00:1122:3344:101::15]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:51601 req_id = f7ab99fd-3e21-44b6-95e4-5f8d3ae478f3 response_code = 200 uri = /crucible/0/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd/snapshots/0d17ac9e-fef3-4883-b302-e3b15e1ba84d
292026-02-03T00:46:05.201ZINFOcrucible-agent (datafile): path is "/data/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd"
302026-02-03T00:46:05.207ZINFOcrucible-agent (datafile): dataset is oxp_b67d20c2-4625-46d9-9718-418d8e03e712/crucible/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd
312026-02-03T00:46:05.213ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b67d20c2-4625-46d9-9718-418d8e03e712/crucible/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd@0d17ac9e-fef3-4883-b302-e3b15e1ba84d
322026-02-03T00:46:05.213ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b67d20c2-4625-46d9-9718-418d8e03e712/crucible/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd", "0d17ac9e-fef3-4883-b302-e3b15e1ba84d"]
332026-02-03T00:46:05.221ZINFOcrucible-agent (datafile): requesting running snapshot 33feb617-2507-416a-b7d9-db5b4ee99ebd-0d17ac9e-fef3-4883-b302-e3b15e1ba84d state: Requested
342026-02-03T00:46:05.222ZINFOcrucible-agent (dropshot): request completed latency_us = 20265 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:51601 req_id = ab3a027c-89e1-405b-8905-9bab97790a23 response_code = 200 uri = /crucible/0/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd/snapshots/0d17ac9e-fef3-4883-b302-e3b15e1ba84d/run
352026-02-03T00:46:05.222ZINFOcrucible-agent (worker): applying SMF actions for region 33feb617-2507-416a-b7d9-db5b4ee99ebd running snapshot 0d17ac9e-fef3-4883-b302-e3b15e1ba84d (state Requested)...
362026-02-03T00:46:05.226ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-33feb617-2507-416a-b7d9-db5b4ee99ebd-0d17ac9e-fef3-4883-b302-e3b15e1ba84d
372026-02-03T00:46:05.238ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-33feb617-2507-416a-b7d9-db5b4ee99ebd-0d17ac9e-fef3-4883-b302-e3b15e1ba84d
382026-02-03T00:46:05.249ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-33feb617-2507-416a-b7d9-db5b4ee99ebd-0d17ac9e-fef3-4883-b302-e3b15e1ba84d
392026-02-03T00:46:05.249ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd/.zfs/snapshot/0d17ac9e-fef3-4883-b302-e3b15e1ba84d
402026-02-03T00:46:05.249ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-02-03T00:46:05.250ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-02-03T00:46:05.250ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::15
432026-02-03T00:46:05.250ZINFOcrucible-agent (worker): commit
442026-02-03T00:46:05.254ZINFOcrucible-agent (worker): ok!
452026-02-03T00:46:05.259ZINFOcrucible-agent (worker): SMF ok!
462026-02-03T00:46:05.259ZINFOcrucible-agent (datafile): region 33feb617-2507-416a-b7d9-db5b4ee99ebd running snapshot 0d17ac9e-fef3-4883-b302-e3b15e1ba84d state: Requested -> Created
472026-02-03T00:46:05.352ZINFOcrucible-agent (datafile): path is "/data/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd"
482026-02-03T00:46:05.359ZINFOcrucible-agent (datafile): dataset is oxp_b67d20c2-4625-46d9-9718-418d8e03e712/crucible/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd
492026-02-03T00:46:05.365ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_b67d20c2-4625-46d9-9718-418d8e03e712/crucible/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd@0d17ac9e-fef3-4883-b302-e3b15e1ba84d
502026-02-03T00:46:05.365ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_b67d20c2-4625-46d9-9718-418d8e03e712/crucible/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd", "0d17ac9e-fef3-4883-b302-e3b15e1ba84d"]
512026-02-03T00:46:05.370ZINFOcrucible-agent (dropshot): request completed latency_us = 17719 local_addr = [fd00:1122:3344:101::15]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:51601 req_id = 28535e27-d9dc-4023-8ca9-8ea313a6cc83 response_code = 200 uri = /crucible/0/regions/33feb617-2507-416a-b7d9-db5b4ee99ebd/snapshots/0d17ac9e-fef3-4883-b302-e3b15e1ba84d/run