Max level shown:
12026-04-29T22:30:25.828ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::1a]:32345 remote_addr = [fd00:1122:3344:101::b]:37601
22026-04-29T22:30:25.828ZINFOcrucible-agent (datafile): region fe4d88c1-da94-4d2e-961b-0bf744d72e49 state: Requested
32026-04-29T22:30:25.829ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-04-29T22:30:25.829ZINFOcrucible-agent (dropshot): request completed latency_us = 720 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:37601 req_id = 9b207997-244b-4677-8650-11b84200d2ab response_code = 200 uri = /crucible/0/regions
52026-04-29T22:30:25.836ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_accb6ad4-7c1e-430a-b946-3934998c554f/crucible/regions/fe4d88c1-da94-4d2e-961b-0bf744d72e49
62026-04-29T22:30:25.836ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_accb6ad4-7c1e-430a-b946-3934998c554f/crucible/regions/fe4d88c1-da94-4d2e-961b-0bf744d72e49
72026-04-29T22:30:25.888ZINFOcrucible-agent (worker): creating region Region { id: RegionId("fe4d88c1-da94-4d2e-961b-0bf744d72e49"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/fe4d88c1-da94-4d2e-961b-0bf744d72e49" region = fe4d88c1-da94-4d2e-961b-0bf744d72e49
82026-04-29T22:30:25.888ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "fe4d88c1-da94-4d2e-961b-0bf744d72e49" "--data" "/data/regions/fe4d88c1-da94-4d2e-961b-0bf744d72e49" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = fe4d88c1-da94-4d2e-961b-0bf744d72e49
92026-04-29T22:30:25.962ZINFOcrucible-agent (worker): region files created ok region = fe4d88c1-da94-4d2e-961b-0bf744d72e49
102026-04-29T22:30:25.962ZINFOcrucible-agent (datafile): region fe4d88c1-da94-4d2e-961b-0bf744d72e49 state: Requested -> Created
112026-04-29T22:30:25.962ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-04-29T22:30:25.982ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-fe4d88c1-da94-4d2e-961b-0bf744d72e49
132026-04-29T22:30:25.985ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-fe4d88c1-da94-4d2e-961b-0bf744d72e49
142026-04-29T22:30:25.996ZINFOcrucible-agent (worker): creating config property group
152026-04-29T22:30:25.999ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-fe4d88c1-da94-4d2e-961b-0bf744d72e49
162026-04-29T22:30:25.999ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/fe4d88c1-da94-4d2e-961b-0bf744d72e49
172026-04-29T22:30:25.999ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-04-29T22:30:25.999ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
192026-04-29T22:30:25.999ZINFOcrucible-agent (worker): commit
202026-04-29T22:30:26.004ZINFOcrucible-agent (worker): ok!
212026-04-29T22:30:26.008ZINFOcrucible-agent (worker): SMF ok!
222026-04-29T22:30:26.169ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:37601 req_id = a5ec479e-a1ec-4bf3-9b77-caf5a59f728d response_code = 200 uri = /crucible/0/regions
232026-04-29T22:30:27.019ZINFOcrucible-agent (dropshot): request completed latency_us = 70 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:37601 req_id = 66d3b5b8-148c-473a-860f-c4e3cf198d7c response_code = 200 uri = /crucible/0/regions/d5972552-b86b-43dd-8061-d3780bbfaf49
242026-04-29T22:30:27.023ZINFOcrucible-agent (datafile): path is "/data/regions/d5972552-b86b-43dd-8061-d3780bbfaf49"
252026-04-29T22:30:27.029ZINFOcrucible-agent (datafile): dataset is oxp_accb6ad4-7c1e-430a-b946-3934998c554f/crucible/regions/d5972552-b86b-43dd-8061-d3780bbfaf49
262026-04-29T22:30:27.035ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_accb6ad4-7c1e-430a-b946-3934998c554f/crucible/regions/d5972552-b86b-43dd-8061-d3780bbfaf49@c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf
272026-04-29T22:30:27.035ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_accb6ad4-7c1e-430a-b946-3934998c554f/crucible/regions/d5972552-b86b-43dd-8061-d3780bbfaf49", "c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf"]
282026-04-29T22:30:27.044ZINFOcrucible-agent (dropshot): request completed latency_us = 21044 local_addr = [fd00:1122:3344:101::1a]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:37601 req_id = 4c79dd82-62a1-4dd7-9d50-9344d8ea27b3 response_code = 200 uri = /crucible/0/regions/d5972552-b86b-43dd-8061-d3780bbfaf49/snapshots/c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf
292026-04-29T22:30:27.048ZINFOcrucible-agent (datafile): path is "/data/regions/d5972552-b86b-43dd-8061-d3780bbfaf49"
302026-04-29T22:30:27.053ZINFOcrucible-agent (datafile): dataset is oxp_accb6ad4-7c1e-430a-b946-3934998c554f/crucible/regions/d5972552-b86b-43dd-8061-d3780bbfaf49
312026-04-29T22:30:27.061ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_accb6ad4-7c1e-430a-b946-3934998c554f/crucible/regions/d5972552-b86b-43dd-8061-d3780bbfaf49@c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf
322026-04-29T22:30:27.061ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_accb6ad4-7c1e-430a-b946-3934998c554f/crucible/regions/d5972552-b86b-43dd-8061-d3780bbfaf49", "c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf"]
332026-04-29T22:30:27.068ZINFOcrucible-agent (datafile): requesting running snapshot d5972552-b86b-43dd-8061-d3780bbfaf49-c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf state: Requested
342026-04-29T22:30:27.068ZINFOcrucible-agent (dropshot): request completed latency_us = 20190 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:37601 req_id = 06fcd87c-bdaf-43a6-8157-cb5d713c1617 response_code = 200 uri = /crucible/0/regions/d5972552-b86b-43dd-8061-d3780bbfaf49/snapshots/c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf/run
352026-04-29T22:30:27.068ZINFOcrucible-agent (worker): applying SMF actions for region d5972552-b86b-43dd-8061-d3780bbfaf49 running snapshot c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf (state Requested)...
362026-04-29T22:30:27.072ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-d5972552-b86b-43dd-8061-d3780bbfaf49-c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf
372026-04-29T22:30:27.081ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-d5972552-b86b-43dd-8061-d3780bbfaf49-c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf
382026-04-29T22:30:27.090ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-d5972552-b86b-43dd-8061-d3780bbfaf49-c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf
392026-04-29T22:30:27.090ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d5972552-b86b-43dd-8061-d3780bbfaf49/.zfs/snapshot/c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf
402026-04-29T22:30:27.090ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-04-29T22:30:27.090ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-04-29T22:30:27.090ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::1a
432026-04-29T22:30:27.090ZINFOcrucible-agent (worker): commit
442026-04-29T22:30:27.093ZINFOcrucible-agent (worker): ok!
452026-04-29T22:30:27.097ZINFOcrucible-agent (worker): SMF ok!
462026-04-29T22:30:27.097ZINFOcrucible-agent (datafile): region d5972552-b86b-43dd-8061-d3780bbfaf49 running snapshot c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf state: Requested -> Created
472026-04-29T22:30:27.354ZINFOcrucible-agent (datafile): path is "/data/regions/d5972552-b86b-43dd-8061-d3780bbfaf49"
482026-04-29T22:30:27.361ZINFOcrucible-agent (datafile): dataset is oxp_accb6ad4-7c1e-430a-b946-3934998c554f/crucible/regions/d5972552-b86b-43dd-8061-d3780bbfaf49
492026-04-29T22:30:27.367ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_accb6ad4-7c1e-430a-b946-3934998c554f/crucible/regions/d5972552-b86b-43dd-8061-d3780bbfaf49@c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf
502026-04-29T22:30:27.367ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_accb6ad4-7c1e-430a-b946-3934998c554f/crucible/regions/d5972552-b86b-43dd-8061-d3780bbfaf49", "c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf"]
512026-04-29T22:30:27.374ZINFOcrucible-agent (dropshot): request completed latency_us = 20101 local_addr = [fd00:1122:3344:101::1a]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:37601 req_id = d563b162-8609-4c31-a81f-0eb9e1efbf7b response_code = 200 uri = /crucible/0/regions/d5972552-b86b-43dd-8061-d3780bbfaf49/snapshots/c9ed9b9f-1a12-400c-b29c-fc97ac6e53bf/run