Max level shown:
12026-08-28T15:19:02.261ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::c]:48541
22026-08-28T15:19:02.262ZINFOcrucible-agent (datafile): region 7c80194e-0b25-468c-abc7-af0c9936e3cb state: Requested
32026-08-28T15:19:02.262ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
42026-08-28T15:19:02.262ZINFOcrucible-agent (dropshot): request completed latency_us = 767 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:48541 req_id = 985302a0-1820-4ab4-b02b-e448b0dcb743 response_code = 200 uri = /crucible/0/regions
52026-08-28T15:19:02.271ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_ea9d8d7b-0453-4278-803d-4afaf68c20c5/crucible/regions/7c80194e-0b25-468c-abc7-af0c9936e3cb
62026-08-28T15:19:02.271ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_ea9d8d7b-0453-4278-803d-4afaf68c20c5/crucible/regions/7c80194e-0b25-468c-abc7-af0c9936e3cb
72026-08-28T15:19:02.319ZINFOcrucible-agent (worker): creating region Region { id: RegionId("7c80194e-0b25-468c-abc7-af0c9936e3cb"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/7c80194e-0b25-468c-abc7-af0c9936e3cb" region = 7c80194e-0b25-468c-abc7-af0c9936e3cb
82026-08-28T15:19:02.319ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "7c80194e-0b25-468c-abc7-af0c9936e3cb" "--data" "/data/regions/7c80194e-0b25-468c-abc7-af0c9936e3cb" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 7c80194e-0b25-468c-abc7-af0c9936e3cb
92026-08-28T15:19:02.402ZINFOcrucible-agent (worker): region files created ok region = 7c80194e-0b25-468c-abc7-af0c9936e3cb
102026-08-28T15:19:02.402ZINFOcrucible-agent (datafile): region 7c80194e-0b25-468c-abc7-af0c9936e3cb state: Requested -> Created
112026-08-28T15:19:02.402ZINFOcrucible-agent (worker): applying SMF actions post create...
122026-08-28T15:19:02.403ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-7c80194e-0b25-468c-abc7-af0c9936e3cb
132026-08-28T15:19:02.424ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-7c80194e-0b25-468c-abc7-af0c9936e3cb
142026-08-28T15:19:02.440ZINFOcrucible-agent (worker): creating config property group
152026-08-28T15:19:02.443ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-7c80194e-0b25-468c-abc7-af0c9936e3cb
162026-08-28T15:19:02.443ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/7c80194e-0b25-468c-abc7-af0c9936e3cb
172026-08-28T15:19:02.443ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
182026-08-28T15:19:02.443ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
192026-08-28T15:19:02.443ZINFOcrucible-agent (worker): commit
202026-08-28T15:19:02.447ZINFOcrucible-agent (worker): ok!
212026-08-28T15:19:02.452ZINFOcrucible-agent (worker): SMF ok!
222026-08-28T15:19:02.560ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:48541 req_id = ac314471-621b-4f95-b758-737e264914e7 response_code = 200 uri = /crucible/0/regions
232026-08-28T15:19:03.353ZINFOcrucible-agent (dropshot): request completed latency_us = 52 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:48541 req_id = 76bef7d5-d453-443d-aad7-bf89e39f050c response_code = 200 uri = /crucible/0/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e
242026-08-28T15:19:03.356ZINFOcrucible-agent (datafile): path is "/data/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e"
252026-08-28T15:19:03.363ZINFOcrucible-agent (datafile): dataset is oxp_ea9d8d7b-0453-4278-803d-4afaf68c20c5/crucible/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e
262026-08-28T15:19:03.370ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ea9d8d7b-0453-4278-803d-4afaf68c20c5/crucible/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e@6ae651ad-848b-4501-b595-21e4eb41d5d2
272026-08-28T15:19:03.370ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ea9d8d7b-0453-4278-803d-4afaf68c20c5/crucible/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e", "6ae651ad-848b-4501-b595-21e4eb41d5d2"]
282026-08-28T15:19:03.381ZINFOcrucible-agent (dropshot): request completed latency_us = 24610 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:48541 req_id = 89a02807-102a-4e3f-adba-896bc0acce47 response_code = 200 uri = /crucible/0/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e/snapshots/6ae651ad-848b-4501-b595-21e4eb41d5d2
292026-08-28T15:19:03.389ZINFOcrucible-agent (datafile): path is "/data/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e"
302026-08-28T15:19:03.396ZINFOcrucible-agent (datafile): dataset is oxp_ea9d8d7b-0453-4278-803d-4afaf68c20c5/crucible/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e
312026-08-28T15:19:03.401ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ea9d8d7b-0453-4278-803d-4afaf68c20c5/crucible/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e@6ae651ad-848b-4501-b595-21e4eb41d5d2
322026-08-28T15:19:03.401ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ea9d8d7b-0453-4278-803d-4afaf68c20c5/crucible/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e", "6ae651ad-848b-4501-b595-21e4eb41d5d2"]
332026-08-28T15:19:03.409ZINFOcrucible-agent (datafile): requesting running snapshot e0737511-9c20-4c3e-9dd0-1cb3bfb6072e-6ae651ad-848b-4501-b595-21e4eb41d5d2 state: Requested
342026-08-28T15:19:03.414ZINFOcrucible-agent (worker): applying SMF actions for region e0737511-9c20-4c3e-9dd0-1cb3bfb6072e running snapshot 6ae651ad-848b-4501-b595-21e4eb41d5d2 (state Requested)...
352026-08-28T15:19:03.414ZINFOcrucible-agent (dropshot): request completed latency_us = 24594 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:48541 req_id = 92ed2d66-3cd6-4d40-85d8-654bb8f9d6cc response_code = 200 uri = /crucible/0/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e/snapshots/6ae651ad-848b-4501-b595-21e4eb41d5d2/run
362026-08-28T15:19:03.416ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-e0737511-9c20-4c3e-9dd0-1cb3bfb6072e-6ae651ad-848b-4501-b595-21e4eb41d5d2
372026-08-28T15:19:03.419ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-e0737511-9c20-4c3e-9dd0-1cb3bfb6072e-6ae651ad-848b-4501-b595-21e4eb41d5d2
382026-08-28T15:19:03.428ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-e0737511-9c20-4c3e-9dd0-1cb3bfb6072e-6ae651ad-848b-4501-b595-21e4eb41d5d2
392026-08-28T15:19:03.428ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e/.zfs/snapshot/6ae651ad-848b-4501-b595-21e4eb41d5d2
402026-08-28T15:19:03.428ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
412026-08-28T15:19:03.428ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
422026-08-28T15:19:03.429ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
432026-08-28T15:19:03.429ZINFOcrucible-agent (worker): commit
442026-08-28T15:19:03.432ZINFOcrucible-agent (worker): ok!
452026-08-28T15:19:03.437ZINFOcrucible-agent (worker): SMF ok!
462026-08-28T15:19:03.437ZINFOcrucible-agent (datafile): region e0737511-9c20-4c3e-9dd0-1cb3bfb6072e running snapshot 6ae651ad-848b-4501-b595-21e4eb41d5d2 state: Requested -> Created
472026-08-28T15:19:03.689ZINFOcrucible-agent (datafile): path is "/data/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e"
482026-08-28T15:19:03.695ZINFOcrucible-agent (datafile): dataset is oxp_ea9d8d7b-0453-4278-803d-4afaf68c20c5/crucible/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e
492026-08-28T15:19:03.700ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_ea9d8d7b-0453-4278-803d-4afaf68c20c5/crucible/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e@6ae651ad-848b-4501-b595-21e4eb41d5d2
502026-08-28T15:19:03.700ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_ea9d8d7b-0453-4278-803d-4afaf68c20c5/crucible/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e", "6ae651ad-848b-4501-b595-21e4eb41d5d2"]
512026-08-28T15:19:03.707ZINFOcrucible-agent (dropshot): request completed latency_us = 17579 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:48541 req_id = 3850975b-8667-4edc-9b6b-83d27535a387 response_code = 200 uri = /crucible/0/regions/e0737511-9c20-4c3e-9dd0-1cb3bfb6072e/snapshots/6ae651ad-848b-4501-b595-21e4eb41d5d2/run