Max level shown:
12025-01-09T20:50:55.082ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::c]:39713
22025-01-09T20:50:55.085ZINFOcrucible-agent (dropshot): request completed latency_us = 3172 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:39713 req_id = 77912901-6afc-4d00-9333-1bf083a9cdc1 response_code = 200 uri = /crucible/0/regions/525eebcd-a152-475a-a3ad-23446c9aa46b
32025-01-09T20:50:55.089ZINFOcrucible-agent (datafile): path is "/data/regions/525eebcd-a152-475a-a3ad-23446c9aa46b"
42025-01-09T20:50:55.098ZINFOcrucible-agent (datafile): dataset is oxp_7ad58e80-8c74-4773-9b4c-10e6bb74a74f/crucible/regions/525eebcd-a152-475a-a3ad-23446c9aa46b
52025-01-09T20:50:55.107ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_7ad58e80-8c74-4773-9b4c-10e6bb74a74f/crucible/regions/525eebcd-a152-475a-a3ad-23446c9aa46b@767abb4d-61bf-47f3-b595-e1ac8d322b3e
62025-01-09T20:50:55.107ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_7ad58e80-8c74-4773-9b4c-10e6bb74a74f/crucible/regions/525eebcd-a152-475a-a3ad-23446c9aa46b", "767abb4d-61bf-47f3-b595-e1ac8d322b3e"]
72025-01-09T20:50:55.117ZINFOcrucible-agent (dropshot): request completed latency_us = 28371 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:39713 req_id = 80973832-8f56-49ba-8bdd-21b092e407b9 response_code = 200 uri = /crucible/0/regions/525eebcd-a152-475a-a3ad-23446c9aa46b/snapshots/767abb4d-61bf-47f3-b595-e1ac8d322b3e
82025-01-09T20:50:55.121ZINFOcrucible-agent (datafile): path is "/data/regions/525eebcd-a152-475a-a3ad-23446c9aa46b"
92025-01-09T20:50:55.128ZINFOcrucible-agent (datafile): dataset is oxp_7ad58e80-8c74-4773-9b4c-10e6bb74a74f/crucible/regions/525eebcd-a152-475a-a3ad-23446c9aa46b
102025-01-09T20:50:55.136ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_7ad58e80-8c74-4773-9b4c-10e6bb74a74f/crucible/regions/525eebcd-a152-475a-a3ad-23446c9aa46b@767abb4d-61bf-47f3-b595-e1ac8d322b3e
112025-01-09T20:50:55.136ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_7ad58e80-8c74-4773-9b4c-10e6bb74a74f/crucible/regions/525eebcd-a152-475a-a3ad-23446c9aa46b", "767abb4d-61bf-47f3-b595-e1ac8d322b3e"]
122025-01-09T20:50:55.146ZINFOcrucible-agent (datafile): requesting running snapshot 525eebcd-a152-475a-a3ad-23446c9aa46b-767abb4d-61bf-47f3-b595-e1ac8d322b3e state: Requested
132025-01-09T20:50:55.147ZINFOcrucible-agent (worker): applying SMF actions for region 525eebcd-a152-475a-a3ad-23446c9aa46b running snapshot 767abb4d-61bf-47f3-b595-e1ac8d322b3e (state Requested)...
142025-01-09T20:50:55.147ZINFOcrucible-agent (dropshot): request completed latency_us = 26057 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:39713 req_id = ca99dd42-6e0f-4895-a701-a50960abe5a0 response_code = 200 uri = /crucible/0/regions/525eebcd-a152-475a-a3ad-23446c9aa46b/snapshots/767abb4d-61bf-47f3-b595-e1ac8d322b3e/run
152025-01-09T20:50:55.156ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-525eebcd-a152-475a-a3ad-23446c9aa46b-767abb4d-61bf-47f3-b595-e1ac8d322b3e
162025-01-09T20:50:55.159ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-525eebcd-a152-475a-a3ad-23446c9aa46b-767abb4d-61bf-47f3-b595-e1ac8d322b3e
172025-01-09T20:50:55.170ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-525eebcd-a152-475a-a3ad-23446c9aa46b-767abb4d-61bf-47f3-b595-e1ac8d322b3e
182025-01-09T20:50:55.171ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/525eebcd-a152-475a-a3ad-23446c9aa46b/.zfs/snapshot/767abb4d-61bf-47f3-b595-e1ac8d322b3e
192025-01-09T20:50:55.171ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
202025-01-09T20:50:55.171ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
212025-01-09T20:50:55.171ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
222025-01-09T20:50:55.171ZINFOcrucible-agent (worker): commit
232025-01-09T20:50:55.175ZINFOcrucible-agent (worker): ok!
242025-01-09T20:50:55.180ZINFOcrucible-agent (worker): SMF ok!
252025-01-09T20:50:55.180ZINFOcrucible-agent (datafile): region 525eebcd-a152-475a-a3ad-23446c9aa46b running snapshot 767abb4d-61bf-47f3-b595-e1ac8d322b3e state: Requested -> Created
262025-01-09T20:50:55.524ZINFOcrucible-agent (datafile): path is "/data/regions/525eebcd-a152-475a-a3ad-23446c9aa46b"
272025-01-09T20:50:55.532ZINFOcrucible-agent (datafile): dataset is oxp_7ad58e80-8c74-4773-9b4c-10e6bb74a74f/crucible/regions/525eebcd-a152-475a-a3ad-23446c9aa46b
282025-01-09T20:50:55.540ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_7ad58e80-8c74-4773-9b4c-10e6bb74a74f/crucible/regions/525eebcd-a152-475a-a3ad-23446c9aa46b@767abb4d-61bf-47f3-b595-e1ac8d322b3e
292025-01-09T20:50:55.540ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_7ad58e80-8c74-4773-9b4c-10e6bb74a74f/crucible/regions/525eebcd-a152-475a-a3ad-23446c9aa46b", "767abb4d-61bf-47f3-b595-e1ac8d322b3e"]
302025-01-09T20:50:55.548ZINFOcrucible-agent (dropshot): request completed latency_us = 24463 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:39713 req_id = 275847f9-6216-435b-bda2-f101307d28f8 response_code = 200 uri = /crucible/0/regions/525eebcd-a152-475a-a3ad-23446c9aa46b/snapshots/767abb4d-61bf-47f3-b595-e1ac8d322b3e/run
312025-01-09T20:50:59.087ZINFOcrucible-agent (datafile): region 80487933-d5a0-451d-a435-9ce085ef9062 state: Requested
322025-01-09T20:50:59.087ZINFOcrucible-agent (dropshot): request completed latency_us = 262 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:39713 req_id = 13f48b35-c059-41d2-ab52-d593460733f6 response_code = 200 uri = /crucible/0/regions
332025-01-09T20:50:59.087ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
342025-01-09T20:50:59.098ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_7ad58e80-8c74-4773-9b4c-10e6bb74a74f/crucible/regions/80487933-d5a0-451d-a435-9ce085ef9062
352025-01-09T20:50:59.098ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_7ad58e80-8c74-4773-9b4c-10e6bb74a74f/crucible/regions/80487933-d5a0-451d-a435-9ce085ef9062
362025-01-09T20:50:59.158ZINFOcrucible-agent (worker): creating region Region { id: RegionId("80487933-d5a0-451d-a435-9ce085ef9062"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19002, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/80487933-d5a0-451d-a435-9ce085ef9062" region = 80487933-d5a0-451d-a435-9ce085ef9062
372025-01-09T20:50:59.158ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "80487933-d5a0-451d-a435-9ce085ef9062" "--data" "/data/regions/80487933-d5a0-451d-a435-9ce085ef9062" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 80487933-d5a0-451d-a435-9ce085ef9062
382025-01-09T20:50:59.231ZINFOcrucible-agent (dropshot): request completed latency_us = 91 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:39713 req_id = 916d35f6-489f-4577-b109-aab00dd29461 response_code = 200 uri = /crucible/0/regions
392025-01-09T20:50:59.253ZINFOcrucible-agent (worker): region files created ok region = 80487933-d5a0-451d-a435-9ce085ef9062
402025-01-09T20:50:59.253ZINFOcrucible-agent (datafile): region 80487933-d5a0-451d-a435-9ce085ef9062 state: Requested -> Created
412025-01-09T20:50:59.253ZINFOcrucible-agent (worker): applying SMF actions post create...
422025-01-09T20:50:59.255ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-80487933-d5a0-451d-a435-9ce085ef9062
432025-01-09T20:50:59.263ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-80487933-d5a0-451d-a435-9ce085ef9062
442025-01-09T20:50:59.272ZINFOcrucible-agent (worker): creating config property group
452025-01-09T20:50:59.277ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-80487933-d5a0-451d-a435-9ce085ef9062
462025-01-09T20:50:59.277ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/80487933-d5a0-451d-a435-9ce085ef9062
472025-01-09T20:50:59.278ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
482025-01-09T20:50:59.278ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
492025-01-09T20:50:59.278ZINFOcrucible-agent (worker): commit
502025-01-09T20:50:59.283ZINFOcrucible-agent (worker): ok!
512025-01-09T20:50:59.290ZINFOcrucible-agent (worker): SMF ok!
522025-01-09T20:50:59.666ZINFOcrucible-agent (dropshot): request completed latency_us = 709 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:39713 req_id = d04894b9-1bc0-40c9-9cc8-fced2668ee13 response_code = 200 uri = /crucible/0/regions
532025-01-09T20:51:24.096ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:39713 req_id = 04358922-6f5c-4954-a70d-876f96b17ba6 response_code = 200 uri = /crucible/0/regions/80487933-d5a0-451d-a435-9ce085ef9062
542025-01-09T20:51:24.100ZINFOcrucible-agent (datafile): path is "/data/regions/80487933-d5a0-451d-a435-9ce085ef9062"
552025-01-09T20:51:24.113ZINFOcrucible-agent (datafile): dataset is oxp_7ad58e80-8c74-4773-9b4c-10e6bb74a74f/crucible/regions/80487933-d5a0-451d-a435-9ce085ef9062
562025-01-09T20:51:24.123ZINFOcrucible-agent (datafile): region 80487933-d5a0-451d-a435-9ce085ef9062 state: Created -> Tombstoned
572025-01-09T20:51:24.123ZINFOcrucible-agent (worker): applying SMF actions before removal...
582025-01-09T20:51:24.123ZINFOcrucible-agent (dropshot): request completed latency_us = 23741 local_addr = [fd00:1122:3344:101::18]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:39713 req_id = 299d8e63-2ccf-4711-bd78-58605aab57f9 response_code = 204 uri = /crucible/0/regions/80487933-d5a0-451d-a435-9ce085ef9062
592025-01-09T20:51:24.127ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-80487933-d5a0-451d-a435-9ce085ef9062 (instance states: (Some(Online), None))
602025-01-09T20:51:24.127ZINFOcrucible-agent (dropshot): request completed latency_us = 32 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:39713 req_id = 9f7982e4-a002-46f2-ab6b-193f7097262b response_code = 200 uri = /crucible/0/regions/80487933-d5a0-451d-a435-9ce085ef9062
612025-01-09T20:51:24.134ZINFOcrucible-agent (worker): SMF ok!
622025-01-09T20:51:24.146ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_7ad58e80-8c74-4773-9b4c-10e6bb74a74f/crucible/regions/80487933-d5a0-451d-a435-9ce085ef9062" region = 80487933-d5a0-451d-a435-9ce085ef9062
632025-01-09T20:51:24.208ZINFOcrucible-agent (datafile): region 80487933-d5a0-451d-a435-9ce085ef9062 state: Tombstoned -> Destroyed
642025-01-09T20:51:24.241ZINFOcrucible-agent (dropshot): request completed latency_us = 71 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:39713 req_id = 592d5bd6-1dd7-4bea-97fe-647aa7992ea0 response_code = 200 uri = /crucible/0/regions/80487933-d5a0-451d-a435-9ce085ef9062