Max level shown:
1 [ Jul 9 03:52:39 Enabled. ]
2 [ Jul 9 03:52:39 Rereading configuration. ]
3 [ Jul 9 03:52:41 Rereading configuration. ]
4 [ Jul 9 03:52:43 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible -l [fd00:1122:3344:101::18]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-07-09T03:52:43.420ZINFOcrucible-agent: dataset: "oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible"
72024-07-09T03:52:43.425ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::18]:32345
82024-07-09T03:52:43.425ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-07-09T03:52:43.445ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-07-09T03:52:43.565ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::18]:32345
112024-07-09T03:53:12.485ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::c]:51128
122024-07-09T03:53:12.493ZINFOcrucible-agent (datafile): region c97e8e98-0efa-4ac2-9f7b-eb28af248dc0 state: Requested
132024-07-09T03:53:12.500ZINFOcrucible-agent (dropshot): request completed latency_us = 9725 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:51128 req_id = 6eb44369-1c8b-4ebb-976b-3ceb07db47fa response_code = 200 uri = /crucible/0/regions
142024-07-09T03:53:12.500ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152024-07-09T03:53:12.513ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
162024-07-09T03:53:12.513ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
172024-07-09T03:53:12.617ZINFOcrucible-agent (worker): creating region Region { id: RegionId("c97e8e98-0efa-4ac2-9f7b-eb28af248dc0"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/c97e8e98-0efa-4ac2-9f7b-eb28af248dc0" region = c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
182024-07-09T03:53:12.619ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "c97e8e98-0efa-4ac2-9f7b-eb28af248dc0" "--data" "/data/regions/c97e8e98-0efa-4ac2-9f7b-eb28af248dc0" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
192024-07-09T03:53:12.734ZINFOcrucible-agent (dropshot): request completed latency_us = 83 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:51128 req_id = 1dedad4c-80e3-4dc9-af86-809107d0424d response_code = 200 uri = /crucible/0/regions
202024-07-09T03:53:12.920ZINFOcrucible-agent (worker): region files created ok region = c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
212024-07-09T03:53:12.920ZINFOcrucible-agent (datafile): region c97e8e98-0efa-4ac2-9f7b-eb28af248dc0 state: Requested -> Created
222024-07-09T03:53:12.920ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-07-09T03:53:12.921ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
242024-07-09T03:53:12.926ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
252024-07-09T03:53:12.942ZINFOcrucible-agent (worker): creating config property group
262024-07-09T03:53:12.947ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
272024-07-09T03:53:12.948ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
282024-07-09T03:53:12.948ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-07-09T03:53:12.948ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
302024-07-09T03:53:12.948ZINFOcrucible-agent (worker): commit
312024-07-09T03:53:12.954ZINFOcrucible-agent (worker): ok!
322024-07-09T03:53:12.958ZINFOcrucible-agent (worker): SMF ok!
332024-07-09T03:53:13.280ZINFOcrucible-agent (dropshot): request completed latency_us = 94 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:51128 req_id = 390cd074-d543-4364-8ec6-8f2e379ce92b response_code = 200 uri = /crucible/0/regions
342024-07-09T03:53:14.103ZINFOcrucible-agent (dropshot): request completed latency_us = 79 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:51128 req_id = db5da168-ab40-4e64-aec0-818b2f810293 response_code = 200 uri = /crucible/0/regions/c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
352024-07-09T03:53:14.105ZINFOcrucible-agent (datafile): path is "/data/regions/c97e8e98-0efa-4ac2-9f7b-eb28af248dc0"
362024-07-09T03:53:14.114ZINFOcrucible-agent (datafile): dataset is oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
372024-07-09T03:53:14.123ZINFOcrucible-agent (datafile): region c97e8e98-0efa-4ac2-9f7b-eb28af248dc0 state: Created -> Tombstoned
382024-07-09T03:53:14.123ZINFOcrucible-agent (worker): applying SMF actions before removal...
392024-07-09T03:53:14.123ZINFOcrucible-agent (dropshot): request completed latency_us = 18557 local_addr = [fd00:1122:3344:101::18]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:51128 req_id = dfbca3f3-836b-4186-b9ad-6338795f2ea6 response_code = 204 uri = /crucible/0/regions/c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
402024-07-09T03:53:14.124ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-c97e8e98-0efa-4ac2-9f7b-eb28af248dc0 (instance states: (Some(Online), None))
412024-07-09T03:53:14.125ZINFOcrucible-agent (dropshot): request completed latency_us = 29 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:51128 req_id = ab4a177e-5908-416e-b34b-95b0522241ff response_code = 200 uri = /crucible/0/regions/c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
422024-07-09T03:53:14.130ZINFOcrucible-agent (worker): SMF ok!
432024-07-09T03:53:14.140ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/c97e8e98-0efa-4ac2-9f7b-eb28af248dc0" region = c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
442024-07-09T03:53:14.192ZINFOcrucible-agent (dropshot): request completed latency_us = 87 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:51128 req_id = 8d6cc4fd-c75f-47e3-bd4b-aee01290ccaa response_code = 200 uri = /crucible/0/regions/c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
452024-07-09T03:53:14.213ZINFOcrucible-agent (datafile): region c97e8e98-0efa-4ac2-9f7b-eb28af248dc0 state: Tombstoned -> Destroyed
462024-07-09T03:53:14.340ZINFOcrucible-agent (dropshot): request completed latency_us = 48 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:51128 req_id = 2c3786fb-ce4d-46ae-b9cd-2ba56445a1f6 response_code = 200 uri = /crucible/0/regions/c97e8e98-0efa-4ac2-9f7b-eb28af248dc0
472024-07-09T03:54:01.685ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::a]:43967
482024-07-09T03:54:01.685ZINFOcrucible-agent (datafile): region 8bfb0935-fb67-477b-a7d0-f9afca122d75 state: Requested
492024-07-09T03:54:01.687ZINFOcrucible-agent (dropshot): request completed latency_us = 2446 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:43967 req_id = 8bd30d94-9576-4ae9-8ca3-d9567d00f8e9 response_code = 200 uri = /crucible/0/regions
502024-07-09T03:54:01.687ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
512024-07-09T03:54:01.696ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75
522024-07-09T03:54:01.696ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75
532024-07-09T03:54:01.737ZINFOcrucible-agent (worker): creating region Region { id: RegionId("8bfb0935-fb67-477b-a7d0-f9afca122d75"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75" region = 8bfb0935-fb67-477b-a7d0-f9afca122d75
542024-07-09T03:54:01.739ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "8bfb0935-fb67-477b-a7d0-f9afca122d75" "--data" "/data/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 8bfb0935-fb67-477b-a7d0-f9afca122d75
552024-07-09T03:54:01.804ZINFOcrucible-agent (worker): region files created ok region = 8bfb0935-fb67-477b-a7d0-f9afca122d75
562024-07-09T03:54:01.804ZINFOcrucible-agent (datafile): region 8bfb0935-fb67-477b-a7d0-f9afca122d75 state: Requested -> Created
572024-07-09T03:54:01.805ZINFOcrucible-agent (worker): applying SMF actions post create...
582024-07-09T03:54:01.806ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-8bfb0935-fb67-477b-a7d0-f9afca122d75
592024-07-09T03:54:01.809ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-8bfb0935-fb67-477b-a7d0-f9afca122d75
602024-07-09T03:54:01.819ZINFOcrucible-agent (worker): creating config property group
612024-07-09T03:54:01.823ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-8bfb0935-fb67-477b-a7d0-f9afca122d75
622024-07-09T03:54:01.823ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75
632024-07-09T03:54:01.823ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
642024-07-09T03:54:01.823ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
652024-07-09T03:54:01.823ZINFOcrucible-agent (worker): commit
662024-07-09T03:54:01.827ZINFOcrucible-agent (worker): ok!
672024-07-09T03:54:01.832ZINFOcrucible-agent (worker): SMF ok!
682024-07-09T03:54:02.009ZINFOcrucible-agent (dropshot): request completed latency_us = 69 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:43967 req_id = 017a2034-6d2b-4181-ac52-8aec4652ab8d response_code = 200 uri = /crucible/0/regions
692024-07-09T03:59:13.703ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::a]:48351
702024-07-09T03:59:13.703ZINFOcrucible-agent (dropshot): request completed latency_us = 49 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:48351 req_id = 37c17c67-18dc-4ae8-86a1-b30e43d8759b response_code = 200 uri = /crucible/0/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75
712024-07-09T03:59:13.703ZINFOcrucible-agent (datafile): path is "/data/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75"
722024-07-09T03:59:13.712ZINFOcrucible-agent (datafile): dataset is oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75
732024-07-09T03:59:13.719ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75@55ed7137-8ad7-4401-a037-50f3d8186e42
742024-07-09T03:59:13.719ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75", "55ed7137-8ad7-4401-a037-50f3d8186e42"]
752024-07-09T03:59:13.728ZINFOcrucible-agent (dropshot): request completed latency_us = 25014 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:48351 req_id = a7701ae4-8fbc-4e37-9f42-fc5baf133e48 response_code = 200 uri = /crucible/0/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75/snapshots/55ed7137-8ad7-4401-a037-50f3d8186e42
762024-07-09T03:59:13.728ZINFOcrucible-agent (datafile): path is "/data/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75"
772024-07-09T03:59:13.737ZINFOcrucible-agent (datafile): dataset is oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75
782024-07-09T03:59:13.745ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75@55ed7137-8ad7-4401-a037-50f3d8186e42
792024-07-09T03:59:13.745ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75", "55ed7137-8ad7-4401-a037-50f3d8186e42"]
802024-07-09T03:59:13.757ZINFOcrucible-agent (datafile): requesting running snapshot 8bfb0935-fb67-477b-a7d0-f9afca122d75-55ed7137-8ad7-4401-a037-50f3d8186e42 state: Requested
812024-07-09T03:59:13.757ZINFOcrucible-agent (worker): applying SMF actions for region 8bfb0935-fb67-477b-a7d0-f9afca122d75 running snapshot 55ed7137-8ad7-4401-a037-50f3d8186e42 (state Requested)...
822024-07-09T03:59:13.757ZINFOcrucible-agent (dropshot): request completed latency_us = 29354 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48351 req_id = 42b4d262-09fa-4321-8dc7-b0a3546c49bd response_code = 200 uri = /crucible/0/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75/snapshots/55ed7137-8ad7-4401-a037-50f3d8186e42/run
832024-07-09T03:59:13.764ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-8bfb0935-fb67-477b-a7d0-f9afca122d75-55ed7137-8ad7-4401-a037-50f3d8186e42
842024-07-09T03:59:13.772ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-8bfb0935-fb67-477b-a7d0-f9afca122d75-55ed7137-8ad7-4401-a037-50f3d8186e42
852024-07-09T03:59:13.785ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-8bfb0935-fb67-477b-a7d0-f9afca122d75-55ed7137-8ad7-4401-a037-50f3d8186e42
862024-07-09T03:59:13.785ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8bfb0935-fb67-477b-a7d0-f9afca122d75/.zfs/snapshot/55ed7137-8ad7-4401-a037-50f3d8186e42
872024-07-09T03:59:13.785ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
882024-07-09T03:59:13.785ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
892024-07-09T03:59:13.785ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
902024-07-09T03:59:13.785ZINFOcrucible-agent (worker): commit
912024-07-09T03:59:13.789ZINFOcrucible-agent (worker): ok!
922024-07-09T03:59:13.795ZINFOcrucible-agent (worker): SMF ok!
932024-07-09T03:59:13.795ZINFOcrucible-agent (datafile): region 8bfb0935-fb67-477b-a7d0-f9afca122d75 running snapshot 55ed7137-8ad7-4401-a037-50f3d8186e42 state: Requested -> Created
942024-07-09T03:59:16.225ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::a]:58392
952024-07-09T03:59:16.226ZINFOcrucible-agent (datafile): region 92e5c09f-25ea-4166-be22-af2931d08f1c state: Requested
962024-07-09T03:59:16.226ZINFOcrucible-agent (dropshot): request completed latency_us = 762 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:58392 req_id = 3f9170cd-e861-4def-9801-b9c75547e189 response_code = 200 uri = /crucible/0/regions
972024-07-09T03:59:16.226ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
982024-07-09T03:59:16.236ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/92e5c09f-25ea-4166-be22-af2931d08f1c
992024-07-09T03:59:16.236ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/92e5c09f-25ea-4166-be22-af2931d08f1c
1002024-07-09T03:59:16.305ZINFOcrucible-agent (worker): creating region Region { id: RegionId("92e5c09f-25ea-4166-be22-af2931d08f1c"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19002, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/92e5c09f-25ea-4166-be22-af2931d08f1c" region = 92e5c09f-25ea-4166-be22-af2931d08f1c
1012024-07-09T03:59:16.307ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "92e5c09f-25ea-4166-be22-af2931d08f1c" "--data" "/data/regions/92e5c09f-25ea-4166-be22-af2931d08f1c" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 92e5c09f-25ea-4166-be22-af2931d08f1c
1022024-07-09T03:59:16.374ZINFOcrucible-agent (worker): region files created ok region = 92e5c09f-25ea-4166-be22-af2931d08f1c
1032024-07-09T03:59:16.374ZINFOcrucible-agent (datafile): region 92e5c09f-25ea-4166-be22-af2931d08f1c state: Requested -> Created
1042024-07-09T03:59:16.374ZINFOcrucible-agent (worker): applying SMF actions post create...
1052024-07-09T03:59:16.376ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-92e5c09f-25ea-4166-be22-af2931d08f1c
1062024-07-09T03:59:16.382ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-92e5c09f-25ea-4166-be22-af2931d08f1c
1072024-07-09T03:59:16.388ZINFOcrucible-agent (worker): creating config property group
1082024-07-09T03:59:16.393ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-92e5c09f-25ea-4166-be22-af2931d08f1c
1092024-07-09T03:59:16.393ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/92e5c09f-25ea-4166-be22-af2931d08f1c
1102024-07-09T03:59:16.393ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
1112024-07-09T03:59:16.393ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
1122024-07-09T03:59:16.393ZINFOcrucible-agent (worker): commit
1132024-07-09T03:59:16.397ZINFOcrucible-agent (worker): ok!
1142024-07-09T03:59:16.402ZINFOcrucible-agent (worker): SMF ok!
1152024-07-09T03:59:16.549ZINFOcrucible-agent (dropshot): request completed latency_us = 73 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:58392 req_id = 508d985a-daa2-4076-8006-e415758f04d0 response_code = 200 uri = /crucible/0/regions
1162024-07-09T03:59:46.792ZINFOcrucible-agent (dropshot): request completed latency_us = 64 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:58392 req_id = 5956fe37-8f06-4c70-a8aa-6ea9166e2bf3 response_code = 200 uri = /crucible/0/regions/92e5c09f-25ea-4166-be22-af2931d08f1c
1172024-07-09T03:59:46.795ZINFOcrucible-agent (datafile): path is "/data/regions/92e5c09f-25ea-4166-be22-af2931d08f1c"
1182024-07-09T03:59:46.807ZINFOcrucible-agent (datafile): dataset is oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/92e5c09f-25ea-4166-be22-af2931d08f1c
1192024-07-09T03:59:46.815ZINFOcrucible-agent (datafile): region 92e5c09f-25ea-4166-be22-af2931d08f1c state: Created -> Tombstoned
1202024-07-09T03:59:46.816ZINFOcrucible-agent (worker): applying SMF actions before removal...
1212024-07-09T03:59:46.816ZINFOcrucible-agent (dropshot): request completed latency_us = 21149 local_addr = [fd00:1122:3344:101::18]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:58392 req_id = 7ea47e30-5b1e-4a80-94ea-601150be9fd0 response_code = 204 uri = /crucible/0/regions/92e5c09f-25ea-4166-be22-af2931d08f1c
1222024-07-09T03:59:46.818ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-92e5c09f-25ea-4166-be22-af2931d08f1c (instance states: (Some(Online), None))
1232024-07-09T03:59:46.819ZINFOcrucible-agent (dropshot): request completed latency_us = 25 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:58392 req_id = 5cd8e715-f1e6-44fd-8609-af7e98170854 response_code = 200 uri = /crucible/0/regions/92e5c09f-25ea-4166-be22-af2931d08f1c
1242024-07-09T03:59:46.827ZINFOcrucible-agent (worker): SMF ok!
1252024-07-09T03:59:46.836ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_a46ddd9f-b0f0-45dc-86cc-a89f72025cb4/crucible/regions/92e5c09f-25ea-4166-be22-af2931d08f1c" region = 92e5c09f-25ea-4166-be22-af2931d08f1c
1262024-07-09T03:59:46.888ZINFOcrucible-agent (datafile): region 92e5c09f-25ea-4166-be22-af2931d08f1c state: Tombstoned -> Destroyed
1272024-07-09T03:59:46.902ZINFOcrucible-agent (dropshot): request completed latency_us = 48 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:58392 req_id = 70cd31e4-1441-4f70-8dfc-f880afadf0d6 response_code = 200 uri = /crucible/0/regions/92e5c09f-25ea-4166-be22-af2931d08f1c