Max level shown:
1 [ Jul 19 17:47:08 Enabled. ]
2 [ Jul 19 17:47:08 Rereading configuration. ]
3 [ Jul 19 17:47:13 Rereading configuration. ]
4 [ Jul 19 17:47:20 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible -l [fd00:1122:3344:101::17]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-07-19T17:47:20.702ZINFOcrucible-agent: dataset: "oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible"
72024-07-19T17:47:20.707ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::17]:32345
82024-07-19T17:47:20.707ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-07-19T17:47:20.722ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-07-19T17:47:20.874ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::17]:32345
112024-07-19T17:48:08.256ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::b]:39553
122024-07-19T17:48:08.270ZINFOcrucible-agent (datafile): region 5045c681-102f-4d1c-b1f8-d55f6be938fb state: Requested
132024-07-19T17:48:08.272ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
142024-07-19T17:48:08.272ZINFOcrucible-agent (dropshot): request completed latency_us = 9761 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:39553 req_id = 17430fdf-3b97-4693-af0e-843ec3bbed5f response_code = 200 uri = /crucible/0/regions
152024-07-19T17:48:08.284ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/5045c681-102f-4d1c-b1f8-d55f6be938fb
162024-07-19T17:48:08.285ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/5045c681-102f-4d1c-b1f8-d55f6be938fb
172024-07-19T17:48:08.350ZINFOcrucible-agent (worker): creating region Region { id: RegionId("5045c681-102f-4d1c-b1f8-d55f6be938fb"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/5045c681-102f-4d1c-b1f8-d55f6be938fb" region = 5045c681-102f-4d1c-b1f8-d55f6be938fb
182024-07-19T17:48:08.352ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "5045c681-102f-4d1c-b1f8-d55f6be938fb" "--data" "/data/regions/5045c681-102f-4d1c-b1f8-d55f6be938fb" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 5045c681-102f-4d1c-b1f8-d55f6be938fb
192024-07-19T17:48:08.433ZINFOcrucible-agent (dropshot): request completed latency_us = 75 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:39553 req_id = 2eaedb64-4ee1-4ea7-a5de-bea14d014339 response_code = 200 uri = /crucible/0/regions
202024-07-19T17:48:08.611ZINFOcrucible-agent (worker): region files created ok region = 5045c681-102f-4d1c-b1f8-d55f6be938fb
212024-07-19T17:48:08.611ZINFOcrucible-agent (datafile): region 5045c681-102f-4d1c-b1f8-d55f6be938fb state: Requested -> Created
222024-07-19T17:48:08.611ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-07-19T17:48:08.612ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-5045c681-102f-4d1c-b1f8-d55f6be938fb
242024-07-19T17:48:08.618ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-5045c681-102f-4d1c-b1f8-d55f6be938fb
252024-07-19T17:48:08.633ZINFOcrucible-agent (worker): creating config property group
262024-07-19T17:48:08.638ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-5045c681-102f-4d1c-b1f8-d55f6be938fb
272024-07-19T17:48:08.639ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/5045c681-102f-4d1c-b1f8-d55f6be938fb
282024-07-19T17:48:08.641ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-07-19T17:48:08.641ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
302024-07-19T17:48:08.641ZINFOcrucible-agent (worker): commit
312024-07-19T17:48:08.649ZINFOcrucible-agent (worker): ok!
322024-07-19T17:48:08.656ZINFOcrucible-agent (worker): SMF ok!
332024-07-19T17:48:09.086ZINFOcrucible-agent (dropshot): request completed latency_us = 78 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:39553 req_id = 0d48f64b-b178-45a1-8d03-22871db831a7 response_code = 200 uri = /crucible/0/regions
342024-07-19T17:48:09.353ZINFOcrucible-agent (dropshot): request completed latency_us = 107 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:39553 req_id = 8d5c2acf-430e-478d-9373-6048b9e2eab8 response_code = 200 uri = /crucible/0/regions/5045c681-102f-4d1c-b1f8-d55f6be938fb
352024-07-19T17:48:09.356ZINFOcrucible-agent (datafile): path is "/data/regions/5045c681-102f-4d1c-b1f8-d55f6be938fb"
362024-07-19T17:48:09.366ZINFOcrucible-agent (datafile): dataset is oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/5045c681-102f-4d1c-b1f8-d55f6be938fb
372024-07-19T17:48:09.376ZINFOcrucible-agent (datafile): region 5045c681-102f-4d1c-b1f8-d55f6be938fb state: Created -> Tombstoned
382024-07-19T17:48:09.376ZINFOcrucible-agent (worker): applying SMF actions before removal...
392024-07-19T17:48:09.376ZINFOcrucible-agent (dropshot): request completed latency_us = 20564 local_addr = [fd00:1122:3344:101::17]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:39553 req_id = 2610285e-a11e-49fa-8c69-b502d0ef1360 response_code = 204 uri = /crucible/0/regions/5045c681-102f-4d1c-b1f8-d55f6be938fb
402024-07-19T17:48:09.377ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-5045c681-102f-4d1c-b1f8-d55f6be938fb (instance states: (Some(Online), None))
412024-07-19T17:48:09.378ZINFOcrucible-agent (dropshot): request completed latency_us = 29 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:39553 req_id = 7847f5e6-c57d-4b2d-b354-bdea35dc7fbb response_code = 200 uri = /crucible/0/regions/5045c681-102f-4d1c-b1f8-d55f6be938fb
422024-07-19T17:48:09.381ZINFOcrucible-agent (worker): SMF ok!
432024-07-19T17:48:09.391ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/5045c681-102f-4d1c-b1f8-d55f6be938fb" region = 5045c681-102f-4d1c-b1f8-d55f6be938fb
442024-07-19T17:48:09.452ZINFOcrucible-agent (datafile): region 5045c681-102f-4d1c-b1f8-d55f6be938fb state: Tombstoned -> Destroyed
452024-07-19T17:48:09.466ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:39553 req_id = 5c8c9c32-9e10-4db7-8078-0b966eede5c1 response_code = 200 uri = /crucible/0/regions/5045c681-102f-4d1c-b1f8-d55f6be938fb
462024-07-19T17:48:33.766ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:34501
472024-07-19T17:48:33.767ZINFOcrucible-agent (datafile): region 4fdde1a3-5a49-4c30-8122-a827ae4c366a state: Requested
482024-07-19T17:48:33.769ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492024-07-19T17:48:33.769ZINFOcrucible-agent (dropshot): request completed latency_us = 3134 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:34501 req_id = a8c682ad-56ab-4281-a5d8-7af83824ae71 response_code = 200 uri = /crucible/0/regions
502024-07-19T17:48:33.778ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a
512024-07-19T17:48:33.778ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a
522024-07-19T17:48:33.831ZINFOcrucible-agent (worker): creating region Region { id: RegionId("4fdde1a3-5a49-4c30-8122-a827ae4c366a"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a" region = 4fdde1a3-5a49-4c30-8122-a827ae4c366a
532024-07-19T17:48:33.831ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "4fdde1a3-5a49-4c30-8122-a827ae4c366a" "--data" "/data/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 4fdde1a3-5a49-4c30-8122-a827ae4c366a
542024-07-19T17:48:33.891ZINFOcrucible-agent (worker): region files created ok region = 4fdde1a3-5a49-4c30-8122-a827ae4c366a
552024-07-19T17:48:33.891ZINFOcrucible-agent (datafile): region 4fdde1a3-5a49-4c30-8122-a827ae4c366a state: Requested -> Created
562024-07-19T17:48:33.891ZINFOcrucible-agent (worker): applying SMF actions post create...
572024-07-19T17:48:33.892ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-4fdde1a3-5a49-4c30-8122-a827ae4c366a
582024-07-19T17:48:33.895ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-4fdde1a3-5a49-4c30-8122-a827ae4c366a
592024-07-19T17:48:33.903ZINFOcrucible-agent (worker): creating config property group
602024-07-19T17:48:33.905ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-4fdde1a3-5a49-4c30-8122-a827ae4c366a
612024-07-19T17:48:33.906ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a
622024-07-19T17:48:33.906ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
632024-07-19T17:48:33.906ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
642024-07-19T17:48:33.906ZINFOcrucible-agent (worker): commit
652024-07-19T17:48:33.909ZINFOcrucible-agent (worker): ok!
662024-07-19T17:48:33.913ZINFOcrucible-agent (worker): SMF ok!
672024-07-19T17:48:33.968ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:34501 req_id = 64b43344-9984-4e06-838b-3bf664bf2f31 response_code = 200 uri = /crucible/0/regions
682024-07-19T17:53:44.840ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:42644
692024-07-19T17:53:44.840ZINFOcrucible-agent (dropshot): request completed latency_us = 95 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42644 req_id = 8ebf1e2f-2b37-417a-9903-8ee1ee9e64e3 response_code = 200 uri = /crucible/0/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a
702024-07-19T17:53:44.840ZINFOcrucible-agent (datafile): path is "/data/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a"
712024-07-19T17:53:44.849ZINFOcrucible-agent (datafile): dataset is oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a
722024-07-19T17:53:44.859ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a@dddcbe4b-77ae-415e-9b8a-707ca2f9d71e
732024-07-19T17:53:44.859ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a", "dddcbe4b-77ae-415e-9b8a-707ca2f9d71e"]
742024-07-19T17:53:44.871ZINFOcrucible-agent (dropshot): request completed latency_us = 30692 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42644 req_id = 3eeab07e-33e2-4342-aa4b-52ef5bf6a435 response_code = 200 uri = /crucible/0/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a/snapshots/dddcbe4b-77ae-415e-9b8a-707ca2f9d71e
752024-07-19T17:53:44.871ZINFOcrucible-agent (datafile): path is "/data/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a"
762024-07-19T17:53:44.880ZINFOcrucible-agent (datafile): dataset is oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a
772024-07-19T17:53:44.889ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a@dddcbe4b-77ae-415e-9b8a-707ca2f9d71e
782024-07-19T17:53:44.889ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a", "dddcbe4b-77ae-415e-9b8a-707ca2f9d71e"]
792024-07-19T17:53:44.900ZINFOcrucible-agent (datafile): requesting running snapshot 4fdde1a3-5a49-4c30-8122-a827ae4c366a-dddcbe4b-77ae-415e-9b8a-707ca2f9d71e state: Requested
802024-07-19T17:53:44.901ZINFOcrucible-agent (worker): applying SMF actions for region 4fdde1a3-5a49-4c30-8122-a827ae4c366a running snapshot dddcbe4b-77ae-415e-9b8a-707ca2f9d71e (state Requested)...
812024-07-19T17:53:44.901ZINFOcrucible-agent (dropshot): request completed latency_us = 30638 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42644 req_id = d8a2a449-0637-4c2e-bf14-071ff9ee86eb response_code = 200 uri = /crucible/0/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a/snapshots/dddcbe4b-77ae-415e-9b8a-707ca2f9d71e/run
822024-07-19T17:53:44.908ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-4fdde1a3-5a49-4c30-8122-a827ae4c366a-dddcbe4b-77ae-415e-9b8a-707ca2f9d71e
832024-07-19T17:53:44.915ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-4fdde1a3-5a49-4c30-8122-a827ae4c366a-dddcbe4b-77ae-415e-9b8a-707ca2f9d71e
842024-07-19T17:53:44.927ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-4fdde1a3-5a49-4c30-8122-a827ae4c366a-dddcbe4b-77ae-415e-9b8a-707ca2f9d71e
852024-07-19T17:53:44.928ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4fdde1a3-5a49-4c30-8122-a827ae4c366a/.zfs/snapshot/dddcbe4b-77ae-415e-9b8a-707ca2f9d71e
862024-07-19T17:53:44.928ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
872024-07-19T17:53:44.928ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
882024-07-19T17:53:44.928ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
892024-07-19T17:53:44.928ZINFOcrucible-agent (worker): commit
902024-07-19T17:53:44.932ZINFOcrucible-agent (worker): ok!
912024-07-19T17:53:44.936ZINFOcrucible-agent (worker): SMF ok!
922024-07-19T17:53:44.936ZINFOcrucible-agent (datafile): region 4fdde1a3-5a49-4c30-8122-a827ae4c366a running snapshot dddcbe4b-77ae-415e-9b8a-707ca2f9d71e state: Requested -> Created
932024-07-19T17:53:47.635ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::17]:32345 remote_addr = [fd00:1122:3344:101::a]:42215
942024-07-19T17:53:47.635ZINFOcrucible-agent (datafile): region 39ad10e9-0a2e-4ba6-85df-80f45e358d27 state: Requested
952024-07-19T17:53:47.635ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
962024-07-19T17:53:47.635ZINFOcrucible-agent (dropshot): request completed latency_us = 211 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42215 req_id = aec69edf-487c-4974-96ff-9d6cd641d1a0 response_code = 200 uri = /crucible/0/regions
972024-07-19T17:53:47.645ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/39ad10e9-0a2e-4ba6-85df-80f45e358d27
982024-07-19T17:53:47.645ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/39ad10e9-0a2e-4ba6-85df-80f45e358d27
992024-07-19T17:53:47.702ZINFOcrucible-agent (worker): creating region Region { id: RegionId("39ad10e9-0a2e-4ba6-85df-80f45e358d27"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19002, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/39ad10e9-0a2e-4ba6-85df-80f45e358d27" region = 39ad10e9-0a2e-4ba6-85df-80f45e358d27
1002024-07-19T17:53:47.704ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "39ad10e9-0a2e-4ba6-85df-80f45e358d27" "--data" "/data/regions/39ad10e9-0a2e-4ba6-85df-80f45e358d27" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 39ad10e9-0a2e-4ba6-85df-80f45e358d27
1012024-07-19T17:53:47.773ZINFOcrucible-agent (worker): region files created ok region = 39ad10e9-0a2e-4ba6-85df-80f45e358d27
1022024-07-19T17:53:47.773ZINFOcrucible-agent (datafile): region 39ad10e9-0a2e-4ba6-85df-80f45e358d27 state: Requested -> Created
1032024-07-19T17:53:47.773ZINFOcrucible-agent (worker): applying SMF actions post create...
1042024-07-19T17:53:47.774ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-39ad10e9-0a2e-4ba6-85df-80f45e358d27
1052024-07-19T17:53:47.777ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-39ad10e9-0a2e-4ba6-85df-80f45e358d27
1062024-07-19T17:53:47.784ZINFOcrucible-agent (worker): creating config property group
1072024-07-19T17:53:47.787ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-39ad10e9-0a2e-4ba6-85df-80f45e358d27
1082024-07-19T17:53:47.787ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/39ad10e9-0a2e-4ba6-85df-80f45e358d27
1092024-07-19T17:53:47.787ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
1102024-07-19T17:53:47.787ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::17
1112024-07-19T17:53:47.787ZINFOcrucible-agent (worker): commit
1122024-07-19T17:53:47.791ZINFOcrucible-agent (worker): ok!
1132024-07-19T17:53:47.800ZINFOcrucible-agent (worker): SMF ok!
1142024-07-19T17:53:47.849ZINFOcrucible-agent (dropshot): request completed latency_us = 70 local_addr = [fd00:1122:3344:101::17]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42215 req_id = 71f411c6-446c-4ac0-8de7-857395c72a6d response_code = 200 uri = /crucible/0/regions
1152024-07-19T17:54:12.282ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42215 req_id = 54cec7d2-7e63-4edc-9e87-5a30029d7ab8 response_code = 200 uri = /crucible/0/regions/39ad10e9-0a2e-4ba6-85df-80f45e358d27
1162024-07-19T17:54:12.285ZINFOcrucible-agent (datafile): path is "/data/regions/39ad10e9-0a2e-4ba6-85df-80f45e358d27"
1172024-07-19T17:54:12.296ZINFOcrucible-agent (datafile): dataset is oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/39ad10e9-0a2e-4ba6-85df-80f45e358d27
1182024-07-19T17:54:12.303ZINFOcrucible-agent (datafile): region 39ad10e9-0a2e-4ba6-85df-80f45e358d27 state: Created -> Tombstoned
1192024-07-19T17:54:12.303ZINFOcrucible-agent (worker): applying SMF actions before removal...
1202024-07-19T17:54:12.304ZINFOcrucible-agent (dropshot): request completed latency_us = 18945 local_addr = [fd00:1122:3344:101::17]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:42215 req_id = bbfd0c1c-86e3-4d79-88eb-c6ef57d99080 response_code = 204 uri = /crucible/0/regions/39ad10e9-0a2e-4ba6-85df-80f45e358d27
1212024-07-19T17:54:12.305ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-39ad10e9-0a2e-4ba6-85df-80f45e358d27 (instance states: (Some(Online), None))
1222024-07-19T17:54:12.306ZINFOcrucible-agent (dropshot): request completed latency_us = 28 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42215 req_id = 574bf7ac-8df2-4090-8ada-83fa5a9ad284 response_code = 200 uri = /crucible/0/regions/39ad10e9-0a2e-4ba6-85df-80f45e358d27
1232024-07-19T17:54:12.312ZINFOcrucible-agent (worker): SMF ok!
1242024-07-19T17:54:12.320ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_765bc25e-72d0-4d30-8cfe-a58b7db5c2e8/crucible/regions/39ad10e9-0a2e-4ba6-85df-80f45e358d27" region = 39ad10e9-0a2e-4ba6-85df-80f45e358d27
1252024-07-19T17:54:12.370ZINFOcrucible-agent (datafile): region 39ad10e9-0a2e-4ba6-85df-80f45e358d27 state: Tombstoned -> Destroyed
1262024-07-19T17:54:12.457ZINFOcrucible-agent (dropshot): request completed latency_us = 565 local_addr = [fd00:1122:3344:101::17]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42215 req_id = c2dae153-ef83-4ed1-b15c-4941fb961c51 response_code = 200 uri = /crucible/0/regions/39ad10e9-0a2e-4ba6-85df-80f45e358d27