Max level shown:
1 [ May 13 20:46:45 Enabled. ]
2 [ May 13 20:46:45 Rereading configuration. ]
3 [ May 13 20:46:46 Rereading configuration. ]
4 [ May 13 20:46:48 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible -l [fd00:1122:3344:101::14]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-05-13T20:46:48.392ZINFOcrucible-agent: dataset: "oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible"
72024-05-13T20:46:48.397ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::14]:32345
82024-05-13T20:46:48.397ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-05-13T20:46:48.419ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-05-13T20:46:48.495ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::14]:32345
112024-05-13T20:47:23.988ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::b]:48897
122024-05-13T20:47:24.025ZINFOcrucible-agent (datafile): region 602d3fe4-bf7d-49f4-b4b0-793f9c961663 state: Requested
132024-05-13T20:47:24.029ZINFOcrucible-agent (dropshot): request completed latency_us = 13570 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:48897 req_id = 4ad50245-cedf-42a4-89a8-970108f22e07 response_code = 200 uri = /crucible/0/regions
142024-05-13T20:47:24.031ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152024-05-13T20:47:24.040ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/602d3fe4-bf7d-49f4-b4b0-793f9c961663
162024-05-13T20:47:24.040ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/602d3fe4-bf7d-49f4-b4b0-793f9c961663
172024-05-13T20:47:24.111ZINFOcrucible-agent (worker): creating region Region { id: RegionId("602d3fe4-bf7d-49f4-b4b0-793f9c961663"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/602d3fe4-bf7d-49f4-b4b0-793f9c961663" region = 602d3fe4-bf7d-49f4-b4b0-793f9c961663
182024-05-13T20:47:24.113ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "602d3fe4-bf7d-49f4-b4b0-793f9c961663" "--data" "/data/regions/602d3fe4-bf7d-49f4-b4b0-793f9c961663" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 602d3fe4-bf7d-49f4-b4b0-793f9c961663
192024-05-13T20:47:24.291ZINFOcrucible-agent (dropshot): request completed latency_us = 82 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:48897 req_id = aa4a0ef8-e88a-44c6-9e2a-dc70f4100a77 response_code = 200 uri = /crucible/0/regions
202024-05-13T20:47:24.351ZINFOcrucible-agent (worker): region files created ok region = 602d3fe4-bf7d-49f4-b4b0-793f9c961663
212024-05-13T20:47:24.351ZINFOcrucible-agent (datafile): region 602d3fe4-bf7d-49f4-b4b0-793f9c961663 state: Requested -> Created
222024-05-13T20:47:24.351ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-05-13T20:47:24.352ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-602d3fe4-bf7d-49f4-b4b0-793f9c961663
242024-05-13T20:47:24.356ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-602d3fe4-bf7d-49f4-b4b0-793f9c961663
252024-05-13T20:47:24.368ZINFOcrucible-agent (worker): creating config property group
262024-05-13T20:47:24.371ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-602d3fe4-bf7d-49f4-b4b0-793f9c961663
272024-05-13T20:47:24.372ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/602d3fe4-bf7d-49f4-b4b0-793f9c961663
282024-05-13T20:47:24.374ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-05-13T20:47:24.374ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
302024-05-13T20:47:24.374ZINFOcrucible-agent (worker): commit
312024-05-13T20:47:24.380ZINFOcrucible-agent (worker): ok!
322024-05-13T20:47:24.384ZINFOcrucible-agent (worker): SMF ok!
332024-05-13T20:47:24.936ZINFOcrucible-agent (dropshot): request completed latency_us = 80 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:48897 req_id = 327c07a5-3e3c-4a32-a5c2-1e2d956625aa response_code = 200 uri = /crucible/0/regions
342024-05-13T20:47:25.491ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::b]:58600
352024-05-13T20:47:25.491ZINFOcrucible-agent (dropshot): request completed latency_us = 61 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:58600 req_id = 7f46c331-621b-4694-aad4-795c5ab3b208 response_code = 200 uri = /crucible/0/regions/602d3fe4-bf7d-49f4-b4b0-793f9c961663
362024-05-13T20:47:25.491ZINFOcrucible-agent (datafile): path is "/data/regions/602d3fe4-bf7d-49f4-b4b0-793f9c961663"
372024-05-13T20:47:25.501ZINFOcrucible-agent (datafile): dataset is oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/602d3fe4-bf7d-49f4-b4b0-793f9c961663
382024-05-13T20:47:25.509ZINFOcrucible-agent (datafile): region 602d3fe4-bf7d-49f4-b4b0-793f9c961663 state: Created -> Tombstoned
392024-05-13T20:47:25.510ZINFOcrucible-agent (worker): applying SMF actions before removal...
402024-05-13T20:47:25.510ZINFOcrucible-agent (dropshot): request completed latency_us = 18447 local_addr = [fd00:1122:3344:101::14]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:58600 req_id = 84e11dad-1129-455c-98af-ffd88fa29ccb response_code = 204 uri = /crucible/0/regions/602d3fe4-bf7d-49f4-b4b0-793f9c961663
412024-05-13T20:47:25.510ZINFOcrucible-agent (dropshot): request completed latency_us = 21 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:58600 req_id = 4155de75-955e-4643-8ce6-e312e9eeb256 response_code = 200 uri = /crucible/0/regions/602d3fe4-bf7d-49f4-b4b0-793f9c961663
422024-05-13T20:47:25.513ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-602d3fe4-bf7d-49f4-b4b0-793f9c961663 (instance states: (Some(Online), None))
432024-05-13T20:47:25.517ZINFOcrucible-agent (worker): SMF ok!
442024-05-13T20:47:25.527ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/602d3fe4-bf7d-49f4-b4b0-793f9c961663" region = 602d3fe4-bf7d-49f4-b4b0-793f9c961663
452024-05-13T20:47:25.582ZINFOcrucible-agent (datafile): region 602d3fe4-bf7d-49f4-b4b0-793f9c961663 state: Tombstoned -> Destroyed
462024-05-13T20:47:25.638ZINFOcrucible-agent (dropshot): request completed latency_us = 87 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:58600 req_id = 915dfa98-cb05-4b46-88a2-71e800b97631 response_code = 200 uri = /crucible/0/regions/602d3fe4-bf7d-49f4-b4b0-793f9c961663
472024-05-13T20:48:05.807ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:40935
482024-05-13T20:48:05.837ZINFOcrucible-agent (datafile): region 66bed7a2-18ed-48f1-b7c3-22308e86d44c state: Requested
492024-05-13T20:48:05.839ZINFOcrucible-agent (dropshot): request completed latency_us = 2397 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:40935 req_id = 5789be9a-a0a1-4a9f-bbf5-db62f1a5b619 response_code = 200 uri = /crucible/0/regions
502024-05-13T20:48:05.839ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
512024-05-13T20:48:05.848ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c
522024-05-13T20:48:05.848ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c
532024-05-13T20:48:05.908ZINFOcrucible-agent (worker): creating region Region { id: RegionId("66bed7a2-18ed-48f1-b7c3-22308e86d44c"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c" region = 66bed7a2-18ed-48f1-b7c3-22308e86d44c
542024-05-13T20:48:05.910ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "66bed7a2-18ed-48f1-b7c3-22308e86d44c" "--data" "/data/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 66bed7a2-18ed-48f1-b7c3-22308e86d44c
552024-05-13T20:48:05.969ZINFOcrucible-agent (worker): region files created ok region = 66bed7a2-18ed-48f1-b7c3-22308e86d44c
562024-05-13T20:48:05.969ZINFOcrucible-agent (datafile): region 66bed7a2-18ed-48f1-b7c3-22308e86d44c state: Requested -> Created
572024-05-13T20:48:05.969ZINFOcrucible-agent (worker): applying SMF actions post create...
582024-05-13T20:48:05.971ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-66bed7a2-18ed-48f1-b7c3-22308e86d44c
592024-05-13T20:48:05.975ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-66bed7a2-18ed-48f1-b7c3-22308e86d44c
602024-05-13T20:48:05.985ZINFOcrucible-agent (worker): creating config property group
612024-05-13T20:48:05.988ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-66bed7a2-18ed-48f1-b7c3-22308e86d44c
622024-05-13T20:48:05.988ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c
632024-05-13T20:48:05.989ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
642024-05-13T20:48:05.989ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
652024-05-13T20:48:05.989ZINFOcrucible-agent (worker): commit
662024-05-13T20:48:05.994ZINFOcrucible-agent (worker): ok!
672024-05-13T20:48:06.000ZINFOcrucible-agent (worker): SMF ok!
682024-05-13T20:48:06.144ZINFOcrucible-agent (dropshot): request completed latency_us = 73 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:40935 req_id = 70893bf4-5df4-402d-baa3-ecf0f5e16752 response_code = 200 uri = /crucible/0/regions
692024-05-13T20:53:31.101ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:34737
702024-05-13T20:53:31.101ZINFOcrucible-agent (dropshot): request completed latency_us = 68 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:34737 req_id = b0ef8502-ad52-45c8-9610-19abf4cd5c5f response_code = 200 uri = /crucible/0/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c
712024-05-13T20:53:31.101ZINFOcrucible-agent (datafile): path is "/data/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c"
722024-05-13T20:53:31.110ZINFOcrucible-agent (datafile): dataset is oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c
732024-05-13T20:53:31.118ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c@b398d753-a4f4-4314-9aa7-85697a3885bc
742024-05-13T20:53:31.118ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c", "b398d753-a4f4-4314-9aa7-85697a3885bc"]
752024-05-13T20:53:31.134ZINFOcrucible-agent (dropshot): request completed latency_us = 32980 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:34737 req_id = 12ec6b1a-2de8-4b7e-a18b-4f0b7132b5c9 response_code = 200 uri = /crucible/0/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c/snapshots/b398d753-a4f4-4314-9aa7-85697a3885bc
762024-05-13T20:53:31.134ZINFOcrucible-agent (datafile): path is "/data/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c"
772024-05-13T20:53:31.143ZINFOcrucible-agent (datafile): dataset is oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c
782024-05-13T20:53:31.151ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c@b398d753-a4f4-4314-9aa7-85697a3885bc
792024-05-13T20:53:31.151ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c", "b398d753-a4f4-4314-9aa7-85697a3885bc"]
802024-05-13T20:53:31.161ZINFOcrucible-agent (datafile): requesting running snapshot 66bed7a2-18ed-48f1-b7c3-22308e86d44c-b398d753-a4f4-4314-9aa7-85697a3885bc state: Requested
812024-05-13T20:53:31.161ZINFOcrucible-agent (worker): applying SMF actions for region 66bed7a2-18ed-48f1-b7c3-22308e86d44c running snapshot b398d753-a4f4-4314-9aa7-85697a3885bc (state Requested)...
822024-05-13T20:53:31.161ZINFOcrucible-agent (dropshot): request completed latency_us = 26784 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:34737 req_id = 9825948b-e709-4c8e-98ee-d03e51b3ea7f response_code = 200 uri = /crucible/0/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c/snapshots/b398d753-a4f4-4314-9aa7-85697a3885bc/run
832024-05-13T20:53:31.172ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-66bed7a2-18ed-48f1-b7c3-22308e86d44c-b398d753-a4f4-4314-9aa7-85697a3885bc
842024-05-13T20:53:31.179ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-66bed7a2-18ed-48f1-b7c3-22308e86d44c-b398d753-a4f4-4314-9aa7-85697a3885bc
852024-05-13T20:53:31.190ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-66bed7a2-18ed-48f1-b7c3-22308e86d44c-b398d753-a4f4-4314-9aa7-85697a3885bc
862024-05-13T20:53:31.190ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/66bed7a2-18ed-48f1-b7c3-22308e86d44c/.zfs/snapshot/b398d753-a4f4-4314-9aa7-85697a3885bc
872024-05-13T20:53:31.190ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
882024-05-13T20:53:31.190ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
892024-05-13T20:53:31.190ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
902024-05-13T20:53:31.190ZINFOcrucible-agent (worker): commit
912024-05-13T20:53:31.194ZINFOcrucible-agent (worker): ok!
922024-05-13T20:53:31.198ZINFOcrucible-agent (worker): SMF ok!
932024-05-13T20:53:31.198ZINFOcrucible-agent (datafile): region 66bed7a2-18ed-48f1-b7c3-22308e86d44c running snapshot b398d753-a4f4-4314-9aa7-85697a3885bc state: Requested -> Created
942024-05-13T20:53:34.660ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:45758
952024-05-13T20:53:34.722ZINFOcrucible-agent (datafile): region 5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3 state: Requested
962024-05-13T20:53:34.723ZINFOcrucible-agent (dropshot): request completed latency_us = 228 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:45758 req_id = da35c726-c671-4f6f-a76f-4efd16de3209 response_code = 200 uri = /crucible/0/regions
972024-05-13T20:53:34.723ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
982024-05-13T20:53:34.731ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3
992024-05-13T20:53:34.731ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3
1002024-05-13T20:53:34.802ZINFOcrucible-agent (worker): creating region Region { id: RegionId("5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19002, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3" region = 5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3
1012024-05-13T20:53:34.804ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3" "--data" "/data/regions/5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3
1022024-05-13T20:53:34.910ZINFOcrucible-agent (worker): region files created ok region = 5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3
1032024-05-13T20:53:34.910ZINFOcrucible-agent (datafile): region 5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3 state: Requested -> Created
1042024-05-13T20:53:34.910ZINFOcrucible-agent (worker): applying SMF actions post create...
1052024-05-13T20:53:34.911ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3
1062024-05-13T20:53:34.914ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3
1072024-05-13T20:53:34.921ZINFOcrucible-agent (worker): creating config property group
1082024-05-13T20:53:34.923ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3
1092024-05-13T20:53:34.923ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3
1102024-05-13T20:53:34.923ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
1112024-05-13T20:53:34.924ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
1122024-05-13T20:53:34.924ZINFOcrucible-agent (worker): commit
1132024-05-13T20:53:34.927ZINFOcrucible-agent (worker): ok!
1142024-05-13T20:53:34.937ZINFOcrucible-agent (worker): SMF ok!
1152024-05-13T20:53:35.062ZINFOcrucible-agent (dropshot): request completed latency_us = 725 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:45758 req_id = e9dddfa5-f4e0-449a-a08b-88c16247b721 response_code = 200 uri = /crucible/0/regions
1162024-05-13T20:54:00.083ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:33267
1172024-05-13T20:54:00.083ZINFOcrucible-agent (dropshot): request completed latency_us = 56 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33267 req_id = 1b803549-fc53-4fa5-adbb-a75f7087827a response_code = 200 uri = /crucible/0/regions/5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3
1182024-05-13T20:54:00.083ZINFOcrucible-agent (datafile): path is "/data/regions/5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3"
1192024-05-13T20:54:00.093ZINFOcrucible-agent (datafile): dataset is oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3
1202024-05-13T20:54:00.102ZINFOcrucible-agent (datafile): region 5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3 state: Created -> Tombstoned
1212024-05-13T20:54:00.102ZINFOcrucible-agent (worker): applying SMF actions before removal...
1222024-05-13T20:54:00.102ZINFOcrucible-agent (dropshot): request completed latency_us = 18939 local_addr = [fd00:1122:3344:101::14]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:33267 req_id = 97dcd66e-bf18-453e-ac0d-17ac98da8fa3 response_code = 204 uri = /crucible/0/regions/5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3
1232024-05-13T20:54:00.102ZINFOcrucible-agent (dropshot): request completed latency_us = 24 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33267 req_id = 83e451fe-161b-4c6f-b1fb-3b9d37b48aa9 response_code = 200 uri = /crucible/0/regions/5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3
1242024-05-13T20:54:00.104ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3 (instance states: (Some(Online), None))
1252024-05-13T20:54:00.110ZINFOcrucible-agent (worker): SMF ok!
1262024-05-13T20:54:00.119ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_adacd6f7-f256-419c-bcef-46605236baba/crucible/regions/5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3" region = 5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3
1272024-05-13T20:54:00.172ZINFOcrucible-agent (datafile): region 5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3 state: Tombstoned -> Destroyed
1282024-05-13T20:54:00.186ZINFOcrucible-agent (dropshot): request completed latency_us = 82 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:33267 req_id = f497c251-a1c9-431f-b88b-2a46153a8cf3 response_code = 200 uri = /crucible/0/regions/5fb9ab5e-7d1c-42cc-8d59-f445cdfe1ac3