Max level shown:
1 [ Aug 28 17:52:46 Enabled. ]
2 [ Aug 28 17:52:46 Rereading configuration. ]
3 [ Aug 28 17:52:47 Rereading configuration. ]
4 [ Aug 28 17:52:49 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible -l [fd00:1122:3344:101::12]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-08-28T17:52:49.518ZINFOcrucible-agent: dataset: "oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible"
72024-08-28T17:52:49.519ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::12]:32345
82024-08-28T17:52:49.519ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-08-28T17:52:49.544ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-08-28T17:52:49.724ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::12]:32345
112024-08-28T17:53:33.573ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:63201
122024-08-28T17:53:33.594ZINFOcrucible-agent (datafile): region 722e1116-c9da-4555-af81-1ab789024a82 state: Requested
132024-08-28T17:53:33.599ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
142024-08-28T17:53:33.603ZINFOcrucible-agent (dropshot): request completed latency_us = 14423 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:63201 req_id = 1805dcff-0798-4966-9750-74b9d16c2d85 response_code = 200 uri = /crucible/0/regions
152024-08-28T17:53:33.611ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/722e1116-c9da-4555-af81-1ab789024a82
162024-08-28T17:53:33.611ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/722e1116-c9da-4555-af81-1ab789024a82
172024-08-28T17:53:33.742ZINFOcrucible-agent (worker): creating region Region { id: RegionId("722e1116-c9da-4555-af81-1ab789024a82"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/722e1116-c9da-4555-af81-1ab789024a82" region = 722e1116-c9da-4555-af81-1ab789024a82
182024-08-28T17:53:33.744ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "722e1116-c9da-4555-af81-1ab789024a82" "--data" "/data/regions/722e1116-c9da-4555-af81-1ab789024a82" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 722e1116-c9da-4555-af81-1ab789024a82
192024-08-28T17:53:33.830ZINFOcrucible-agent (dropshot): request completed latency_us = 86 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:63201 req_id = 127861dc-3d9a-41eb-a8b2-8ebdb86c2513 response_code = 200 uri = /crucible/0/regions
202024-08-28T17:53:33.939ZINFOcrucible-agent (worker): region files created ok region = 722e1116-c9da-4555-af81-1ab789024a82
212024-08-28T17:53:33.939ZINFOcrucible-agent (datafile): region 722e1116-c9da-4555-af81-1ab789024a82 state: Requested -> Created
222024-08-28T17:53:33.939ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-08-28T17:53:33.939ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-722e1116-c9da-4555-af81-1ab789024a82
242024-08-28T17:53:33.948ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-722e1116-c9da-4555-af81-1ab789024a82
252024-08-28T17:53:33.989ZINFOcrucible-agent (worker): creating config property group
262024-08-28T17:53:34.003ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-722e1116-c9da-4555-af81-1ab789024a82
272024-08-28T17:53:34.018ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/722e1116-c9da-4555-af81-1ab789024a82
282024-08-28T17:53:34.020ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-08-28T17:53:34.020ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
302024-08-28T17:53:34.020ZINFOcrucible-agent (worker): commit
312024-08-28T17:53:34.027ZINFOcrucible-agent (worker): ok!
322024-08-28T17:53:34.038ZINFOcrucible-agent (worker): SMF ok!
332024-08-28T17:53:34.345ZINFOcrucible-agent (dropshot): request completed latency_us = 79 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:63201 req_id = f46a9bc6-d6c4-4fdb-bd09-7fed55d20b8b response_code = 200 uri = /crucible/0/regions
342024-08-28T17:53:35.855ZINFOcrucible-agent (dropshot): request completed latency_us = 59 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:63201 req_id = 64c495eb-8994-42d0-b29e-457c05ffc4c8 response_code = 200 uri = /crucible/0/regions/722e1116-c9da-4555-af81-1ab789024a82
352024-08-28T17:53:35.859ZINFOcrucible-agent (datafile): path is "/data/regions/722e1116-c9da-4555-af81-1ab789024a82"
362024-08-28T17:53:35.869ZINFOcrucible-agent (datafile): dataset is oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/722e1116-c9da-4555-af81-1ab789024a82
372024-08-28T17:53:35.879ZINFOcrucible-agent (datafile): region 722e1116-c9da-4555-af81-1ab789024a82 state: Created -> Tombstoned
382024-08-28T17:53:35.879ZINFOcrucible-agent (worker): applying SMF actions before removal...
392024-08-28T17:53:35.879ZINFOcrucible-agent (dropshot): request completed latency_us = 21096 local_addr = [fd00:1122:3344:101::12]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:63201 req_id = a2cc2168-340c-4ab3-a984-75483d1cc5c1 response_code = 204 uri = /crucible/0/regions/722e1116-c9da-4555-af81-1ab789024a82
402024-08-28T17:53:35.880ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-722e1116-c9da-4555-af81-1ab789024a82 (instance states: (Some(Online), None))
412024-08-28T17:53:35.882ZINFOcrucible-agent (dropshot): request completed latency_us = 24 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:63201 req_id = 0212d411-9167-4b7f-9678-d9f2352521be response_code = 200 uri = /crucible/0/regions/722e1116-c9da-4555-af81-1ab789024a82
422024-08-28T17:53:35.885ZINFOcrucible-agent (worker): SMF ok!
432024-08-28T17:53:35.896ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/722e1116-c9da-4555-af81-1ab789024a82" region = 722e1116-c9da-4555-af81-1ab789024a82
442024-08-28T17:53:35.966ZINFOcrucible-agent (datafile): region 722e1116-c9da-4555-af81-1ab789024a82 state: Tombstoned -> Destroyed
452024-08-28T17:53:35.967ZINFOcrucible-agent (dropshot): request completed latency_us = 594 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:63201 req_id = 33a463c4-4bf4-413e-b5f6-db2a3761041a response_code = 200 uri = /crucible/0/regions/722e1116-c9da-4555-af81-1ab789024a82
462024-08-28T17:53:38.516ZINFOcrucible-agent (datafile): region d2ef745f-4fa6-4988-81a1-431c95fef043 state: Requested
472024-08-28T17:53:38.516ZINFOcrucible-agent (dropshot): request completed latency_us = 199 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:63201 req_id = b86c1a8c-ee53-4876-84a0-a17f11257ed0 response_code = 200 uri = /crucible/0/regions
482024-08-28T17:53:38.516ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492024-08-28T17:53:38.526ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/d2ef745f-4fa6-4988-81a1-431c95fef043
502024-08-28T17:53:38.526ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/d2ef745f-4fa6-4988-81a1-431c95fef043
512024-08-28T17:53:38.647ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d2ef745f-4fa6-4988-81a1-431c95fef043"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d2ef745f-4fa6-4988-81a1-431c95fef043" region = d2ef745f-4fa6-4988-81a1-431c95fef043
522024-08-28T17:53:38.649ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d2ef745f-4fa6-4988-81a1-431c95fef043" "--data" "/data/regions/d2ef745f-4fa6-4988-81a1-431c95fef043" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = d2ef745f-4fa6-4988-81a1-431c95fef043
532024-08-28T17:53:38.773ZINFOcrucible-agent (worker): region files created ok region = d2ef745f-4fa6-4988-81a1-431c95fef043
542024-08-28T17:53:38.773ZINFOcrucible-agent (datafile): region d2ef745f-4fa6-4988-81a1-431c95fef043 state: Requested -> Created
552024-08-28T17:53:38.773ZINFOcrucible-agent (worker): applying SMF actions post create...
562024-08-28T17:53:38.775ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d2ef745f-4fa6-4988-81a1-431c95fef043
572024-08-28T17:53:38.779ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d2ef745f-4fa6-4988-81a1-431c95fef043
582024-08-28T17:53:38.797ZINFOcrucible-agent (worker): creating config property group
592024-08-28T17:53:38.806ZINFOcrucible-agent (dropshot): request completed latency_us = 78 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:63201 req_id = b647bdc8-c3cb-4587-943f-c9cfbeb740a9 response_code = 200 uri = /crucible/0/regions
602024-08-28T17:53:38.806ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d2ef745f-4fa6-4988-81a1-431c95fef043
612024-08-28T17:53:38.806ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d2ef745f-4fa6-4988-81a1-431c95fef043
622024-08-28T17:53:38.807ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
632024-08-28T17:53:38.807ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
642024-08-28T17:53:38.807ZINFOcrucible-agent (worker): commit
652024-08-28T17:53:38.814ZINFOcrucible-agent (worker): ok!
662024-08-28T17:53:38.833ZINFOcrucible-agent (worker): SMF ok!
672024-08-28T17:58:47.592ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::12]:32345 remote_addr = [fd00:1122:3344:101::a]:60497
682024-08-28T17:58:47.592ZINFOcrucible-agent (dropshot): request completed latency_us = 589 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:60497 req_id = 3ad01484-9693-4ee4-8e48-8fdf016571e6 response_code = 200 uri = /crucible/0/regions/d2ef745f-4fa6-4988-81a1-431c95fef043
692024-08-28T17:58:47.598ZINFOcrucible-agent (datafile): path is "/data/regions/d2ef745f-4fa6-4988-81a1-431c95fef043"
702024-08-28T17:58:47.608ZINFOcrucible-agent (datafile): dataset is oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/d2ef745f-4fa6-4988-81a1-431c95fef043
712024-08-28T17:58:47.617ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/d2ef745f-4fa6-4988-81a1-431c95fef043@64cb378d-e99a-4ed4-9cad-aa2b6d4509f4
722024-08-28T17:58:47.617ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/d2ef745f-4fa6-4988-81a1-431c95fef043", "64cb378d-e99a-4ed4-9cad-aa2b6d4509f4"]
732024-08-28T17:58:47.629ZINFOcrucible-agent (dropshot): request completed latency_us = 34532 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:60497 req_id = f4eaa853-44b7-4881-9e35-b6d83a2b5482 response_code = 200 uri = /crucible/0/regions/d2ef745f-4fa6-4988-81a1-431c95fef043/snapshots/64cb378d-e99a-4ed4-9cad-aa2b6d4509f4
742024-08-28T17:58:47.635ZINFOcrucible-agent (datafile): path is "/data/regions/d2ef745f-4fa6-4988-81a1-431c95fef043"
752024-08-28T17:58:47.644ZINFOcrucible-agent (datafile): dataset is oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/d2ef745f-4fa6-4988-81a1-431c95fef043
762024-08-28T17:58:47.652ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/d2ef745f-4fa6-4988-81a1-431c95fef043@64cb378d-e99a-4ed4-9cad-aa2b6d4509f4
772024-08-28T17:58:47.652ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/d2ef745f-4fa6-4988-81a1-431c95fef043", "64cb378d-e99a-4ed4-9cad-aa2b6d4509f4"]
782024-08-28T17:58:47.662ZINFOcrucible-agent (datafile): requesting running snapshot d2ef745f-4fa6-4988-81a1-431c95fef043-64cb378d-e99a-4ed4-9cad-aa2b6d4509f4 state: Requested
792024-08-28T17:58:47.663ZINFOcrucible-agent (worker): applying SMF actions for region d2ef745f-4fa6-4988-81a1-431c95fef043 running snapshot 64cb378d-e99a-4ed4-9cad-aa2b6d4509f4 (state Requested)...
802024-08-28T17:58:47.663ZINFOcrucible-agent (dropshot): request completed latency_us = 28276 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:60497 req_id = 1ef4acd8-2eba-49de-91fb-b6350ed4de70 response_code = 200 uri = /crucible/0/regions/d2ef745f-4fa6-4988-81a1-431c95fef043/snapshots/64cb378d-e99a-4ed4-9cad-aa2b6d4509f4/run
812024-08-28T17:58:47.672ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-d2ef745f-4fa6-4988-81a1-431c95fef043-64cb378d-e99a-4ed4-9cad-aa2b6d4509f4
822024-08-28T17:58:47.679ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-d2ef745f-4fa6-4988-81a1-431c95fef043-64cb378d-e99a-4ed4-9cad-aa2b6d4509f4
832024-08-28T17:58:47.689ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-d2ef745f-4fa6-4988-81a1-431c95fef043-64cb378d-e99a-4ed4-9cad-aa2b6d4509f4
842024-08-28T17:58:47.690ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d2ef745f-4fa6-4988-81a1-431c95fef043/.zfs/snapshot/64cb378d-e99a-4ed4-9cad-aa2b6d4509f4
852024-08-28T17:58:47.690ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
862024-08-28T17:58:47.690ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
872024-08-28T17:58:47.690ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
882024-08-28T17:58:47.690ZINFOcrucible-agent (worker): commit
892024-08-28T17:58:47.693ZINFOcrucible-agent (worker): ok!
902024-08-28T17:58:47.698ZINFOcrucible-agent (worker): SMF ok!
912024-08-28T17:58:47.698ZINFOcrucible-agent (datafile): region d2ef745f-4fa6-4988-81a1-431c95fef043 running snapshot 64cb378d-e99a-4ed4-9cad-aa2b6d4509f4 state: Requested -> Created
922024-08-28T17:58:47.941ZINFOcrucible-agent (datafile): path is "/data/regions/d2ef745f-4fa6-4988-81a1-431c95fef043"
932024-08-28T17:58:47.949ZINFOcrucible-agent (datafile): dataset is oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/d2ef745f-4fa6-4988-81a1-431c95fef043
942024-08-28T17:58:47.958ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/d2ef745f-4fa6-4988-81a1-431c95fef043@64cb378d-e99a-4ed4-9cad-aa2b6d4509f4
952024-08-28T17:58:47.958ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/d2ef745f-4fa6-4988-81a1-431c95fef043", "64cb378d-e99a-4ed4-9cad-aa2b6d4509f4"]
962024-08-28T17:58:47.967ZINFOcrucible-agent (dropshot): request completed latency_us = 26092 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:60497 req_id = a4508508-f298-4d93-b64c-18eb072a23b1 response_code = 200 uri = /crucible/0/regions/d2ef745f-4fa6-4988-81a1-431c95fef043/snapshots/64cb378d-e99a-4ed4-9cad-aa2b6d4509f4/run
972024-08-28T17:58:51.446ZINFOcrucible-agent (datafile): region 21284524-66a8-4bf3-9978-c64f595edc08 state: Requested
982024-08-28T17:58:51.446ZINFOcrucible-agent (dropshot): request completed latency_us = 822 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:60497 req_id = 251b6773-144c-4a8d-abd4-9fb5c6bc3997 response_code = 200 uri = /crucible/0/regions
992024-08-28T17:58:51.446ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
1002024-08-28T17:58:51.455ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/21284524-66a8-4bf3-9978-c64f595edc08
1012024-08-28T17:58:51.455ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/21284524-66a8-4bf3-9978-c64f595edc08
1022024-08-28T17:58:51.524ZINFOcrucible-agent (worker): creating region Region { id: RegionId("21284524-66a8-4bf3-9978-c64f595edc08"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19002, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/21284524-66a8-4bf3-9978-c64f595edc08" region = 21284524-66a8-4bf3-9978-c64f595edc08
1032024-08-28T17:58:51.524ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "21284524-66a8-4bf3-9978-c64f595edc08" "--data" "/data/regions/21284524-66a8-4bf3-9978-c64f595edc08" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 21284524-66a8-4bf3-9978-c64f595edc08
1042024-08-28T17:58:51.590ZINFOcrucible-agent (worker): region files created ok region = 21284524-66a8-4bf3-9978-c64f595edc08
1052024-08-28T17:58:51.590ZINFOcrucible-agent (datafile): region 21284524-66a8-4bf3-9978-c64f595edc08 state: Requested -> Created
1062024-08-28T17:58:51.590ZINFOcrucible-agent (worker): applying SMF actions post create...
1072024-08-28T17:58:51.591ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-21284524-66a8-4bf3-9978-c64f595edc08
1082024-08-28T17:58:51.594ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-21284524-66a8-4bf3-9978-c64f595edc08
1092024-08-28T17:58:51.603ZINFOcrucible-agent (worker): creating config property group
1102024-08-28T17:58:51.607ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-21284524-66a8-4bf3-9978-c64f595edc08
1112024-08-28T17:58:51.608ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/21284524-66a8-4bf3-9978-c64f595edc08
1122024-08-28T17:58:51.608ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
1132024-08-28T17:58:51.608ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::12
1142024-08-28T17:58:51.608ZINFOcrucible-agent (worker): commit
1152024-08-28T17:58:51.613ZINFOcrucible-agent (worker): ok!
1162024-08-28T17:58:51.625ZINFOcrucible-agent (worker): SMF ok!
1172024-08-28T17:58:51.643ZINFOcrucible-agent (dropshot): request completed latency_us = 93 local_addr = [fd00:1122:3344:101::12]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:60497 req_id = 2bbebab9-56f0-4ab0-85df-aab5bff3ad5b response_code = 200 uri = /crucible/0/regions
1182024-08-28T17:59:33.599ZINFOcrucible-agent (dropshot): request completed latency_us = 55 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:60497 req_id = 68f641a3-d206-440c-aec7-e538fde6fb44 response_code = 200 uri = /crucible/0/regions/21284524-66a8-4bf3-9978-c64f595edc08
1192024-08-28T17:59:33.602ZINFOcrucible-agent (datafile): path is "/data/regions/21284524-66a8-4bf3-9978-c64f595edc08"
1202024-08-28T17:59:33.620ZINFOcrucible-agent (datafile): dataset is oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/21284524-66a8-4bf3-9978-c64f595edc08
1212024-08-28T17:59:33.629ZINFOcrucible-agent (datafile): region 21284524-66a8-4bf3-9978-c64f595edc08 state: Created -> Tombstoned
1222024-08-28T17:59:33.629ZINFOcrucible-agent (worker): applying SMF actions before removal...
1232024-08-28T17:59:33.629ZINFOcrucible-agent (dropshot): request completed latency_us = 27262 local_addr = [fd00:1122:3344:101::12]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:60497 req_id = 2748763b-d462-4f30-aa8b-a1be1c0bd2f9 response_code = 204 uri = /crucible/0/regions/21284524-66a8-4bf3-9978-c64f595edc08
1242024-08-28T17:59:33.631ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-21284524-66a8-4bf3-9978-c64f595edc08 (instance states: (Some(Online), None))
1252024-08-28T17:59:33.632ZINFOcrucible-agent (dropshot): request completed latency_us = 25 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:60497 req_id = 21aff219-3ac4-4fd8-8a77-60cc908d046f response_code = 200 uri = /crucible/0/regions/21284524-66a8-4bf3-9978-c64f595edc08
1262024-08-28T17:59:33.645ZINFOcrucible-agent (worker): SMF ok!
1272024-08-28T17:59:33.656ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_59354e76-aa85-421f-a86f-f08ce39b4a17/crucible/regions/21284524-66a8-4bf3-9978-c64f595edc08" region = 21284524-66a8-4bf3-9978-c64f595edc08
1282024-08-28T17:59:33.718ZINFOcrucible-agent (datafile): region 21284524-66a8-4bf3-9978-c64f595edc08 state: Tombstoned -> Destroyed
1292024-08-28T17:59:33.747ZINFOcrucible-agent (dropshot): request completed latency_us = 71 local_addr = [fd00:1122:3344:101::12]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:60497 req_id = 5af1f791-bbe4-42f6-be85-a4bb80d66a48 response_code = 200 uri = /crucible/0/regions/21284524-66a8-4bf3-9978-c64f595edc08