Max level shown:
1 [ Mar 4 03:17:42 Enabled. ]
2 [ Mar 4 03:17:42 Rereading configuration. ]
3 [ Mar 4 03:17:49 Rereading configuration. ]
4 [ Mar 4 03:17:57 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible -l [fd00:1122:3344:101::19]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-03-04T03:17:58.032ZINFOcrucible-agent: dataset: "oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible"
72025-03-04T03:17:58.034ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::19]:32345
82025-03-04T03:17:58.035ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92025-03-04T03:17:58.040ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102025-03-04T03:17:58.142ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::19]:32345
112025-03-04T03:18:43.213ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::b]:50367
122025-03-04T03:18:43.256ZINFOcrucible-agent (datafile): region 4706a44f-3b80-4c07-aa12-84078bb59eae state: Requested
132025-03-04T03:18:43.259ZINFOcrucible-agent (dropshot): request completed latency_us = 19830 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:50367 req_id = 766300f1-95b8-4699-bc50-f99102477377 response_code = 200 uri = /crucible/0/regions
142025-03-04T03:18:43.259ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152025-03-04T03:18:43.269ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/4706a44f-3b80-4c07-aa12-84078bb59eae
162025-03-04T03:18:43.269ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/4706a44f-3b80-4c07-aa12-84078bb59eae
172025-03-04T03:18:43.324ZINFOcrucible-agent (worker): creating region Region { id: RegionId("4706a44f-3b80-4c07-aa12-84078bb59eae"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/4706a44f-3b80-4c07-aa12-84078bb59eae" region = 4706a44f-3b80-4c07-aa12-84078bb59eae
182025-03-04T03:18:43.326ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "4706a44f-3b80-4c07-aa12-84078bb59eae" "--data" "/data/regions/4706a44f-3b80-4c07-aa12-84078bb59eae" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 4706a44f-3b80-4c07-aa12-84078bb59eae
192025-03-04T03:18:43.406ZINFOcrucible-agent (dropshot): request completed latency_us = 75 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:50367 req_id = abb66930-00e2-468b-b226-6e2c2da278f3 response_code = 200 uri = /crucible/0/regions
202025-03-04T03:18:43.629ZINFOcrucible-agent (worker): region files created ok region = 4706a44f-3b80-4c07-aa12-84078bb59eae
212025-03-04T03:18:43.629ZINFOcrucible-agent (datafile): region 4706a44f-3b80-4c07-aa12-84078bb59eae state: Requested -> Created
222025-03-04T03:18:43.629ZINFOcrucible-agent (worker): applying SMF actions post create...
232025-03-04T03:18:43.630ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-4706a44f-3b80-4c07-aa12-84078bb59eae
242025-03-04T03:18:43.635ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-4706a44f-3b80-4c07-aa12-84078bb59eae
252025-03-04T03:18:43.648ZINFOcrucible-agent (worker): creating config property group
262025-03-04T03:18:43.651ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-4706a44f-3b80-4c07-aa12-84078bb59eae
272025-03-04T03:18:43.652ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4706a44f-3b80-4c07-aa12-84078bb59eae
282025-03-04T03:18:43.652ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292025-03-04T03:18:43.652ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
302025-03-04T03:18:43.652ZINFOcrucible-agent (worker): commit
312025-03-04T03:18:43.658ZINFOcrucible-agent (worker): ok!
322025-03-04T03:18:43.673ZINFOcrucible-agent (worker): SMF ok!
332025-03-04T03:18:43.687ZINFOcrucible-agent (dropshot): request completed latency_us = 48 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:50367 req_id = eb191157-d4d6-4860-88b6-38cffe77599b response_code = 200 uri = /crucible/0/regions
342025-03-04T03:18:44.634ZINFOcrucible-agent (dropshot): request completed latency_us = 104 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:50367 req_id = 609c60a9-9e34-4440-8d75-111f75dcca52 response_code = 200 uri = /crucible/0/regions/4706a44f-3b80-4c07-aa12-84078bb59eae
352025-03-04T03:18:44.637ZINFOcrucible-agent (datafile): path is "/data/regions/4706a44f-3b80-4c07-aa12-84078bb59eae"
362025-03-04T03:18:44.648ZINFOcrucible-agent (datafile): dataset is oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/4706a44f-3b80-4c07-aa12-84078bb59eae
372025-03-04T03:18:44.658ZINFOcrucible-agent (datafile): region 4706a44f-3b80-4c07-aa12-84078bb59eae state: Created -> Tombstoned
382025-03-04T03:18:44.658ZINFOcrucible-agent (worker): applying SMF actions before removal...
392025-03-04T03:18:44.658ZINFOcrucible-agent (dropshot): request completed latency_us = 21074 local_addr = [fd00:1122:3344:101::19]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:50367 req_id = 45e9e52c-b847-414d-9a56-6e9f8fcc4c48 response_code = 204 uri = /crucible/0/regions/4706a44f-3b80-4c07-aa12-84078bb59eae
402025-03-04T03:18:44.659ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-4706a44f-3b80-4c07-aa12-84078bb59eae (instance states: (Some(Online), None))
412025-03-04T03:18:44.662ZINFOcrucible-agent (dropshot): request completed latency_us = 32 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:50367 req_id = eeb0bbac-fdc5-4aa5-be93-56fdca38cf1d response_code = 200 uri = /crucible/0/regions/4706a44f-3b80-4c07-aa12-84078bb59eae
422025-03-04T03:18:44.669ZINFOcrucible-agent (worker): SMF ok!
432025-03-04T03:18:44.681ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/4706a44f-3b80-4c07-aa12-84078bb59eae" region = 4706a44f-3b80-4c07-aa12-84078bb59eae
442025-03-04T03:18:44.749ZINFOcrucible-agent (datafile): region 4706a44f-3b80-4c07-aa12-84078bb59eae state: Tombstoned -> Destroyed
452025-03-04T03:18:44.780ZINFOcrucible-agent (dropshot): request completed latency_us = 66 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:50367 req_id = 37385ea6-ab3b-4002-b9f9-073f91ae95e3 response_code = 200 uri = /crucible/0/regions/4706a44f-3b80-4c07-aa12-84078bb59eae
462025-03-04T03:18:47.132ZINFOcrucible-agent (datafile): region 8b787d6a-addb-4f24-9be8-65c6ddba91c5 state: Requested
472025-03-04T03:18:47.133ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
482025-03-04T03:18:47.133ZINFOcrucible-agent (dropshot): request completed latency_us = 214 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:50367 req_id = c9f0743b-7ef5-41c6-9eb7-821335b77b4d response_code = 200 uri = /crucible/0/regions
492025-03-04T03:18:47.141ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5
502025-03-04T03:18:47.141ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5
512025-03-04T03:18:47.196ZINFOcrucible-agent (worker): creating region Region { id: RegionId("8b787d6a-addb-4f24-9be8-65c6ddba91c5"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5" region = 8b787d6a-addb-4f24-9be8-65c6ddba91c5
522025-03-04T03:18:47.198ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "8b787d6a-addb-4f24-9be8-65c6ddba91c5" "--data" "/data/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 8b787d6a-addb-4f24-9be8-65c6ddba91c5
532025-03-04T03:18:47.294ZINFOcrucible-agent (worker): region files created ok region = 8b787d6a-addb-4f24-9be8-65c6ddba91c5
542025-03-04T03:18:47.294ZINFOcrucible-agent (datafile): region 8b787d6a-addb-4f24-9be8-65c6ddba91c5 state: Requested -> Created
552025-03-04T03:18:47.294ZINFOcrucible-agent (worker): applying SMF actions post create...
562025-03-04T03:18:47.295ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-8b787d6a-addb-4f24-9be8-65c6ddba91c5
572025-03-04T03:18:47.300ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-8b787d6a-addb-4f24-9be8-65c6ddba91c5
582025-03-04T03:18:47.302ZINFOcrucible-agent (dropshot): request completed latency_us = 83 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:50367 req_id = 6d406dfb-a952-46e7-b237-8d664a92641d response_code = 200 uri = /crucible/0/regions
592025-03-04T03:18:47.311ZINFOcrucible-agent (worker): creating config property group
602025-03-04T03:18:47.318ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-8b787d6a-addb-4f24-9be8-65c6ddba91c5
612025-03-04T03:18:47.318ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5
622025-03-04T03:18:47.318ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
632025-03-04T03:18:47.318ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
642025-03-04T03:18:47.319ZINFOcrucible-agent (worker): commit
652025-03-04T03:18:47.325ZINFOcrucible-agent (worker): ok!
662025-03-04T03:18:47.331ZINFOcrucible-agent (worker): SMF ok!
672025-03-04T03:29:04.449ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::b]:38961
682025-03-04T03:29:04.449ZINFOcrucible-agent (dropshot): request completed latency_us = 49 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:38961 req_id = 2d2e242a-6061-4b91-b554-83b879c2486e response_code = 200 uri = /crucible/0/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5
692025-03-04T03:29:04.455ZINFOcrucible-agent (datafile): path is "/data/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5"
702025-03-04T03:29:04.464ZINFOcrucible-agent (datafile): dataset is oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5
712025-03-04T03:29:04.474ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5@55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2
722025-03-04T03:29:04.474ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5", "55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2"]
732025-03-04T03:29:04.483ZINFOcrucible-agent (dropshot): request completed latency_us = 30036 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:38961 req_id = da9e4c6c-aaa0-47aa-973b-ad603d7be7c7 response_code = 200 uri = /crucible/0/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5/snapshots/55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2
742025-03-04T03:29:04.486ZINFOcrucible-agent (datafile): path is "/data/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5"
752025-03-04T03:29:04.495ZINFOcrucible-agent (datafile): dataset is oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5
762025-03-04T03:29:04.502ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5@55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2
772025-03-04T03:29:04.502ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5", "55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2"]
782025-03-04T03:29:04.511ZINFOcrucible-agent (datafile): requesting running snapshot 8b787d6a-addb-4f24-9be8-65c6ddba91c5-55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2 state: Requested
792025-03-04T03:29:04.512ZINFOcrucible-agent (worker): applying SMF actions for region 8b787d6a-addb-4f24-9be8-65c6ddba91c5 running snapshot 55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2 (state Requested)...
802025-03-04T03:29:04.512ZINFOcrucible-agent (dropshot): request completed latency_us = 25689 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:38961 req_id = c772f219-5e9c-4acd-9d46-f7681f3f3cb9 response_code = 200 uri = /crucible/0/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5/snapshots/55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2/run
812025-03-04T03:29:04.519ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-8b787d6a-addb-4f24-9be8-65c6ddba91c5-55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2
822025-03-04T03:29:04.530ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-8b787d6a-addb-4f24-9be8-65c6ddba91c5-55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2
832025-03-04T03:29:04.542ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-8b787d6a-addb-4f24-9be8-65c6ddba91c5-55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2
842025-03-04T03:29:04.542ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5/.zfs/snapshot/55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2
852025-03-04T03:29:04.542ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
862025-03-04T03:29:04.542ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
872025-03-04T03:29:04.542ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
882025-03-04T03:29:04.542ZINFOcrucible-agent (worker): commit
892025-03-04T03:29:04.546ZINFOcrucible-agent (worker): ok!
902025-03-04T03:29:04.551ZINFOcrucible-agent (worker): SMF ok!
912025-03-04T03:29:04.551ZINFOcrucible-agent (datafile): region 8b787d6a-addb-4f24-9be8-65c6ddba91c5 running snapshot 55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2 state: Requested -> Created
922025-03-04T03:29:04.801ZINFOcrucible-agent (datafile): path is "/data/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5"
932025-03-04T03:29:04.810ZINFOcrucible-agent (datafile): dataset is oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5
942025-03-04T03:29:04.818ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5@55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2
952025-03-04T03:29:04.818ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5", "55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2"]
962025-03-04T03:29:04.826ZINFOcrucible-agent (dropshot): request completed latency_us = 24875 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:38961 req_id = a485efec-5f38-494b-9e99-7c1c3dc4824c response_code = 200 uri = /crucible/0/regions/8b787d6a-addb-4f24-9be8-65c6ddba91c5/snapshots/55fb3511-44f6-4fe1-ac53-f4bc63e8d6b2/run
972025-03-04T03:29:08.611ZINFOcrucible-agent (datafile): region 5d517826-5119-4f29-aff8-af52b250211f state: Requested
982025-03-04T03:29:08.612ZINFOcrucible-agent (dropshot): request completed latency_us = 248 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:38961 req_id = 7f285a47-228f-4e30-a0b7-347869874d10 response_code = 200 uri = /crucible/0/regions
992025-03-04T03:29:08.612ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
1002025-03-04T03:29:08.621ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/5d517826-5119-4f29-aff8-af52b250211f
1012025-03-04T03:29:08.621ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/5d517826-5119-4f29-aff8-af52b250211f
1022025-03-04T03:29:08.681ZINFOcrucible-agent (worker): creating region Region { id: RegionId("5d517826-5119-4f29-aff8-af52b250211f"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19002, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/5d517826-5119-4f29-aff8-af52b250211f" region = 5d517826-5119-4f29-aff8-af52b250211f
1032025-03-04T03:29:08.683ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "5d517826-5119-4f29-aff8-af52b250211f" "--data" "/data/regions/5d517826-5119-4f29-aff8-af52b250211f" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 5d517826-5119-4f29-aff8-af52b250211f
1042025-03-04T03:29:08.769ZINFOcrucible-agent (worker): region files created ok region = 5d517826-5119-4f29-aff8-af52b250211f
1052025-03-04T03:29:08.769ZINFOcrucible-agent (datafile): region 5d517826-5119-4f29-aff8-af52b250211f state: Requested -> Created
1062025-03-04T03:29:08.769ZINFOcrucible-agent (worker): applying SMF actions post create...
1072025-03-04T03:29:08.770ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-5d517826-5119-4f29-aff8-af52b250211f
1082025-03-04T03:29:08.773ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-5d517826-5119-4f29-aff8-af52b250211f
1092025-03-04T03:29:08.780ZINFOcrucible-agent (worker): creating config property group
1102025-03-04T03:29:08.785ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-5d517826-5119-4f29-aff8-af52b250211f
1112025-03-04T03:29:08.785ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/5d517826-5119-4f29-aff8-af52b250211f
1122025-03-04T03:29:08.785ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
1132025-03-04T03:29:08.785ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::19
1142025-03-04T03:29:08.785ZINFOcrucible-agent (worker): commit
1152025-03-04T03:29:08.789ZINFOcrucible-agent (worker): ok!
1162025-03-04T03:29:08.800ZINFOcrucible-agent (worker): SMF ok!
1172025-03-04T03:29:09.085ZINFOcrucible-agent (dropshot): request completed latency_us = 60 local_addr = [fd00:1122:3344:101::19]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:38961 req_id = 64142d4f-8ab7-44a9-9568-73e97e9479f1 response_code = 200 uri = /crucible/0/regions
1182025-03-04T03:29:09.181ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::19]:32345 remote_addr = [fd00:1122:3344:101::b]:46462
1192025-03-04T03:29:09.181ZINFOcrucible-agent (dropshot): request completed latency_us = 44 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:46462 req_id = 7c20c2c2-0fe3-4e73-981f-b70ed88f368a response_code = 200 uri = /crucible/0/regions/5d517826-5119-4f29-aff8-af52b250211f
1202025-03-04T03:29:35.025ZINFOcrucible-agent (dropshot): request completed latency_us = 53 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:38961 req_id = 1ffbe980-90a7-4460-acc0-f9dfbc948318 response_code = 200 uri = /crucible/0/regions/5d517826-5119-4f29-aff8-af52b250211f
1212025-03-04T03:29:35.029ZINFOcrucible-agent (datafile): path is "/data/regions/5d517826-5119-4f29-aff8-af52b250211f"
1222025-03-04T03:29:35.041ZINFOcrucible-agent (datafile): dataset is oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/5d517826-5119-4f29-aff8-af52b250211f
1232025-03-04T03:29:35.052ZINFOcrucible-agent (datafile): region 5d517826-5119-4f29-aff8-af52b250211f state: Created -> Tombstoned
1242025-03-04T03:29:35.052ZINFOcrucible-agent (worker): applying SMF actions before removal...
1252025-03-04T03:29:35.052ZINFOcrucible-agent (dropshot): request completed latency_us = 23044 local_addr = [fd00:1122:3344:101::19]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:38961 req_id = bfca51d4-9ad1-466f-a1c9-7798ad3266fc response_code = 204 uri = /crucible/0/regions/5d517826-5119-4f29-aff8-af52b250211f
1262025-03-04T03:29:35.054ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-5d517826-5119-4f29-aff8-af52b250211f (instance states: (Some(Online), None))
1272025-03-04T03:29:35.055ZINFOcrucible-agent (dropshot): request completed latency_us = 28 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:38961 req_id = 994d8a2a-51b0-4b8e-bfe5-c4abd0fc093a response_code = 200 uri = /crucible/0/regions/5d517826-5119-4f29-aff8-af52b250211f
1282025-03-04T03:29:35.060ZINFOcrucible-agent (worker): SMF ok!
1292025-03-04T03:29:35.072ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_0c4184e0-2976-440b-bd79-f20b4a02ca26/crucible/regions/5d517826-5119-4f29-aff8-af52b250211f" region = 5d517826-5119-4f29-aff8-af52b250211f
1302025-03-04T03:29:35.124ZINFOcrucible-agent (datafile): region 5d517826-5119-4f29-aff8-af52b250211f state: Tombstoned -> Destroyed
1312025-03-04T03:29:35.197ZINFOcrucible-agent (dropshot): request completed latency_us = 60 local_addr = [fd00:1122:3344:101::19]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:38961 req_id = cb5442f1-cf43-438a-99e4-73bf5b620943 response_code = 200 uri = /crucible/0/regions/5d517826-5119-4f29-aff8-af52b250211f