Max level shown:
1 [ Nov 20 17:17:04 Enabled. ]
2 [ Nov 20 17:17:04 Rereading configuration. ]
3 [ Nov 20 17:17:05 Rereading configuration. ]
4 [ Nov 20 17:17:10 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible -l [fd00:1122:3344:101::13]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62025-11-20T17:17:10.346ZINFOcrucible-agent: Crucible Version: Crucible Version: 0.0.1 Commit SHA: 7103cd3a3d7b0112d2949dd135db06fef0c156bb Commit timestamp: 2025-11-10T22:24:53.000000000Z branch: main rustc: 1.90.0 stable x86_64-unknown-illumos Cargo: x86_64-unknown-illumos Debug: false Opt level: 3
72025-11-20T17:17:10.347ZINFOcrucible-agent: dataset: "oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible"
82025-11-20T17:17:10.347ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::13]:32345
92025-11-20T17:17:10.347ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
102025-11-20T17:17:10.360ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
112025-11-20T17:17:10.489ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::13]:32345
122025-11-20T17:17:55.919ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::c]:39824
132025-11-20T17:17:55.973ZINFOcrucible-agent (datafile): region dd021a78-7655-457e-a499-9aaba01687b6 state: Requested
142025-11-20T17:17:55.976ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152025-11-20T17:17:55.976ZINFOcrucible-agent (dropshot): request completed latency_us = 26358 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:39824 req_id = d1398380-8112-49e1-96d0-0d5599b27c02 response_code = 200 uri = /crucible/0/regions
162025-11-20T17:17:55.986ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/dd021a78-7655-457e-a499-9aaba01687b6
172025-11-20T17:17:55.986ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/dd021a78-7655-457e-a499-9aaba01687b6
182025-11-20T17:17:56.087ZINFOcrucible-agent (worker): creating region Region { id: RegionId("dd021a78-7655-457e-a499-9aaba01687b6"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/dd021a78-7655-457e-a499-9aaba01687b6" region = dd021a78-7655-457e-a499-9aaba01687b6
192025-11-20T17:17:56.089ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "dd021a78-7655-457e-a499-9aaba01687b6" "--data" "/data/regions/dd021a78-7655-457e-a499-9aaba01687b6" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = dd021a78-7655-457e-a499-9aaba01687b6
202025-11-20T17:17:56.272ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:39824 req_id = b4ebac8a-499f-43eb-9d1d-327a5c9bc5bb response_code = 200 uri = /crucible/0/regions
212025-11-20T17:17:56.428ZINFOcrucible-agent (worker): region files created ok region = dd021a78-7655-457e-a499-9aaba01687b6
222025-11-20T17:17:56.428ZINFOcrucible-agent (datafile): region dd021a78-7655-457e-a499-9aaba01687b6 state: Requested -> Created
232025-11-20T17:17:56.428ZINFOcrucible-agent (worker): applying SMF actions post create...
242025-11-20T17:17:56.428ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-dd021a78-7655-457e-a499-9aaba01687b6
252025-11-20T17:17:56.436ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-dd021a78-7655-457e-a499-9aaba01687b6
262025-11-20T17:17:56.451ZINFOcrucible-agent (worker): creating config property group
272025-11-20T17:17:56.455ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-dd021a78-7655-457e-a499-9aaba01687b6
282025-11-20T17:17:56.456ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/dd021a78-7655-457e-a499-9aaba01687b6
292025-11-20T17:17:56.456ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
302025-11-20T17:17:56.456ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
312025-11-20T17:17:56.456ZINFOcrucible-agent (worker): commit
322025-11-20T17:17:56.461ZINFOcrucible-agent (worker): ok!
332025-11-20T17:17:56.466ZINFOcrucible-agent (worker): SMF ok!
342025-11-20T17:17:56.680ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:39824 req_id = bddce1dd-56ba-4be7-8021-6c40602eb78f response_code = 200 uri = /crucible/0/regions
352025-11-20T17:17:57.303ZINFOcrucible-agent (dropshot): request completed latency_us = 82 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:39824 req_id = 4fcce160-6eb4-43ff-8588-e5e51505568c response_code = 200 uri = /crucible/0/regions/dd021a78-7655-457e-a499-9aaba01687b6
362025-11-20T17:17:57.308ZINFOcrucible-agent (datafile): path is "/data/regions/dd021a78-7655-457e-a499-9aaba01687b6"
372025-11-20T17:17:57.316ZINFOcrucible-agent (datafile): dataset is oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/dd021a78-7655-457e-a499-9aaba01687b6
382025-11-20T17:17:57.323ZINFOcrucible-agent (datafile): region dd021a78-7655-457e-a499-9aaba01687b6 state: Created -> Tombstoned
392025-11-20T17:17:57.323ZINFOcrucible-agent (worker): applying SMF actions before removal...
402025-11-20T17:17:57.323ZINFOcrucible-agent (dropshot): request completed latency_us = 14806 local_addr = [fd00:1122:3344:101::13]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:39824 req_id = 9abdc772-46af-4ad0-9d07-3551f3ce9d37 response_code = 204 uri = /crucible/0/regions/dd021a78-7655-457e-a499-9aaba01687b6
412025-11-20T17:17:57.324ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-dd021a78-7655-457e-a499-9aaba01687b6 (instance states: (Some(Online), None))
422025-11-20T17:17:57.327ZINFOcrucible-agent (dropshot): request completed latency_us = 27 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:39824 req_id = 7cdaea7a-f42f-4664-85ea-424aa21c5d92 response_code = 200 uri = /crucible/0/regions/dd021a78-7655-457e-a499-9aaba01687b6
432025-11-20T17:17:57.329ZINFOcrucible-agent (worker): SMF ok!
442025-11-20T17:17:57.336ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/dd021a78-7655-457e-a499-9aaba01687b6" region = dd021a78-7655-457e-a499-9aaba01687b6
452025-11-20T17:17:57.406ZINFOcrucible-agent (datafile): region dd021a78-7655-457e-a499-9aaba01687b6 state: Tombstoned -> Destroyed
462025-11-20T17:17:57.452ZINFOcrucible-agent (dropshot): request completed latency_us = 56 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:39824 req_id = 84c7759e-3911-4924-b946-57a54e8e6c68 response_code = 200 uri = /crucible/0/regions/dd021a78-7655-457e-a499-9aaba01687b6
472025-11-20T17:17:59.709ZINFOcrucible-agent (datafile): region d83b2dfb-e927-4767-810f-3227f072d47b state: Requested
482025-11-20T17:17:59.709ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492025-11-20T17:17:59.709ZINFOcrucible-agent (dropshot): request completed latency_us = 258 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:39824 req_id = 2e1b1c2c-6ed3-47e2-9af9-e57c787091e3 response_code = 200 uri = /crucible/0/regions
502025-11-20T17:17:59.716ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/d83b2dfb-e927-4767-810f-3227f072d47b
512025-11-20T17:17:59.716ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/d83b2dfb-e927-4767-810f-3227f072d47b
522025-11-20T17:17:59.781ZINFOcrucible-agent (worker): creating region Region { id: RegionId("d83b2dfb-e927-4767-810f-3227f072d47b"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/d83b2dfb-e927-4767-810f-3227f072d47b" region = d83b2dfb-e927-4767-810f-3227f072d47b
532025-11-20T17:17:59.781ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "d83b2dfb-e927-4767-810f-3227f072d47b" "--data" "/data/regions/d83b2dfb-e927-4767-810f-3227f072d47b" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = d83b2dfb-e927-4767-810f-3227f072d47b
542025-11-20T17:17:59.853ZINFOcrucible-agent (worker): region files created ok region = d83b2dfb-e927-4767-810f-3227f072d47b
552025-11-20T17:17:59.853ZINFOcrucible-agent (datafile): region d83b2dfb-e927-4767-810f-3227f072d47b state: Requested -> Created
562025-11-20T17:17:59.853ZINFOcrucible-agent (worker): applying SMF actions post create...
572025-11-20T17:17:59.854ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-d83b2dfb-e927-4767-810f-3227f072d47b
582025-11-20T17:17:59.857ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-d83b2dfb-e927-4767-810f-3227f072d47b
592025-11-20T17:17:59.865ZINFOcrucible-agent (worker): creating config property group
602025-11-20T17:17:59.868ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-d83b2dfb-e927-4767-810f-3227f072d47b
612025-11-20T17:17:59.868ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d83b2dfb-e927-4767-810f-3227f072d47b
622025-11-20T17:17:59.868ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
632025-11-20T17:17:59.868ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
642025-11-20T17:17:59.868ZINFOcrucible-agent (worker): commit
652025-11-20T17:17:59.872ZINFOcrucible-agent (worker): ok!
662025-11-20T17:17:59.876ZINFOcrucible-agent (worker): SMF ok!
672025-11-20T17:18:00.052ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:39824 req_id = 31ae8390-7ce0-4c08-b770-c435e7af7557 response_code = 200 uri = /crucible/0/regions
682025-11-20T17:29:03.712ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::13]:32345 remote_addr = [fd00:1122:3344:101::c]:47176
692025-11-20T17:29:03.713ZINFOcrucible-agent (dropshot): request completed latency_us = 37 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:47176 req_id = 29b3a80b-50dc-44bb-a56a-985dafab207b response_code = 200 uri = /crucible/0/regions/d83b2dfb-e927-4767-810f-3227f072d47b
702025-11-20T17:29:03.723ZINFOcrucible-agent (datafile): path is "/data/regions/d83b2dfb-e927-4767-810f-3227f072d47b"
712025-11-20T17:29:03.730ZINFOcrucible-agent (datafile): dataset is oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/d83b2dfb-e927-4767-810f-3227f072d47b
722025-11-20T17:29:03.736ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/d83b2dfb-e927-4767-810f-3227f072d47b@4f346ed9-024c-495a-8f6b-940dcddd6e5b
732025-11-20T17:29:03.736ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/d83b2dfb-e927-4767-810f-3227f072d47b", "4f346ed9-024c-495a-8f6b-940dcddd6e5b"]
742025-11-20T17:29:03.746ZINFOcrucible-agent (dropshot): request completed latency_us = 22602 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:47176 req_id = b8930998-d295-45c1-b067-097363575a37 response_code = 200 uri = /crucible/0/regions/d83b2dfb-e927-4767-810f-3227f072d47b/snapshots/4f346ed9-024c-495a-8f6b-940dcddd6e5b
752025-11-20T17:29:03.752ZINFOcrucible-agent (datafile): path is "/data/regions/d83b2dfb-e927-4767-810f-3227f072d47b"
762025-11-20T17:29:03.757ZINFOcrucible-agent (datafile): dataset is oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/d83b2dfb-e927-4767-810f-3227f072d47b
772025-11-20T17:29:03.762ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/d83b2dfb-e927-4767-810f-3227f072d47b@4f346ed9-024c-495a-8f6b-940dcddd6e5b
782025-11-20T17:29:03.762ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/d83b2dfb-e927-4767-810f-3227f072d47b", "4f346ed9-024c-495a-8f6b-940dcddd6e5b"]
792025-11-20T17:29:03.770ZINFOcrucible-agent (datafile): requesting running snapshot d83b2dfb-e927-4767-810f-3227f072d47b-4f346ed9-024c-495a-8f6b-940dcddd6e5b state: Requested
802025-11-20T17:29:03.771ZINFOcrucible-agent (dropshot): request completed latency_us = 19305 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:47176 req_id = 73ad90fe-8558-44f4-8c5e-168615963697 response_code = 200 uri = /crucible/0/regions/d83b2dfb-e927-4767-810f-3227f072d47b/snapshots/4f346ed9-024c-495a-8f6b-940dcddd6e5b/run
812025-11-20T17:29:03.771ZINFOcrucible-agent (worker): applying SMF actions for region d83b2dfb-e927-4767-810f-3227f072d47b running snapshot 4f346ed9-024c-495a-8f6b-940dcddd6e5b (state Requested)...
822025-11-20T17:29:03.795ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-d83b2dfb-e927-4767-810f-3227f072d47b-4f346ed9-024c-495a-8f6b-940dcddd6e5b
832025-11-20T17:29:03.807ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-d83b2dfb-e927-4767-810f-3227f072d47b-4f346ed9-024c-495a-8f6b-940dcddd6e5b
842025-11-20T17:29:03.822ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-d83b2dfb-e927-4767-810f-3227f072d47b-4f346ed9-024c-495a-8f6b-940dcddd6e5b
852025-11-20T17:29:03.823ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/d83b2dfb-e927-4767-810f-3227f072d47b/.zfs/snapshot/4f346ed9-024c-495a-8f6b-940dcddd6e5b
862025-11-20T17:29:03.823ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
872025-11-20T17:29:03.823ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
882025-11-20T17:29:03.823ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
892025-11-20T17:29:03.823ZINFOcrucible-agent (worker): commit
902025-11-20T17:29:03.826ZINFOcrucible-agent (worker): ok!
912025-11-20T17:29:03.831ZINFOcrucible-agent (worker): SMF ok!
922025-11-20T17:29:03.831ZINFOcrucible-agent (datafile): region d83b2dfb-e927-4767-810f-3227f072d47b running snapshot 4f346ed9-024c-495a-8f6b-940dcddd6e5b state: Requested -> Created
932025-11-20T17:29:04.136ZINFOcrucible-agent (datafile): path is "/data/regions/d83b2dfb-e927-4767-810f-3227f072d47b"
942025-11-20T17:29:04.142ZINFOcrucible-agent (datafile): dataset is oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/d83b2dfb-e927-4767-810f-3227f072d47b
952025-11-20T17:29:04.148ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/d83b2dfb-e927-4767-810f-3227f072d47b@4f346ed9-024c-495a-8f6b-940dcddd6e5b
962025-11-20T17:29:04.148ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/d83b2dfb-e927-4767-810f-3227f072d47b", "4f346ed9-024c-495a-8f6b-940dcddd6e5b"]
972025-11-20T17:29:04.155ZINFOcrucible-agent (dropshot): request completed latency_us = 18951 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:47176 req_id = f6b9d40d-759f-432a-870e-616c4afadd19 response_code = 200 uri = /crucible/0/regions/d83b2dfb-e927-4767-810f-3227f072d47b/snapshots/4f346ed9-024c-495a-8f6b-940dcddd6e5b/run
982025-11-20T17:29:09.217ZINFOcrucible-agent (datafile): region b4b5d89d-62d1-4623-80b1-7f8b68f3320d state: Requested
992025-11-20T17:29:09.217ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
1002025-11-20T17:29:09.217ZINFOcrucible-agent (dropshot): request completed latency_us = 246 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:47176 req_id = 59df257e-7412-4511-8800-556ec0a604f1 response_code = 200 uri = /crucible/0/regions
1012025-11-20T17:29:09.224ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/b4b5d89d-62d1-4623-80b1-7f8b68f3320d
1022025-11-20T17:29:09.224ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/b4b5d89d-62d1-4623-80b1-7f8b68f3320d
1032025-11-20T17:29:09.276ZINFOcrucible-agent (worker): creating region Region { id: RegionId("b4b5d89d-62d1-4623-80b1-7f8b68f3320d"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19002, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/b4b5d89d-62d1-4623-80b1-7f8b68f3320d" region = b4b5d89d-62d1-4623-80b1-7f8b68f3320d
1042025-11-20T17:29:09.276ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "b4b5d89d-62d1-4623-80b1-7f8b68f3320d" "--data" "/data/regions/b4b5d89d-62d1-4623-80b1-7f8b68f3320d" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = b4b5d89d-62d1-4623-80b1-7f8b68f3320d
1052025-11-20T17:29:09.345ZINFOcrucible-agent (worker): region files created ok region = b4b5d89d-62d1-4623-80b1-7f8b68f3320d
1062025-11-20T17:29:09.345ZINFOcrucible-agent (datafile): region b4b5d89d-62d1-4623-80b1-7f8b68f3320d state: Requested -> Created
1072025-11-20T17:29:09.345ZINFOcrucible-agent (worker): applying SMF actions post create...
1082025-11-20T17:29:09.346ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-b4b5d89d-62d1-4623-80b1-7f8b68f3320d
1092025-11-20T17:29:09.349ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-b4b5d89d-62d1-4623-80b1-7f8b68f3320d
1102025-11-20T17:29:09.357ZINFOcrucible-agent (worker): creating config property group
1112025-11-20T17:29:09.360ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-b4b5d89d-62d1-4623-80b1-7f8b68f3320d
1122025-11-20T17:29:09.360ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/b4b5d89d-62d1-4623-80b1-7f8b68f3320d
1132025-11-20T17:29:09.360ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
1142025-11-20T17:29:09.360ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::13
1152025-11-20T17:29:09.360ZINFOcrucible-agent (worker): commit
1162025-11-20T17:29:09.364ZINFOcrucible-agent (worker): ok!
1172025-11-20T17:29:09.376ZINFOcrucible-agent (worker): SMF ok!
1182025-11-20T17:29:09.482ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::13]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:47176 req_id = 960e0df0-85cf-448e-9832-ae28b471a917 response_code = 200 uri = /crucible/0/regions
1192025-11-20T17:29:35.544ZINFOcrucible-agent (dropshot): request completed latency_us = 51 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:47176 req_id = 92f36fc2-944a-4e3c-b6f4-f0d2d3702107 response_code = 200 uri = /crucible/0/regions/b4b5d89d-62d1-4623-80b1-7f8b68f3320d
1202025-11-20T17:29:35.548ZINFOcrucible-agent (datafile): path is "/data/regions/b4b5d89d-62d1-4623-80b1-7f8b68f3320d"
1212025-11-20T17:29:35.560ZINFOcrucible-agent (datafile): dataset is oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/b4b5d89d-62d1-4623-80b1-7f8b68f3320d
1222025-11-20T17:29:35.566ZINFOcrucible-agent (datafile): region b4b5d89d-62d1-4623-80b1-7f8b68f3320d state: Created -> Tombstoned
1232025-11-20T17:29:35.566ZINFOcrucible-agent (worker): applying SMF actions before removal...
1242025-11-20T17:29:35.566ZINFOcrucible-agent (dropshot): request completed latency_us = 18640 local_addr = [fd00:1122:3344:101::13]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:47176 req_id = 9604bdec-3cd6-42bc-a2cf-61d33caa204c response_code = 204 uri = /crucible/0/regions/b4b5d89d-62d1-4623-80b1-7f8b68f3320d
1252025-11-20T17:29:35.568ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-b4b5d89d-62d1-4623-80b1-7f8b68f3320d (instance states: (Some(Online), None))
1262025-11-20T17:29:35.569ZINFOcrucible-agent (dropshot): request completed latency_us = 26 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:47176 req_id = d0ebd2b5-8149-4fbf-81d5-fd6390ab40cd response_code = 200 uri = /crucible/0/regions/b4b5d89d-62d1-4623-80b1-7f8b68f3320d
1272025-11-20T17:29:35.582ZINFOcrucible-agent (worker): SMF ok!
1282025-11-20T17:29:35.589ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_a6f08b8b-db33-4adb-bc91-ad6cab10227e/crucible/regions/b4b5d89d-62d1-4623-80b1-7f8b68f3320d" region = b4b5d89d-62d1-4623-80b1-7f8b68f3320d
1292025-11-20T17:29:35.641ZINFOcrucible-agent (datafile): region b4b5d89d-62d1-4623-80b1-7f8b68f3320d state: Tombstoned -> Destroyed
1302025-11-20T17:29:35.709ZINFOcrucible-agent (dropshot): request completed latency_us = 51 local_addr = [fd00:1122:3344:101::13]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:47176 req_id = 93bd910e-6ba5-43af-9834-a8da2b39eaa4 response_code = 200 uri = /crucible/0/regions/b4b5d89d-62d1-4623-80b1-7f8b68f3320d