Max level shown:
1 [ Mar 11 22:15:48 Enabled. ]
2 [ Mar 11 22:15:48 Rereading configuration. ]
3 [ Mar 11 22:15:50 Rereading configuration. ]
4 [ Mar 11 22:15:55 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible -l [fd00:1122:3344:101::14]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62026-03-11T22:15:55.708ZINFOcrucible-agent: Crucible Version: Crucible Version: 0.0.1 Commit SHA: 7103cd3a3d7b0112d2949dd135db06fef0c156bb Commit timestamp: 2025-11-10T22:24:53.000000000Z branch: main rustc: 1.90.0 stable x86_64-unknown-illumos Cargo: x86_64-unknown-illumos Debug: false Opt level: 3
72026-03-11T22:15:55.709ZINFOcrucible-agent: dataset: "oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible"
82026-03-11T22:15:55.709ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::14]:32345
92026-03-11T22:15:55.709ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
102026-03-11T22:15:55.722ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
112026-03-11T22:15:55.862ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::14]:32345
122026-03-11T22:16:25.578ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::b]:50600
132026-03-11T22:16:25.610ZINFOcrucible-agent (datafile): region 94ab338b-919c-4baa-8f60-0b08f33fde3b state: Requested
142026-03-11T22:16:25.613ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
152026-03-11T22:16:25.613ZINFOcrucible-agent (dropshot): request completed latency_us = 18546 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:50600 req_id = 6f4475bc-6e56-4a32-a866-5b24b3ef1a35 response_code = 200 uri = /crucible/0/regions
162026-03-11T22:16:25.622ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b
172026-03-11T22:16:25.622ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b
182026-03-11T22:16:25.713ZINFOcrucible-agent (worker): creating region Region { id: RegionId("94ab338b-919c-4baa-8f60-0b08f33fde3b"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b" region = 94ab338b-919c-4baa-8f60-0b08f33fde3b
192026-03-11T22:16:25.715ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "94ab338b-919c-4baa-8f60-0b08f33fde3b" "--data" "/data/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 94ab338b-919c-4baa-8f60-0b08f33fde3b
202026-03-11T22:16:25.931ZINFOcrucible-agent (dropshot): request completed latency_us = 67 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:50600 req_id = 6f60d648-3484-4c8d-8be4-ded8169e3b56 response_code = 200 uri = /crucible/0/regions
212026-03-11T22:16:26.103ZINFOcrucible-agent (worker): region files created ok region = 94ab338b-919c-4baa-8f60-0b08f33fde3b
222026-03-11T22:16:26.103ZINFOcrucible-agent (datafile): region 94ab338b-919c-4baa-8f60-0b08f33fde3b state: Requested -> Created
232026-03-11T22:16:26.103ZINFOcrucible-agent (worker): applying SMF actions post create...
242026-03-11T22:16:26.104ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-94ab338b-919c-4baa-8f60-0b08f33fde3b
252026-03-11T22:16:26.110ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-94ab338b-919c-4baa-8f60-0b08f33fde3b
262026-03-11T22:16:26.126ZINFOcrucible-agent (worker): creating config property group
272026-03-11T22:16:26.129ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-94ab338b-919c-4baa-8f60-0b08f33fde3b
282026-03-11T22:16:26.130ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b
292026-03-11T22:16:26.130ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
302026-03-11T22:16:26.130ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
312026-03-11T22:16:26.130ZINFOcrucible-agent (worker): commit
322026-03-11T22:16:26.134ZINFOcrucible-agent (worker): ok!
332026-03-11T22:16:26.139ZINFOcrucible-agent (worker): SMF ok!
342026-03-11T22:16:26.349ZINFOcrucible-agent (dropshot): request completed latency_us = 64 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:50600 req_id = d9754425-73e0-49a6-84ed-f0bc1c030b42 response_code = 200 uri = /crucible/0/regions
352026-03-11T22:28:05.583ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::b]:52324
362026-03-11T22:28:05.583ZINFOcrucible-agent (datafile): region 47e47cf5-d22c-45c2-b72d-38d7db4def05 state: Requested
372026-03-11T22:28:05.584ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
382026-03-11T22:28:05.584ZINFOcrucible-agent (dropshot): request completed latency_us = 757 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:52324 req_id = d7c5b1f6-e0f8-4da4-8e3a-4239f9f824cd response_code = 200 uri = /crucible/0/regions
392026-03-11T22:28:05.593ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/47e47cf5-d22c-45c2-b72d-38d7db4def05
402026-03-11T22:28:05.593ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/47e47cf5-d22c-45c2-b72d-38d7db4def05
412026-03-11T22:28:05.665ZINFOcrucible-agent (worker): creating region Region { id: RegionId("47e47cf5-d22c-45c2-b72d-38d7db4def05"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19001, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/47e47cf5-d22c-45c2-b72d-38d7db4def05" region = 47e47cf5-d22c-45c2-b72d-38d7db4def05
422026-03-11T22:28:05.665ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "47e47cf5-d22c-45c2-b72d-38d7db4def05" "--data" "/data/regions/47e47cf5-d22c-45c2-b72d-38d7db4def05" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 47e47cf5-d22c-45c2-b72d-38d7db4def05
432026-03-11T22:28:05.816ZINFOcrucible-agent (worker): region files created ok region = 47e47cf5-d22c-45c2-b72d-38d7db4def05
442026-03-11T22:28:05.816ZINFOcrucible-agent (datafile): region 47e47cf5-d22c-45c2-b72d-38d7db4def05 state: Requested -> Created
452026-03-11T22:28:05.816ZINFOcrucible-agent (worker): applying SMF actions post create...
462026-03-11T22:28:05.816ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-47e47cf5-d22c-45c2-b72d-38d7db4def05
472026-03-11T22:28:05.834ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-47e47cf5-d22c-45c2-b72d-38d7db4def05
482026-03-11T22:28:05.865ZINFOcrucible-agent (worker): creating config property group
492026-03-11T22:28:05.873ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-47e47cf5-d22c-45c2-b72d-38d7db4def05
502026-03-11T22:28:05.874ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/47e47cf5-d22c-45c2-b72d-38d7db4def05
512026-03-11T22:28:05.874ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
522026-03-11T22:28:05.874ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
532026-03-11T22:28:05.874ZINFOcrucible-agent (worker): commit
542026-03-11T22:28:05.881ZINFOcrucible-agent (worker): ok!
552026-03-11T22:28:05.901ZINFOcrucible-agent (dropshot): request completed latency_us = 67 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:52324 req_id = 5a017e49-c226-4475-ad82-c7c99c00eae5 response_code = 200 uri = /crucible/0/regions
562026-03-11T22:28:05.906ZINFOcrucible-agent (worker): SMF ok!
572026-03-11T22:28:07.074ZINFOcrucible-agent (dropshot): request completed latency_us = 80 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:52324 req_id = b352a35c-726f-4a41-99d8-89d946e64928 response_code = 200 uri = /crucible/0/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b
582026-03-11T22:28:07.078ZINFOcrucible-agent (datafile): path is "/data/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b"
592026-03-11T22:28:07.084ZINFOcrucible-agent (datafile): dataset is oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b
602026-03-11T22:28:07.089ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b@a24c7a39-7237-474e-ad7f-0f7e0e35068f
612026-03-11T22:28:07.089ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b", "a24c7a39-7237-474e-ad7f-0f7e0e35068f"]
622026-03-11T22:28:07.098ZINFOcrucible-agent (dropshot): request completed latency_us = 20649 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:52324 req_id = 0f8aed26-ee9a-49d8-8488-d3897d14121f response_code = 200 uri = /crucible/0/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b/snapshots/a24c7a39-7237-474e-ad7f-0f7e0e35068f
632026-03-11T22:28:07.103ZINFOcrucible-agent (datafile): path is "/data/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b"
642026-03-11T22:28:07.109ZINFOcrucible-agent (datafile): dataset is oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b
652026-03-11T22:28:07.115ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b@a24c7a39-7237-474e-ad7f-0f7e0e35068f
662026-03-11T22:28:07.115ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b", "a24c7a39-7237-474e-ad7f-0f7e0e35068f"]
672026-03-11T22:28:07.123ZINFOcrucible-agent (datafile): requesting running snapshot 94ab338b-919c-4baa-8f60-0b08f33fde3b-a24c7a39-7237-474e-ad7f-0f7e0e35068f state: Requested
682026-03-11T22:28:07.123ZINFOcrucible-agent (dropshot): request completed latency_us = 20381 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:52324 req_id = 80d1b8a1-004b-4442-8fa3-9a9e87551f18 response_code = 200 uri = /crucible/0/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b/snapshots/a24c7a39-7237-474e-ad7f-0f7e0e35068f/run
692026-03-11T22:28:07.123ZINFOcrucible-agent (worker): applying SMF actions for region 94ab338b-919c-4baa-8f60-0b08f33fde3b running snapshot a24c7a39-7237-474e-ad7f-0f7e0e35068f (state Requested)...
702026-03-11T22:28:07.128ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-94ab338b-919c-4baa-8f60-0b08f33fde3b-a24c7a39-7237-474e-ad7f-0f7e0e35068f
712026-03-11T22:28:07.145ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-94ab338b-919c-4baa-8f60-0b08f33fde3b-a24c7a39-7237-474e-ad7f-0f7e0e35068f
722026-03-11T22:28:07.167ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-94ab338b-919c-4baa-8f60-0b08f33fde3b-a24c7a39-7237-474e-ad7f-0f7e0e35068f
732026-03-11T22:28:07.167ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b/.zfs/snapshot/a24c7a39-7237-474e-ad7f-0f7e0e35068f
742026-03-11T22:28:07.167ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
752026-03-11T22:28:07.167ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
762026-03-11T22:28:07.167ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
772026-03-11T22:28:07.167ZINFOcrucible-agent (worker): commit
782026-03-11T22:28:07.174ZINFOcrucible-agent (worker): ok!
792026-03-11T22:28:07.183ZINFOcrucible-agent (worker): SMF ok!
802026-03-11T22:28:07.183ZINFOcrucible-agent (datafile): region 94ab338b-919c-4baa-8f60-0b08f33fde3b running snapshot a24c7a39-7237-474e-ad7f-0f7e0e35068f state: Requested -> Created
812026-03-11T22:28:07.446ZINFOcrucible-agent (datafile): path is "/data/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b"
822026-03-11T22:28:07.452ZINFOcrucible-agent (datafile): dataset is oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b
832026-03-11T22:28:07.459ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b@a24c7a39-7237-474e-ad7f-0f7e0e35068f
842026-03-11T22:28:07.459ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b", "a24c7a39-7237-474e-ad7f-0f7e0e35068f"]
852026-03-11T22:28:07.465ZINFOcrucible-agent (dropshot): request completed latency_us = 19441 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:52324 req_id = 9a1e3b16-be9d-44d8-be82-76ef1cb0e08c response_code = 200 uri = /crucible/0/regions/94ab338b-919c-4baa-8f60-0b08f33fde3b/snapshots/a24c7a39-7237-474e-ad7f-0f7e0e35068f/run
862026-03-11T22:28:14.308ZINFOcrucible-agent (datafile): region fcfe8209-39e9-4d83-a957-8f88275d1031 state: Requested
872026-03-11T22:28:14.308ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
882026-03-11T22:28:14.309ZINFOcrucible-agent (dropshot): request completed latency_us = 216 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:52324 req_id = 965f3309-e5bf-4dfa-8f2d-9039ed1709f7 response_code = 200 uri = /crucible/0/regions
892026-03-11T22:28:14.315ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/fcfe8209-39e9-4d83-a957-8f88275d1031
902026-03-11T22:28:14.315ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/fcfe8209-39e9-4d83-a957-8f88275d1031
912026-03-11T22:28:14.378ZINFOcrucible-agent (worker): creating region Region { id: RegionId("fcfe8209-39e9-4d83-a957-8f88275d1031"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19003, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/fcfe8209-39e9-4d83-a957-8f88275d1031" region = fcfe8209-39e9-4d83-a957-8f88275d1031
922026-03-11T22:28:14.379ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "fcfe8209-39e9-4d83-a957-8f88275d1031" "--data" "/data/regions/fcfe8209-39e9-4d83-a957-8f88275d1031" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = fcfe8209-39e9-4d83-a957-8f88275d1031
932026-03-11T22:28:14.473ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:52324 req_id = 61b89473-aa3a-41b0-9ccb-caeea88d7919 response_code = 200 uri = /crucible/0/regions
942026-03-11T22:28:14.595ZINFOcrucible-agent (worker): region files created ok region = fcfe8209-39e9-4d83-a957-8f88275d1031
952026-03-11T22:28:14.595ZINFOcrucible-agent (datafile): region fcfe8209-39e9-4d83-a957-8f88275d1031 state: Requested -> Created
962026-03-11T22:28:14.595ZINFOcrucible-agent (worker): applying SMF actions post create...
972026-03-11T22:28:14.597ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-fcfe8209-39e9-4d83-a957-8f88275d1031
982026-03-11T22:28:14.601ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-fcfe8209-39e9-4d83-a957-8f88275d1031
992026-03-11T22:28:14.616ZINFOcrucible-agent (worker): creating config property group
1002026-03-11T22:28:14.624ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-fcfe8209-39e9-4d83-a957-8f88275d1031
1012026-03-11T22:28:14.624ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/fcfe8209-39e9-4d83-a957-8f88275d1031
1022026-03-11T22:28:14.624ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19003
1032026-03-11T22:28:14.624ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
1042026-03-11T22:28:14.624ZINFOcrucible-agent (worker): commit
1052026-03-11T22:28:14.631ZINFOcrucible-agent (worker): ok!
1062026-03-11T22:28:14.641ZINFOcrucible-agent (worker): SMF ok!
1072026-03-11T22:28:15.132ZINFOcrucible-agent (dropshot): request completed latency_us = 74 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:52324 req_id = 8c19c077-7b26-4b35-b964-56627c04040f response_code = 200 uri = /crucible/0/regions
1082026-03-11T22:28:47.408ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::b]:56708
1092026-03-11T22:28:47.408ZINFOcrucible-agent (dropshot): request completed latency_us = 61 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:56708 req_id = 19e14ce2-915f-41ef-b83b-8f9bda3a6b72 response_code = 200 uri = /crucible/0/regions/fcfe8209-39e9-4d83-a957-8f88275d1031
1102026-03-11T22:28:47.411ZINFOcrucible-agent (datafile): path is "/data/regions/fcfe8209-39e9-4d83-a957-8f88275d1031"
1112026-03-11T22:28:47.422ZINFOcrucible-agent (datafile): dataset is oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/fcfe8209-39e9-4d83-a957-8f88275d1031
1122026-03-11T22:28:47.429ZINFOcrucible-agent (datafile): region fcfe8209-39e9-4d83-a957-8f88275d1031 state: Created -> Tombstoned
1132026-03-11T22:28:47.429ZINFOcrucible-agent (worker): applying SMF actions before removal...
1142026-03-11T22:28:47.429ZINFOcrucible-agent (dropshot): request completed latency_us = 18547 local_addr = [fd00:1122:3344:101::14]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:56708 req_id = 10e89897-2ba0-493a-a37d-d659c3d8775d response_code = 204 uri = /crucible/0/regions/fcfe8209-39e9-4d83-a957-8f88275d1031
1152026-03-11T22:28:47.430ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-fcfe8209-39e9-4d83-a957-8f88275d1031 (instance states: (Some(Online), None))
1162026-03-11T22:28:47.433ZINFOcrucible-agent (dropshot): request completed latency_us = 36 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:56708 req_id = ee111d53-1f8c-4844-bcc7-a5b43fac0fad response_code = 200 uri = /crucible/0/regions/fcfe8209-39e9-4d83-a957-8f88275d1031
1172026-03-11T22:28:47.445ZINFOcrucible-agent (worker): SMF ok!
1182026-03-11T22:28:47.453ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_35581ec7-b56d-4287-a4bd-df3b3177fd50/crucible/regions/fcfe8209-39e9-4d83-a957-8f88275d1031" region = fcfe8209-39e9-4d83-a957-8f88275d1031
1192026-03-11T22:28:47.502ZINFOcrucible-agent (datafile): region fcfe8209-39e9-4d83-a957-8f88275d1031 state: Tombstoned -> Destroyed
1202026-03-11T22:28:47.572ZINFOcrucible-agent (dropshot): request completed latency_us = 62 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:56708 req_id = eaa4bb51-87d5-461e-bbdc-3996d20d0d67 response_code = 200 uri = /crucible/0/regions/fcfe8209-39e9-4d83-a957-8f88275d1031