Max level shown:
1 [ Oct 3 20:31:48 Enabled. ]
2 [ Oct 3 20:31:48 Rereading configuration. ]
3 [ Oct 3 20:31:50 Rereading configuration. ]
4 [ Oct 3 20:31:52 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible -l [fd00:1122:3344:101::14]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-10-03T20:31:52.907ZINFOcrucible-agent: dataset: "oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible"
72024-10-03T20:31:52.909ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::14]:32345
82024-10-03T20:31:52.909ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-10-03T20:31:52.921ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-10-03T20:31:53.040ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::14]:32345
112024-10-03T20:32:42.220ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::b]:37202
122024-10-03T20:32:42.250ZINFOcrucible-agent (datafile): region 65e69107-8969-458f-82a7-9eb00e23f97f state: Requested
132024-10-03T20:32:42.254ZINFOcrucible-agent (dropshot): request completed latency_us = 12247 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:37202 req_id = 316ae1d0-0711-4ea2-9cef-b68ffdadeaa7 response_code = 200 uri = /crucible/0/regions
142024-10-03T20:32:42.256ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152024-10-03T20:32:42.265ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/65e69107-8969-458f-82a7-9eb00e23f97f
162024-10-03T20:32:42.265ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/65e69107-8969-458f-82a7-9eb00e23f97f
172024-10-03T20:32:42.324ZINFOcrucible-agent (worker): creating region Region { id: RegionId("65e69107-8969-458f-82a7-9eb00e23f97f"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/65e69107-8969-458f-82a7-9eb00e23f97f" region = 65e69107-8969-458f-82a7-9eb00e23f97f
182024-10-03T20:32:42.328ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "65e69107-8969-458f-82a7-9eb00e23f97f" "--data" "/data/regions/65e69107-8969-458f-82a7-9eb00e23f97f" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 65e69107-8969-458f-82a7-9eb00e23f97f
192024-10-03T20:32:42.443ZINFOcrucible-agent (worker): region files created ok region = 65e69107-8969-458f-82a7-9eb00e23f97f
202024-10-03T20:32:42.443ZINFOcrucible-agent (datafile): region 65e69107-8969-458f-82a7-9eb00e23f97f state: Requested -> Created
212024-10-03T20:32:42.443ZINFOcrucible-agent (worker): applying SMF actions post create...
222024-10-03T20:32:42.445ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-65e69107-8969-458f-82a7-9eb00e23f97f
232024-10-03T20:32:42.449ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-65e69107-8969-458f-82a7-9eb00e23f97f
242024-10-03T20:32:42.467ZINFOcrucible-agent (worker): creating config property group
252024-10-03T20:32:42.470ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-65e69107-8969-458f-82a7-9eb00e23f97f
262024-10-03T20:32:42.475ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/65e69107-8969-458f-82a7-9eb00e23f97f
272024-10-03T20:32:42.475ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
282024-10-03T20:32:42.475ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
292024-10-03T20:32:42.475ZINFOcrucible-agent (worker): commit
302024-10-03T20:32:42.479ZINFOcrucible-agent (worker): ok!
312024-10-03T20:32:42.483ZINFOcrucible-agent (worker): SMF ok!
322024-10-03T20:32:42.500ZINFOcrucible-agent (dropshot): request completed latency_us = 63 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::b]:37202 req_id = 6f8f9208-f9b4-4f11-8c99-b40613960342 response_code = 200 uri = /crucible/0/regions
332024-10-03T20:32:42.849ZINFOcrucible-agent (dropshot): request completed latency_us = 85 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:37202 req_id = 62aa29e1-3932-4e0d-81a3-3a649f9c0e3f response_code = 200 uri = /crucible/0/regions/65e69107-8969-458f-82a7-9eb00e23f97f
342024-10-03T20:32:42.854ZINFOcrucible-agent (datafile): path is "/data/regions/65e69107-8969-458f-82a7-9eb00e23f97f"
352024-10-03T20:32:42.865ZINFOcrucible-agent (datafile): dataset is oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/65e69107-8969-458f-82a7-9eb00e23f97f
362024-10-03T20:32:42.874ZINFOcrucible-agent (datafile): region 65e69107-8969-458f-82a7-9eb00e23f97f state: Created -> Tombstoned
372024-10-03T20:32:42.874ZINFOcrucible-agent (worker): applying SMF actions before removal...
382024-10-03T20:32:42.874ZINFOcrucible-agent (dropshot): request completed latency_us = 19964 local_addr = [fd00:1122:3344:101::14]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::b]:37202 req_id = 817d02cf-105b-45df-ae28-213369af5d71 response_code = 204 uri = /crucible/0/regions/65e69107-8969-458f-82a7-9eb00e23f97f
392024-10-03T20:32:42.875ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-65e69107-8969-458f-82a7-9eb00e23f97f (instance states: (Some(Online), None))
402024-10-03T20:32:42.877ZINFOcrucible-agent (dropshot): request completed latency_us = 36 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:37202 req_id = a738240a-008e-4d18-8df2-9f0f95806f3c response_code = 200 uri = /crucible/0/regions/65e69107-8969-458f-82a7-9eb00e23f97f
412024-10-03T20:32:42.880ZINFOcrucible-agent (worker): SMF ok!
422024-10-03T20:32:42.891ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/65e69107-8969-458f-82a7-9eb00e23f97f" region = 65e69107-8969-458f-82a7-9eb00e23f97f
432024-10-03T20:32:42.951ZINFOcrucible-agent (datafile): region 65e69107-8969-458f-82a7-9eb00e23f97f state: Tombstoned -> Destroyed
442024-10-03T20:32:43.023ZINFOcrucible-agent (dropshot): request completed latency_us = 730 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::b]:37202 req_id = e651c3ef-3b28-4974-b7c1-55888767367e response_code = 200 uri = /crucible/0/regions/65e69107-8969-458f-82a7-9eb00e23f97f
452024-10-03T20:33:08.270ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:48166
462024-10-03T20:33:08.270ZINFOcrucible-agent (datafile): region fb92a9b9-79ff-4c09-9ec6-3dc468cb1969 state: Requested
472024-10-03T20:33:08.270ZINFOcrucible-agent (dropshot): request completed latency_us = 220 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48166 req_id = 6025d469-b301-4bcb-96e7-b084aec69bf8 response_code = 200 uri = /crucible/0/regions
482024-10-03T20:33:08.270ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492024-10-03T20:33:08.279ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969
502024-10-03T20:33:08.279ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969
512024-10-03T20:33:08.330ZINFOcrucible-agent (worker): creating region Region { id: RegionId("fb92a9b9-79ff-4c09-9ec6-3dc468cb1969"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969" region = fb92a9b9-79ff-4c09-9ec6-3dc468cb1969
522024-10-03T20:33:08.332ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "fb92a9b9-79ff-4c09-9ec6-3dc468cb1969" "--data" "/data/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = fb92a9b9-79ff-4c09-9ec6-3dc468cb1969
532024-10-03T20:33:08.394ZINFOcrucible-agent (worker): region files created ok region = fb92a9b9-79ff-4c09-9ec6-3dc468cb1969
542024-10-03T20:33:08.394ZINFOcrucible-agent (datafile): region fb92a9b9-79ff-4c09-9ec6-3dc468cb1969 state: Requested -> Created
552024-10-03T20:33:08.395ZINFOcrucible-agent (worker): applying SMF actions post create...
562024-10-03T20:33:08.396ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-fb92a9b9-79ff-4c09-9ec6-3dc468cb1969
572024-10-03T20:33:08.399ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-fb92a9b9-79ff-4c09-9ec6-3dc468cb1969
582024-10-03T20:33:08.405ZINFOcrucible-agent (worker): creating config property group
592024-10-03T20:33:08.409ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-fb92a9b9-79ff-4c09-9ec6-3dc468cb1969
602024-10-03T20:33:08.409ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969
612024-10-03T20:33:08.409ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
622024-10-03T20:33:08.409ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
632024-10-03T20:33:08.410ZINFOcrucible-agent (worker): commit
642024-10-03T20:33:08.413ZINFOcrucible-agent (worker): ok!
652024-10-03T20:33:08.416ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:48166 req_id = e5b87c5b-9730-4c5d-bc9b-02ecd5db2de2 response_code = 200 uri = /crucible/0/regions
662024-10-03T20:33:08.417ZINFOcrucible-agent (worker): SMF ok!
672024-10-03T20:36:13.621ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:47258
682024-10-03T20:36:13.621ZINFOcrucible-agent (dropshot): request completed latency_us = 91 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:47258 req_id = 846f0f03-cfd8-4554-b883-d750f52859ef response_code = 200 uri = /crucible/0/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969
692024-10-03T20:36:13.625ZINFOcrucible-agent (datafile): path is "/data/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969"
702024-10-03T20:36:13.635ZINFOcrucible-agent (datafile): dataset is oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969
712024-10-03T20:36:13.644ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969@bec3d930-a591-451d-82e5-be08acca0f21
722024-10-03T20:36:13.644ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969", "bec3d930-a591-451d-82e5-be08acca0f21"]
732024-10-03T20:36:13.660ZINFOcrucible-agent (dropshot): request completed latency_us = 35362 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:47258 req_id = 3e6a1bb8-b439-4886-97b5-ff36989f1cd4 response_code = 200 uri = /crucible/0/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969/snapshots/bec3d930-a591-451d-82e5-be08acca0f21
742024-10-03T20:36:13.664ZINFOcrucible-agent (datafile): path is "/data/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969"
752024-10-03T20:36:13.674ZINFOcrucible-agent (datafile): dataset is oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969
762024-10-03T20:36:13.682ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969@bec3d930-a591-451d-82e5-be08acca0f21
772024-10-03T20:36:13.682ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969", "bec3d930-a591-451d-82e5-be08acca0f21"]
782024-10-03T20:36:13.692ZINFOcrucible-agent (datafile): requesting running snapshot fb92a9b9-79ff-4c09-9ec6-3dc468cb1969-bec3d930-a591-451d-82e5-be08acca0f21 state: Requested
792024-10-03T20:36:13.693ZINFOcrucible-agent (worker): applying SMF actions for region fb92a9b9-79ff-4c09-9ec6-3dc468cb1969 running snapshot bec3d930-a591-451d-82e5-be08acca0f21 (state Requested)...
802024-10-03T20:36:13.693ZINFOcrucible-agent (dropshot): request completed latency_us = 29273 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:47258 req_id = 6fa9f5f0-fe6d-420c-95b9-f898ece9cb4a response_code = 200 uri = /crucible/0/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969/snapshots/bec3d930-a591-451d-82e5-be08acca0f21/run
812024-10-03T20:36:13.702ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-fb92a9b9-79ff-4c09-9ec6-3dc468cb1969-bec3d930-a591-451d-82e5-be08acca0f21
822024-10-03T20:36:13.709ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-fb92a9b9-79ff-4c09-9ec6-3dc468cb1969-bec3d930-a591-451d-82e5-be08acca0f21
832024-10-03T20:36:13.727ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-fb92a9b9-79ff-4c09-9ec6-3dc468cb1969-bec3d930-a591-451d-82e5-be08acca0f21
842024-10-03T20:36:13.727ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969/.zfs/snapshot/bec3d930-a591-451d-82e5-be08acca0f21
852024-10-03T20:36:13.727ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
862024-10-03T20:36:13.727ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
872024-10-03T20:36:13.727ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
882024-10-03T20:36:13.727ZINFOcrucible-agent (worker): commit
892024-10-03T20:36:13.734ZINFOcrucible-agent (worker): ok!
902024-10-03T20:36:13.740ZINFOcrucible-agent (worker): SMF ok!
912024-10-03T20:36:13.740ZINFOcrucible-agent (datafile): region fb92a9b9-79ff-4c09-9ec6-3dc468cb1969 running snapshot bec3d930-a591-451d-82e5-be08acca0f21 state: Requested -> Created
922024-10-03T20:36:13.995ZINFOcrucible-agent (datafile): path is "/data/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969"
932024-10-03T20:36:14.004ZINFOcrucible-agent (datafile): dataset is oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969
942024-10-03T20:36:14.013ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969@bec3d930-a591-451d-82e5-be08acca0f21
952024-10-03T20:36:14.013ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969", "bec3d930-a591-451d-82e5-be08acca0f21"]
962024-10-03T20:36:14.022ZINFOcrucible-agent (dropshot): request completed latency_us = 26484 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:47258 req_id = 0d1de593-1678-4778-8a05-e0ac0206e039 response_code = 200 uri = /crucible/0/regions/fb92a9b9-79ff-4c09-9ec6-3dc468cb1969/snapshots/bec3d930-a591-451d-82e5-be08acca0f21/run
972024-10-03T20:36:17.145ZINFOcrucible-agent (datafile): region 54249848-dd06-4e42-a336-c8e751c02810 state: Requested
982024-10-03T20:36:17.145ZINFOcrucible-agent (dropshot): request completed latency_us = 225 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:47258 req_id = 2bc1721f-3c5c-4e6c-a099-092289b73b61 response_code = 200 uri = /crucible/0/regions
992024-10-03T20:36:17.145ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
1002024-10-03T20:36:17.156ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/54249848-dd06-4e42-a336-c8e751c02810
1012024-10-03T20:36:17.156ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/54249848-dd06-4e42-a336-c8e751c02810
1022024-10-03T20:36:17.226ZINFOcrucible-agent (worker): creating region Region { id: RegionId("54249848-dd06-4e42-a336-c8e751c02810"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19002, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/54249848-dd06-4e42-a336-c8e751c02810" region = 54249848-dd06-4e42-a336-c8e751c02810
1032024-10-03T20:36:17.228ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "54249848-dd06-4e42-a336-c8e751c02810" "--data" "/data/regions/54249848-dd06-4e42-a336-c8e751c02810" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 54249848-dd06-4e42-a336-c8e751c02810
1042024-10-03T20:36:17.336ZINFOcrucible-agent (worker): region files created ok region = 54249848-dd06-4e42-a336-c8e751c02810
1052024-10-03T20:36:17.336ZINFOcrucible-agent (datafile): region 54249848-dd06-4e42-a336-c8e751c02810 state: Requested -> Created
1062024-10-03T20:36:17.336ZINFOcrucible-agent (worker): applying SMF actions post create...
1072024-10-03T20:36:17.338ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-54249848-dd06-4e42-a336-c8e751c02810
1082024-10-03T20:36:17.341ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-54249848-dd06-4e42-a336-c8e751c02810
1092024-10-03T20:36:17.353ZINFOcrucible-agent (worker): creating config property group
1102024-10-03T20:36:17.358ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-54249848-dd06-4e42-a336-c8e751c02810
1112024-10-03T20:36:17.359ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/54249848-dd06-4e42-a336-c8e751c02810
1122024-10-03T20:36:17.359ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
1132024-10-03T20:36:17.359ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
1142024-10-03T20:36:17.359ZINFOcrucible-agent (worker): commit
1152024-10-03T20:36:17.364ZINFOcrucible-agent (worker): ok!
1162024-10-03T20:36:17.380ZINFOcrucible-agent (worker): SMF ok!
1172024-10-03T20:36:17.476ZINFOcrucible-agent (dropshot): request completed latency_us = 60 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:47258 req_id = dae4d344-cdf6-4d41-ad2b-03d5e7207cd8 response_code = 200 uri = /crucible/0/regions
1182024-10-03T20:36:42.115ZINFOcrucible-agent (dropshot): request completed latency_us = 501 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:47258 req_id = 8b4af37a-a94d-4035-a890-087c8d6890b8 response_code = 200 uri = /crucible/0/regions/54249848-dd06-4e42-a336-c8e751c02810
1192024-10-03T20:36:42.118ZINFOcrucible-agent (datafile): path is "/data/regions/54249848-dd06-4e42-a336-c8e751c02810"
1202024-10-03T20:36:42.130ZINFOcrucible-agent (datafile): dataset is oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/54249848-dd06-4e42-a336-c8e751c02810
1212024-10-03T20:36:42.140ZINFOcrucible-agent (datafile): region 54249848-dd06-4e42-a336-c8e751c02810 state: Created -> Tombstoned
1222024-10-03T20:36:42.140ZINFOcrucible-agent (worker): applying SMF actions before removal...
1232024-10-03T20:36:42.140ZINFOcrucible-agent (dropshot): request completed latency_us = 22132 local_addr = [fd00:1122:3344:101::14]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:47258 req_id = 6a66ec0b-7b0e-4c2f-9c62-c63cbead4488 response_code = 204 uri = /crucible/0/regions/54249848-dd06-4e42-a336-c8e751c02810
1242024-10-03T20:36:42.142ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-54249848-dd06-4e42-a336-c8e751c02810 (instance states: (Some(Online), None))
1252024-10-03T20:36:42.144ZINFOcrucible-agent (dropshot): request completed latency_us = 30 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:47258 req_id = e98272e7-9284-478f-97ea-76a0d5951be8 response_code = 200 uri = /crucible/0/regions/54249848-dd06-4e42-a336-c8e751c02810
1262024-10-03T20:36:42.150ZINFOcrucible-agent (worker): SMF ok!
1272024-10-03T20:36:42.160ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_cced9d79-ed80-4c89-994e-8eaec18a3ffe/crucible/regions/54249848-dd06-4e42-a336-c8e751c02810" region = 54249848-dd06-4e42-a336-c8e751c02810
1282024-10-03T20:36:42.216ZINFOcrucible-agent (dropshot): request completed latency_us = 100 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:47258 req_id = 8b2b21cb-2786-484d-995b-25d0680ea198 response_code = 200 uri = /crucible/0/regions/54249848-dd06-4e42-a336-c8e751c02810
1292024-10-03T20:36:42.230ZINFOcrucible-agent (datafile): region 54249848-dd06-4e42-a336-c8e751c02810 state: Tombstoned -> Destroyed
1302024-10-03T20:36:42.297ZINFOcrucible-agent (dropshot): request completed latency_us = 35 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:47258 req_id = b0416c19-7bd9-4364-a8e9-a3308d8e771a response_code = 200 uri = /crucible/0/regions/54249848-dd06-4e42-a336-c8e751c02810