Max level shown:
1 [ Oct 9 17:19:23 Enabled. ]
2 [ Oct 9 17:19:23 Rereading configuration. ]
3 [ Oct 9 17:19:25 Rereading configuration. ]
4 [ Oct 9 17:19:28 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible -l [fd00:1122:3344:101::14]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-10-09T17:19:28.301ZINFOcrucible-agent: dataset: "oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible"
72024-10-09T17:19:28.303ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::14]:32345
82024-10-09T17:19:28.303ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-10-09T17:19:28.331ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-10-09T17:19:28.435ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::14]:32345
112024-10-09T17:20:27.267ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::c]:57065
122024-10-09T17:20:27.280ZINFOcrucible-agent (datafile): region fb15ba01-d6c7-4eed-adf7-685621455455 state: Requested
132024-10-09T17:20:27.285ZINFOcrucible-agent (dropshot): request completed latency_us = 8600 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:57065 req_id = ba89bbee-b491-431e-8aeb-7ea6a945c926 response_code = 200 uri = /crucible/0/regions
142024-10-09T17:20:27.286ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152024-10-09T17:20:27.295ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/fb15ba01-d6c7-4eed-adf7-685621455455
162024-10-09T17:20:27.295ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/fb15ba01-d6c7-4eed-adf7-685621455455
172024-10-09T17:20:27.371ZINFOcrucible-agent (worker): creating region Region { id: RegionId("fb15ba01-d6c7-4eed-adf7-685621455455"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/fb15ba01-d6c7-4eed-adf7-685621455455" region = fb15ba01-d6c7-4eed-adf7-685621455455
182024-10-09T17:20:27.373ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "fb15ba01-d6c7-4eed-adf7-685621455455" "--data" "/data/regions/fb15ba01-d6c7-4eed-adf7-685621455455" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = fb15ba01-d6c7-4eed-adf7-685621455455
192024-10-09T17:20:27.497ZINFOcrucible-agent (worker): region files created ok region = fb15ba01-d6c7-4eed-adf7-685621455455
202024-10-09T17:20:27.497ZINFOcrucible-agent (datafile): region fb15ba01-d6c7-4eed-adf7-685621455455 state: Requested -> Created
212024-10-09T17:20:27.497ZINFOcrucible-agent (worker): applying SMF actions post create...
222024-10-09T17:20:27.497ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-fb15ba01-d6c7-4eed-adf7-685621455455
232024-10-09T17:20:27.501ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-fb15ba01-d6c7-4eed-adf7-685621455455
242024-10-09T17:20:27.517ZINFOcrucible-agent (worker): creating config property group
252024-10-09T17:20:27.522ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-fb15ba01-d6c7-4eed-adf7-685621455455
262024-10-09T17:20:27.529ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/fb15ba01-d6c7-4eed-adf7-685621455455
272024-10-09T17:20:27.529ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
282024-10-09T17:20:27.529ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
292024-10-09T17:20:27.529ZINFOcrucible-agent (worker): commit
302024-10-09T17:20:27.533ZINFOcrucible-agent (worker): ok!
312024-10-09T17:20:27.538ZINFOcrucible-agent (worker): SMF ok!
322024-10-09T17:20:27.548ZINFOcrucible-agent (dropshot): request completed latency_us = 69 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:57065 req_id = 320cdd33-7053-4457-b6e2-fef04a22cc8b response_code = 200 uri = /crucible/0/regions
332024-10-09T17:20:28.261ZINFOcrucible-agent (dropshot): request completed latency_us = 438 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:57065 req_id = c7871f71-96c7-4d18-8a4b-b17530681d42 response_code = 200 uri = /crucible/0/regions/fb15ba01-d6c7-4eed-adf7-685621455455
342024-10-09T17:20:28.264ZINFOcrucible-agent (datafile): path is "/data/regions/fb15ba01-d6c7-4eed-adf7-685621455455"
352024-10-09T17:20:28.275ZINFOcrucible-agent (datafile): dataset is oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/fb15ba01-d6c7-4eed-adf7-685621455455
362024-10-09T17:20:28.284ZINFOcrucible-agent (datafile): region fb15ba01-d6c7-4eed-adf7-685621455455 state: Created -> Tombstoned
372024-10-09T17:20:28.284ZINFOcrucible-agent (worker): applying SMF actions before removal...
382024-10-09T17:20:28.284ZINFOcrucible-agent (dropshot): request completed latency_us = 20279 local_addr = [fd00:1122:3344:101::14]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:57065 req_id = 0a44b6aa-5e20-45ea-a834-5e40deb47f15 response_code = 204 uri = /crucible/0/regions/fb15ba01-d6c7-4eed-adf7-685621455455
392024-10-09T17:20:28.285ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-fb15ba01-d6c7-4eed-adf7-685621455455 (instance states: (Some(Online), None))
402024-10-09T17:20:28.287ZINFOcrucible-agent (dropshot): request completed latency_us = 35 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:57065 req_id = b240fd10-4150-4e16-8aec-4939ef58c2bd response_code = 200 uri = /crucible/0/regions/fb15ba01-d6c7-4eed-adf7-685621455455
412024-10-09T17:20:28.289ZINFOcrucible-agent (worker): SMF ok!
422024-10-09T17:20:28.300ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/fb15ba01-d6c7-4eed-adf7-685621455455" region = fb15ba01-d6c7-4eed-adf7-685621455455
432024-10-09T17:20:28.368ZINFOcrucible-agent (datafile): region fb15ba01-d6c7-4eed-adf7-685621455455 state: Tombstoned -> Destroyed
442024-10-09T17:20:28.441ZINFOcrucible-agent (dropshot): request completed latency_us = 81 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:57065 req_id = 3f748510-83ce-4948-8210-366565efadbc response_code = 200 uri = /crucible/0/regions/fb15ba01-d6c7-4eed-adf7-685621455455
452024-10-09T17:20:41.611ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:54171
462024-10-09T17:20:41.611ZINFOcrucible-agent (datafile): region 1c54eac8-4ecf-4dc6-a559-9928bdc705b7 state: Requested
472024-10-09T17:20:41.612ZINFOcrucible-agent (dropshot): request completed latency_us = 855 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:54171 req_id = 938d7c2d-51a8-4b5d-a475-265c5674802e response_code = 200 uri = /crucible/0/regions
482024-10-09T17:20:41.612ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492024-10-09T17:20:41.621ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7
502024-10-09T17:20:41.621ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7
512024-10-09T17:20:41.686ZINFOcrucible-agent (worker): creating region Region { id: RegionId("1c54eac8-4ecf-4dc6-a559-9928bdc705b7"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7" region = 1c54eac8-4ecf-4dc6-a559-9928bdc705b7
522024-10-09T17:20:41.686ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "1c54eac8-4ecf-4dc6-a559-9928bdc705b7" "--data" "/data/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 1c54eac8-4ecf-4dc6-a559-9928bdc705b7
532024-10-09T17:20:41.753ZINFOcrucible-agent (worker): region files created ok region = 1c54eac8-4ecf-4dc6-a559-9928bdc705b7
542024-10-09T17:20:41.753ZINFOcrucible-agent (datafile): region 1c54eac8-4ecf-4dc6-a559-9928bdc705b7 state: Requested -> Created
552024-10-09T17:20:41.753ZINFOcrucible-agent (worker): applying SMF actions post create...
562024-10-09T17:20:41.754ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-1c54eac8-4ecf-4dc6-a559-9928bdc705b7
572024-10-09T17:20:41.757ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-1c54eac8-4ecf-4dc6-a559-9928bdc705b7
582024-10-09T17:20:41.764ZINFOcrucible-agent (worker): creating config property group
592024-10-09T17:20:41.767ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-1c54eac8-4ecf-4dc6-a559-9928bdc705b7
602024-10-09T17:20:41.767ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7
612024-10-09T17:20:41.767ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
622024-10-09T17:20:41.767ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
632024-10-09T17:20:41.767ZINFOcrucible-agent (worker): commit
642024-10-09T17:20:41.770ZINFOcrucible-agent (worker): ok!
652024-10-09T17:20:41.774ZINFOcrucible-agent (worker): SMF ok!
662024-10-09T17:20:41.952ZINFOcrucible-agent (dropshot): request completed latency_us = 65 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:54171 req_id = f4f86f63-6371-4bcf-b92a-cc07d1174cd0 response_code = 200 uri = /crucible/0/regions
672024-10-09T17:24:01.777ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::14]:32345 remote_addr = [fd00:1122:3344:101::a]:34696
682024-10-09T17:24:01.777ZINFOcrucible-agent (dropshot): request completed latency_us = 69 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:34696 req_id = 78fef413-72d9-4d3f-a58a-591150d81931 response_code = 200 uri = /crucible/0/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7
692024-10-09T17:24:01.780ZINFOcrucible-agent (datafile): path is "/data/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7"
702024-10-09T17:24:01.789ZINFOcrucible-agent (datafile): dataset is oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7
712024-10-09T17:24:01.798ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7@5a769270-2edb-4f6e-8f69-40585f1d8665
722024-10-09T17:24:01.798ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7", "5a769270-2edb-4f6e-8f69-40585f1d8665"]
732024-10-09T17:24:01.811ZINFOcrucible-agent (dropshot): request completed latency_us = 31539 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:34696 req_id = be561bb6-d524-496d-b6de-f029adfb7a4d response_code = 200 uri = /crucible/0/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7/snapshots/5a769270-2edb-4f6e-8f69-40585f1d8665
742024-10-09T17:24:01.815ZINFOcrucible-agent (datafile): path is "/data/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7"
752024-10-09T17:24:01.823ZINFOcrucible-agent (datafile): dataset is oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7
762024-10-09T17:24:01.830ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7@5a769270-2edb-4f6e-8f69-40585f1d8665
772024-10-09T17:24:01.830ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7", "5a769270-2edb-4f6e-8f69-40585f1d8665"]
782024-10-09T17:24:01.839ZINFOcrucible-agent (datafile): requesting running snapshot 1c54eac8-4ecf-4dc6-a559-9928bdc705b7-5a769270-2edb-4f6e-8f69-40585f1d8665 state: Requested
792024-10-09T17:24:01.840ZINFOcrucible-agent (worker): applying SMF actions for region 1c54eac8-4ecf-4dc6-a559-9928bdc705b7 running snapshot 5a769270-2edb-4f6e-8f69-40585f1d8665 (state Requested)...
802024-10-09T17:24:01.840ZINFOcrucible-agent (dropshot): request completed latency_us = 24555 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:34696 req_id = bc5b7c8a-8d01-4b9c-abee-473067a649e6 response_code = 200 uri = /crucible/0/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7/snapshots/5a769270-2edb-4f6e-8f69-40585f1d8665/run
812024-10-09T17:24:01.849ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-1c54eac8-4ecf-4dc6-a559-9928bdc705b7-5a769270-2edb-4f6e-8f69-40585f1d8665
822024-10-09T17:24:01.855ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-1c54eac8-4ecf-4dc6-a559-9928bdc705b7-5a769270-2edb-4f6e-8f69-40585f1d8665
832024-10-09T17:24:01.865ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-1c54eac8-4ecf-4dc6-a559-9928bdc705b7-5a769270-2edb-4f6e-8f69-40585f1d8665
842024-10-09T17:24:01.866ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7/.zfs/snapshot/5a769270-2edb-4f6e-8f69-40585f1d8665
852024-10-09T17:24:01.866ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
862024-10-09T17:24:01.866ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
872024-10-09T17:24:01.866ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
882024-10-09T17:24:01.866ZINFOcrucible-agent (worker): commit
892024-10-09T17:24:01.869ZINFOcrucible-agent (worker): ok!
902024-10-09T17:24:01.873ZINFOcrucible-agent (worker): SMF ok!
912024-10-09T17:24:01.873ZINFOcrucible-agent (datafile): region 1c54eac8-4ecf-4dc6-a559-9928bdc705b7 running snapshot 5a769270-2edb-4f6e-8f69-40585f1d8665 state: Requested -> Created
922024-10-09T17:24:02.169ZINFOcrucible-agent (datafile): path is "/data/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7"
932024-10-09T17:24:02.179ZINFOcrucible-agent (datafile): dataset is oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7
942024-10-09T17:24:02.189ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7@5a769270-2edb-4f6e-8f69-40585f1d8665
952024-10-09T17:24:02.189ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7", "5a769270-2edb-4f6e-8f69-40585f1d8665"]
962024-10-09T17:24:02.197ZINFOcrucible-agent (dropshot): request completed latency_us = 27644 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:34696 req_id = eb3d3934-5212-478c-8f20-f5cd43ef07cd response_code = 200 uri = /crucible/0/regions/1c54eac8-4ecf-4dc6-a559-9928bdc705b7/snapshots/5a769270-2edb-4f6e-8f69-40585f1d8665/run
972024-10-09T17:24:05.460ZINFOcrucible-agent (datafile): region 077b7733-1f19-49db-99a9-a7d238ce48b5 state: Requested
982024-10-09T17:24:05.460ZINFOcrucible-agent (dropshot): request completed latency_us = 219 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:34696 req_id = 3bf4d16a-566a-4751-923e-0c08e839b5ce response_code = 200 uri = /crucible/0/regions
992024-10-09T17:24:05.460ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
1002024-10-09T17:24:05.468ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/077b7733-1f19-49db-99a9-a7d238ce48b5
1012024-10-09T17:24:05.468ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/077b7733-1f19-49db-99a9-a7d238ce48b5
1022024-10-09T17:24:05.523ZINFOcrucible-agent (worker): creating region Region { id: RegionId("077b7733-1f19-49db-99a9-a7d238ce48b5"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19002, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/077b7733-1f19-49db-99a9-a7d238ce48b5" region = 077b7733-1f19-49db-99a9-a7d238ce48b5
1032024-10-09T17:24:05.525ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "077b7733-1f19-49db-99a9-a7d238ce48b5" "--data" "/data/regions/077b7733-1f19-49db-99a9-a7d238ce48b5" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 077b7733-1f19-49db-99a9-a7d238ce48b5
1042024-10-09T17:24:05.592ZINFOcrucible-agent (worker): region files created ok region = 077b7733-1f19-49db-99a9-a7d238ce48b5
1052024-10-09T17:24:05.592ZINFOcrucible-agent (datafile): region 077b7733-1f19-49db-99a9-a7d238ce48b5 state: Requested -> Created
1062024-10-09T17:24:05.592ZINFOcrucible-agent (worker): applying SMF actions post create...
1072024-10-09T17:24:05.593ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-077b7733-1f19-49db-99a9-a7d238ce48b5
1082024-10-09T17:24:05.596ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-077b7733-1f19-49db-99a9-a7d238ce48b5
1092024-10-09T17:24:05.602ZINFOcrucible-agent (worker): creating config property group
1102024-10-09T17:24:05.605ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-077b7733-1f19-49db-99a9-a7d238ce48b5
1112024-10-09T17:24:05.605ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/077b7733-1f19-49db-99a9-a7d238ce48b5
1122024-10-09T17:24:05.605ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
1132024-10-09T17:24:05.605ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::14
1142024-10-09T17:24:05.605ZINFOcrucible-agent (worker): commit
1152024-10-09T17:24:05.608ZINFOcrucible-agent (worker): ok!
1162024-10-09T17:24:05.618ZINFOcrucible-agent (worker): SMF ok!
1172024-10-09T17:24:05.796ZINFOcrucible-agent (dropshot): request completed latency_us = 54 local_addr = [fd00:1122:3344:101::14]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:34696 req_id = a1e2f273-9c6b-4380-90a7-b5b9ea6cea49 response_code = 200 uri = /crucible/0/regions
1182024-10-09T17:24:48.893ZINFOcrucible-agent (dropshot): request completed latency_us = 51 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:34696 req_id = 4bf439a7-1c92-42ed-93de-43bd34c5464b response_code = 200 uri = /crucible/0/regions/077b7733-1f19-49db-99a9-a7d238ce48b5
1192024-10-09T17:24:48.898ZINFOcrucible-agent (datafile): path is "/data/regions/077b7733-1f19-49db-99a9-a7d238ce48b5"
1202024-10-09T17:24:48.916ZINFOcrucible-agent (datafile): dataset is oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/077b7733-1f19-49db-99a9-a7d238ce48b5
1212024-10-09T17:24:48.925ZINFOcrucible-agent (datafile): region 077b7733-1f19-49db-99a9-a7d238ce48b5 state: Created -> Tombstoned
1222024-10-09T17:24:48.925ZINFOcrucible-agent (worker): applying SMF actions before removal...
1232024-10-09T17:24:48.925ZINFOcrucible-agent (dropshot): request completed latency_us = 27877 local_addr = [fd00:1122:3344:101::14]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:34696 req_id = cf26f9fe-611b-4776-a318-951e9da2561c response_code = 204 uri = /crucible/0/regions/077b7733-1f19-49db-99a9-a7d238ce48b5
1242024-10-09T17:24:48.927ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-077b7733-1f19-49db-99a9-a7d238ce48b5 (instance states: (Some(Online), None))
1252024-10-09T17:24:48.928ZINFOcrucible-agent (dropshot): request completed latency_us = 25 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:34696 req_id = c19d2e6a-7e31-433a-aaa3-38f151e853ba response_code = 200 uri = /crucible/0/regions/077b7733-1f19-49db-99a9-a7d238ce48b5
1262024-10-09T17:24:48.969ZINFOcrucible-agent (worker): SMF ok!
1272024-10-09T17:24:48.979ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_e1224a3e-3bfb-4008-ad54-095a519153b9/crucible/regions/077b7733-1f19-49db-99a9-a7d238ce48b5" region = 077b7733-1f19-49db-99a9-a7d238ce48b5
1282024-10-09T17:24:49.013ZINFOcrucible-agent (dropshot): request completed latency_us = 66 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:34696 req_id = c4b85dcd-3857-417e-b278-7e070bb71aee response_code = 200 uri = /crucible/0/regions/077b7733-1f19-49db-99a9-a7d238ce48b5
1292024-10-09T17:24:49.131ZINFOcrucible-agent (datafile): region 077b7733-1f19-49db-99a9-a7d238ce48b5 state: Tombstoned -> Destroyed
1302024-10-09T17:24:49.155ZINFOcrucible-agent (dropshot): request completed latency_us = 26 local_addr = [fd00:1122:3344:101::14]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:34696 req_id = 03381b20-6b46-4d57-b1e4-4f3e64902e99 response_code = 200 uri = /crucible/0/regions/077b7733-1f19-49db-99a9-a7d238ce48b5