Max level shown:
1 [ Sep 24 04:33:00 Enabled. ]
2 [ Sep 24 04:33:00 Rereading configuration. ]
3 [ Sep 24 04:33:01 Rereading configuration. ]
4 [ Sep 24 04:33:02 Executing start method ("/opt/oxide/crucible/bin/crucible-agent run -D /opt/oxide/crucible/bin/crucible-downstairs --dataset oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible -l [fd00:1122:3344:101::18]:32345 -P 19000 -p downstairs -s snapshot"). ]
5 note: configured to log to "/dev/stdout"
62024-09-24T04:33:03.015ZINFOcrucible-agent: dataset: "oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible"
72024-09-24T04:33:03.022ZINFOcrucible-agent: listen IP: [fd00:1122:3344:101::18]:32345
82024-09-24T04:33:03.022ZINFOcrucible-agent: SMF instance name downstairs_prefix: "downstairs"
92024-09-24T04:33:03.048ZINFOcrucible-agent (datafile): Using conf_path:"/data/crucible.json"
102024-09-24T04:33:03.178ZINFOcrucible-agent (dropshot): listening local_addr = [fd00:1122:3344:101::18]:32345
112024-09-24T04:33:34.539ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::c]:45815
122024-09-24T04:33:34.569ZINFOcrucible-agent (datafile): region 2d35064c-9e30-4a04-8795-0277a5eeab86 state: Requested
132024-09-24T04:33:34.575ZINFOcrucible-agent (dropshot): request completed latency_us = 22401 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:45815 req_id = cd5d1c5c-dfea-4825-8ab9-d934745ed01d response_code = 200 uri = /crucible/0/regions
142024-09-24T04:33:34.575ZINFOcrucible-agent (worker): Region size:1073741824 reservation:1342177280 quota:3221225472
152024-09-24T04:33:34.590ZINFOcrucible-agent (worker): zfs set reservation of 1342177280 for oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/2d35064c-9e30-4a04-8795-0277a5eeab86
162024-09-24T04:33:34.590ZINFOcrucible-agent (worker): zfs set quota of 3221225472 for oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/2d35064c-9e30-4a04-8795-0277a5eeab86
172024-09-24T04:33:34.684ZINFOcrucible-agent (worker): creating region Region { id: RegionId("2d35064c-9e30-4a04-8795-0277a5eeab86"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 16, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/2d35064c-9e30-4a04-8795-0277a5eeab86" region = 2d35064c-9e30-4a04-8795-0277a5eeab86
182024-09-24T04:33:34.688ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "2d35064c-9e30-4a04-8795-0277a5eeab86" "--data" "/data/regions/2d35064c-9e30-4a04-8795-0277a5eeab86" "--block-size" "512" "--extent-size" "131072" "--extent-count" "16" "--encrypted" region = 2d35064c-9e30-4a04-8795-0277a5eeab86
192024-09-24T04:33:34.724ZINFOcrucible-agent (dropshot): request completed latency_us = 931 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:45815 req_id = 8636ac92-6ac1-4f29-9b93-3dda1917a362 response_code = 200 uri = /crucible/0/regions
202024-09-24T04:33:34.825ZINFOcrucible-agent (worker): region files created ok region = 2d35064c-9e30-4a04-8795-0277a5eeab86
212024-09-24T04:33:34.825ZINFOcrucible-agent (datafile): region 2d35064c-9e30-4a04-8795-0277a5eeab86 state: Requested -> Created
222024-09-24T04:33:34.825ZINFOcrucible-agent (worker): applying SMF actions post create...
232024-09-24T04:33:34.827ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-2d35064c-9e30-4a04-8795-0277a5eeab86
242024-09-24T04:33:34.832ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-2d35064c-9e30-4a04-8795-0277a5eeab86
252024-09-24T04:33:34.850ZINFOcrucible-agent (worker): creating config property group
262024-09-24T04:33:34.854ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-2d35064c-9e30-4a04-8795-0277a5eeab86
272024-09-24T04:33:34.863ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/2d35064c-9e30-4a04-8795-0277a5eeab86
282024-09-24T04:33:34.866ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
292024-09-24T04:33:34.866ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
302024-09-24T04:33:34.866ZINFOcrucible-agent (worker): commit
312024-09-24T04:33:34.870ZINFOcrucible-agent (worker): ok!
322024-09-24T04:33:34.878ZINFOcrucible-agent (worker): SMF ok!
332024-09-24T04:33:35.312ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::c]:45815 req_id = 4ba59573-e387-4cad-ae4a-10e69272853f response_code = 200 uri = /crucible/0/regions
342024-09-24T04:33:35.752ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:45815 req_id = efd87a68-6719-4265-aa59-b444ab5a424d response_code = 200 uri = /crucible/0/regions/2d35064c-9e30-4a04-8795-0277a5eeab86
352024-09-24T04:33:35.755ZINFOcrucible-agent (datafile): path is "/data/regions/2d35064c-9e30-4a04-8795-0277a5eeab86"
362024-09-24T04:33:35.766ZINFOcrucible-agent (datafile): dataset is oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/2d35064c-9e30-4a04-8795-0277a5eeab86
372024-09-24T04:33:35.777ZINFOcrucible-agent (datafile): region 2d35064c-9e30-4a04-8795-0277a5eeab86 state: Created -> Tombstoned
382024-09-24T04:33:35.777ZINFOcrucible-agent (worker): applying SMF actions before removal...
392024-09-24T04:33:35.777ZINFOcrucible-agent (dropshot): request completed latency_us = 22383 local_addr = [fd00:1122:3344:101::18]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::c]:45815 req_id = 9538fe71-b4ec-4145-8a73-f57805257a91 response_code = 204 uri = /crucible/0/regions/2d35064c-9e30-4a04-8795-0277a5eeab86
402024-09-24T04:33:35.778ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-2d35064c-9e30-4a04-8795-0277a5eeab86 (instance states: (Some(Online), None))
412024-09-24T04:33:35.782ZINFOcrucible-agent (dropshot): request completed latency_us = 644 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:45815 req_id = 4c6b9fdd-d73e-4dcd-bf48-c40eff9b6769 response_code = 200 uri = /crucible/0/regions/2d35064c-9e30-4a04-8795-0277a5eeab86
422024-09-24T04:33:35.784ZINFOcrucible-agent (worker): SMF ok!
432024-09-24T04:33:35.795ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/2d35064c-9e30-4a04-8795-0277a5eeab86" region = 2d35064c-9e30-4a04-8795-0277a5eeab86
442024-09-24T04:33:35.873ZINFOcrucible-agent (datafile): region 2d35064c-9e30-4a04-8795-0277a5eeab86 state: Tombstoned -> Destroyed
452024-09-24T04:33:35.933ZINFOcrucible-agent (dropshot): request completed latency_us = 84 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::c]:45815 req_id = a5210ad7-0e84-4bb2-a4d9-950cc32b68c1 response_code = 200 uri = /crucible/0/regions/2d35064c-9e30-4a04-8795-0277a5eeab86
462024-09-24T04:34:29.792ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::a]:63267
472024-09-24T04:34:29.792ZINFOcrucible-agent (datafile): region 4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035 state: Requested
482024-09-24T04:34:29.793ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
492024-09-24T04:34:29.793ZINFOcrucible-agent (dropshot): request completed latency_us = 1044 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:63267 req_id = 98e81fb4-6c62-49f9-bb24-6ff4362e1808 response_code = 200 uri = /crucible/0/regions
502024-09-24T04:34:29.803ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035
512024-09-24T04:34:29.803ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035
522024-09-24T04:34:29.861ZINFOcrucible-agent (worker): creating region Region { id: RegionId("4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19000, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035" region = 4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035
532024-09-24T04:34:29.861ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035" "--data" "/data/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = 4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035
542024-09-24T04:34:29.933ZINFOcrucible-agent (worker): region files created ok region = 4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035
552024-09-24T04:34:29.933ZINFOcrucible-agent (datafile): region 4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035 state: Requested -> Created
562024-09-24T04:34:29.933ZINFOcrucible-agent (worker): applying SMF actions post create...
572024-09-24T04:34:29.934ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035
582024-09-24T04:34:29.938ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035
592024-09-24T04:34:29.949ZINFOcrucible-agent (worker): creating config property group
602024-09-24T04:34:29.953ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035
612024-09-24T04:34:29.953ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035
622024-09-24T04:34:29.953ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19000
632024-09-24T04:34:29.953ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
642024-09-24T04:34:29.953ZINFOcrucible-agent (worker): commit
652024-09-24T04:34:29.957ZINFOcrucible-agent (worker): ok!
662024-09-24T04:34:29.962ZINFOcrucible-agent (worker): SMF ok!
672024-09-24T04:34:29.975ZINFOcrucible-agent (dropshot): request completed latency_us = 72 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:63267 req_id = 8c2a1414-c4c2-4992-87ce-81048ea01c00 response_code = 200 uri = /crucible/0/regions
682024-09-24T04:39:59.516ZINFOcrucible-agent (dropshot): accepted connection local_addr = [fd00:1122:3344:101::18]:32345 remote_addr = [fd00:1122:3344:101::a]:42630
692024-09-24T04:39:59.516ZINFOcrucible-agent (dropshot): request completed latency_us = 48 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42630 req_id = ae56d4ea-1933-46c0-bb59-8b603a246d2f response_code = 200 uri = /crucible/0/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035
702024-09-24T04:39:59.518ZINFOcrucible-agent (datafile): path is "/data/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035"
712024-09-24T04:39:59.528ZINFOcrucible-agent (datafile): dataset is oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035
722024-09-24T04:39:59.538ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035@ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8
732024-09-24T04:39:59.538ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035", "ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8"]
742024-09-24T04:39:59.550ZINFOcrucible-agent (dropshot): request completed latency_us = 31955 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42630 req_id = e05a6fee-998e-414e-9d5f-161696ef0601 response_code = 200 uri = /crucible/0/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035/snapshots/ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8
752024-09-24T04:39:59.555ZINFOcrucible-agent (datafile): path is "/data/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035"
762024-09-24T04:39:59.563ZINFOcrucible-agent (datafile): dataset is oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035
772024-09-24T04:39:59.571ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035@ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8
782024-09-24T04:39:59.571ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035", "ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8"]
792024-09-24T04:39:59.581ZINFOcrucible-agent (datafile): requesting running snapshot 4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035-ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8 state: Requested
802024-09-24T04:39:59.581ZINFOcrucible-agent (worker): applying SMF actions for region 4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035 running snapshot ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8 (state Requested)...
812024-09-24T04:39:59.581ZINFOcrucible-agent (dropshot): request completed latency_us = 26503 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42630 req_id = 6ca031b1-3823-48d8-98dd-e91b949789a4 response_code = 200 uri = /crucible/0/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035/snapshots/ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8/run
822024-09-24T04:39:59.590ZINFOcrucible-agent (worker): creating missing snapshot instance snapshot-4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035-ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8
832024-09-24T04:39:59.597ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:snapshot-4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035-ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8
842024-09-24T04:39:59.607ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:snapshot-4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035-ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8
852024-09-24T04:39:59.608ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035/.zfs/snapshot/ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8
862024-09-24T04:39:59.608ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19001
872024-09-24T04:39:59.608ZINFOcrucible-agent (worker): ensure mode SCF_TYPE_ASTRING ro
882024-09-24T04:39:59.608ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
892024-09-24T04:39:59.608ZINFOcrucible-agent (worker): commit
902024-09-24T04:39:59.612ZINFOcrucible-agent (worker): ok!
912024-09-24T04:39:59.617ZINFOcrucible-agent (worker): SMF ok!
922024-09-24T04:39:59.617ZINFOcrucible-agent (datafile): region 4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035 running snapshot ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8 state: Requested -> Created
932024-09-24T04:39:59.753ZINFOcrucible-agent (datafile): path is "/data/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035"
942024-09-24T04:39:59.762ZINFOcrucible-agent (datafile): dataset is oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035
952024-09-24T04:39:59.769ZINFOcrucible-agent (ZfsSnapshotInterface): snapshot is oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035@ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8
962024-09-24T04:39:59.769ZINFOcrucible-agent (ZfsSnapshotInterface): parts is ["oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035", "ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8"]
972024-09-24T04:39:59.778ZINFOcrucible-agent (dropshot): request completed latency_us = 24351 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42630 req_id = 09225c50-bc74-4b27-a8e8-2fc46e58e322 response_code = 200 uri = /crucible/0/regions/4a76a218-eb3e-48f3-bf8f-3b8f2d3a6035/snapshots/ce7eb4d5-40e4-4cef-8eb5-d87306d75bc8/run
982024-09-24T04:40:03.505ZINFOcrucible-agent (datafile): region bbb9b41d-125d-4b5b-9c98-7034924fc92b state: Requested
992024-09-24T04:40:03.505ZINFOcrucible-agent (dropshot): request completed latency_us = 818 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42630 req_id = c590e16b-d6dd-4e20-a83e-1a0d3b224b54 response_code = 200 uri = /crucible/0/regions
1002024-09-24T04:40:03.505ZINFOcrucible-agent (worker): Region size:2147483648 reservation:2684354560 quota:6442450944
1012024-09-24T04:40:03.515ZINFOcrucible-agent (worker): zfs set reservation of 2684354560 for oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/bbb9b41d-125d-4b5b-9c98-7034924fc92b
1022024-09-24T04:40:03.515ZINFOcrucible-agent (worker): zfs set quota of 6442450944 for oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/bbb9b41d-125d-4b5b-9c98-7034924fc92b
1032024-09-24T04:40:03.579ZINFOcrucible-agent (worker): creating region Region { id: RegionId("bbb9b41d-125d-4b5b-9c98-7034924fc92b"), state: Requested, block_size: 512, extent_size: 131072, extent_count: 32, encrypted: true, port_number: 19002, cert_pem: None, key_pem: None, root_pem: None, source: None, read_only: false } at "/data/regions/bbb9b41d-125d-4b5b-9c98-7034924fc92b" region = bbb9b41d-125d-4b5b-9c98-7034924fc92b
1042024-09-24T04:40:03.579ZINFOcrucible-agent (worker): downstairs create with: env -i "/opt/oxide/crucible/bin/crucible-downstairs" "create" "--uuid" "bbb9b41d-125d-4b5b-9c98-7034924fc92b" "--data" "/data/regions/bbb9b41d-125d-4b5b-9c98-7034924fc92b" "--block-size" "512" "--extent-size" "131072" "--extent-count" "32" "--encrypted" region = bbb9b41d-125d-4b5b-9c98-7034924fc92b
1052024-09-24T04:40:03.690ZINFOcrucible-agent (worker): region files created ok region = bbb9b41d-125d-4b5b-9c98-7034924fc92b
1062024-09-24T04:40:03.690ZINFOcrucible-agent (datafile): region bbb9b41d-125d-4b5b-9c98-7034924fc92b state: Requested -> Created
1072024-09-24T04:40:03.690ZINFOcrucible-agent (worker): applying SMF actions post create...
1082024-09-24T04:40:03.692ZINFOcrucible-agent (worker): creating missing downstairs instance downstairs-bbb9b41d-125d-4b5b-9c98-7034924fc92b
1092024-09-24T04:40:03.695ZINFOcrucible-agent (worker): ok, have svc:/oxide/crucible/downstairs:downstairs-bbb9b41d-125d-4b5b-9c98-7034924fc92b
1102024-09-24T04:40:03.701ZINFOcrucible-agent (worker): creating config property group
1112024-09-24T04:40:03.705ZINFOcrucible-agent (worker): reconfiguring svc:/oxide/crucible/downstairs:downstairs-bbb9b41d-125d-4b5b-9c98-7034924fc92b
1122024-09-24T04:40:03.705ZINFOcrucible-agent (worker): ensure directory SCF_TYPE_ASTRING /data/regions/bbb9b41d-125d-4b5b-9c98-7034924fc92b
1132024-09-24T04:40:03.706ZINFOcrucible-agent (worker): ensure port SCF_TYPE_COUNT 19002
1142024-09-24T04:40:03.706ZINFOcrucible-agent (worker): ensure address SCF_TYPE_ASTRING fd00:1122:3344:101::18
1152024-09-24T04:40:03.706ZINFOcrucible-agent (worker): commit
1162024-09-24T04:40:03.710ZINFOcrucible-agent (worker): ok!
1172024-09-24T04:40:03.716ZINFOcrucible-agent (worker): SMF ok!
1182024-09-24T04:40:03.825ZINFOcrucible-agent (dropshot): request completed latency_us = 74 local_addr = [fd00:1122:3344:101::18]:32345 method = POST remote_addr = [fd00:1122:3344:101::a]:42630 req_id = 1ddf2cb3-702f-4e66-9399-e7eb6e5bde44 response_code = 200 uri = /crucible/0/regions
1192024-09-24T04:40:35.685ZINFOcrucible-agent (dropshot): request completed latency_us = 57 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42630 req_id = 8700f00f-edbd-4e9c-a35f-781320809061 response_code = 200 uri = /crucible/0/regions/bbb9b41d-125d-4b5b-9c98-7034924fc92b
1202024-09-24T04:40:35.691ZINFOcrucible-agent (datafile): path is "/data/regions/bbb9b41d-125d-4b5b-9c98-7034924fc92b"
1212024-09-24T04:40:35.703ZINFOcrucible-agent (datafile): dataset is oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/bbb9b41d-125d-4b5b-9c98-7034924fc92b
1222024-09-24T04:40:35.714ZINFOcrucible-agent (datafile): region bbb9b41d-125d-4b5b-9c98-7034924fc92b state: Created -> Tombstoned
1232024-09-24T04:40:35.715ZINFOcrucible-agent (worker): applying SMF actions before removal...
1242024-09-24T04:40:35.715ZINFOcrucible-agent (dropshot): request completed latency_us = 23892 local_addr = [fd00:1122:3344:101::18]:32345 method = DELETE remote_addr = [fd00:1122:3344:101::a]:42630 req_id = ee46005e-e954-4b37-bc31-b14e1a9a7f08 response_code = 204 uri = /crucible/0/regions/bbb9b41d-125d-4b5b-9c98-7034924fc92b
1252024-09-24T04:40:35.716ZINFOcrucible-agent (worker): disabling downstairs instance: downstairs-bbb9b41d-125d-4b5b-9c98-7034924fc92b (instance states: (Some(Online), None))
1262024-09-24T04:40:35.717ZINFOcrucible-agent (dropshot): request completed latency_us = 25 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42630 req_id = 5c657a88-b561-4422-a213-d7e108ef5368 response_code = 200 uri = /crucible/0/regions/bbb9b41d-125d-4b5b-9c98-7034924fc92b
1272024-09-24T04:40:35.724ZINFOcrucible-agent (worker): SMF ok!
1282024-09-24T04:40:35.735ZINFOcrucible-agent (worker): deleting zfs dataset "oxp_a09f232d-c31e-4961-bb00-74c2dac76055/crucible/regions/bbb9b41d-125d-4b5b-9c98-7034924fc92b" region = bbb9b41d-125d-4b5b-9c98-7034924fc92b
1292024-09-24T04:40:35.788ZINFOcrucible-agent (datafile): region bbb9b41d-125d-4b5b-9c98-7034924fc92b state: Tombstoned -> Destroyed
1302024-09-24T04:40:35.847ZINFOcrucible-agent (dropshot): request completed latency_us = 76 local_addr = [fd00:1122:3344:101::18]:32345 method = GET remote_addr = [fd00:1122:3344:101::a]:42630 req_id = ea397785-f643-4cb8-a19e-0b23abde4f55 response_code = 200 uri = /crucible/0/regions/bbb9b41d-125d-4b5b-9c98-7034924fc92b