Max level shown:
12024-11-14T00:30:03.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
22024-11-14T00:30:03.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32024-11-14T00:30:03.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42024-11-14T00:30:03.139ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
52024-11-14T00:30:03.139ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62024-11-14T00:30:03.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
72024-11-14T00:30:03.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82024-11-14T00:30:03.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
92024-11-14T00:30:03.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
102024-11-14T00:30:03.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
112024-11-14T00:30:03.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
122024-11-14T00:30:03.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132024-11-14T00:30:03.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142024-11-14T00:30:03.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6178f930-5fa8-46a0-8fe5-b460818f9d6f, lookup_type: ById(6178f930-5fa8-46a0-8fe5-b460818f9d6f) } result = Ok(())
152024-11-14T00:30:03.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
162024-11-14T00:30:03.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
172024-11-14T00:30:03.161ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f original_time_created = 2024-11-14 00:18:47.526379 UTC target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f time_created = 2024-11-14 00:18:47.526379 UTC
182024-11-14T00:30:03.161ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.921243ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"6178f930-5fa8-46a0-8fe5-b460818f9d6f","time_created":"2024-11-14T00:18:47.526379Z"}
192024-11-14T00:30:03.169ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
202024-11-14T00:30:03.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
212024-11-14T00:30:03.170ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.459643ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
222024-11-14T00:30:03.187ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = dcf46cec-2b5a-43bc-9fa0-6c1d68ce1657 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
232024-11-14T00:30:03.188ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = dcf46cec-2b5a-43bc-9fa0-6c1d68ce1657 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
242024-11-14T00:30:03.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1718 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = dcf46cec-2b5a-43bc-9fa0-6c1d68ce1657 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
252024-11-14T00:30:04.215ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = d216ecb5-b241-4c85-afdb-72d4dbf6abc2 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
262024-11-14T00:30:04.215ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = d216ecb5-b241-4c85-afdb-72d4dbf6abc2 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
272024-11-14T00:30:04.217ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2210 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = d216ecb5-b241-4c85-afdb-72d4dbf6abc2 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
282024-11-14T00:30:05.238ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 865aaa84-86eb-4efe-89ed-1420637b464e roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
292024-11-14T00:30:05.238ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 865aaa84-86eb-4efe-89ed-1420637b464e resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
302024-11-14T00:30:05.241ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2590 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 865aaa84-86eb-4efe-89ed-1420637b464e response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
312024-11-14T00:30:05.938ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = b075ff4b-2cd5-4b36-b04d-9dda349220f3 roles = RoleSet { roles: {} } uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
322024-11-14T00:30:05.938ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = b075ff4b-2cd5-4b36-b04d-9dda349220f3 resource = Database result = Ok(()) uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
332024-11-14T00:30:05.941ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3459 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = b075ff4b-2cd5-4b36-b04d-9dda349220f3 response_code = 200 uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
342024-11-14T00:30:05.941ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = f4178777-81dc-4a4b-b65d-6860361d60df roles = RoleSet { roles: {} } uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
352024-11-14T00:30:05.941ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = f4178777-81dc-4a4b-b65d-6860361d60df resource = Database result = Ok(()) uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
362024-11-14T00:30:05.944ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2474 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = f4178777-81dc-4a4b-b65d-6860361d60df response_code = 200 uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
372024-11-14T00:30:06.208ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2151 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 74827a84-0484-483d-b883-f1f6670c5693 remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ce233dd2-477b-41ee-a68b-0e21edc0212b uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-11-14T00:30:06.186156923Z }
382024-11-14T00:30:06.208ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ce233dd2-477b-41ee-a68b-0e21edc0212b roles = RoleSet { roles: {} } uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
392024-11-14T00:30:06.208ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ce233dd2-477b-41ee-a68b-0e21edc0212b resource = Database result = Ok(()) uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
402024-11-14T00:30:06.261ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 74629 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ce233dd2-477b-41ee-a68b-0e21edc0212b response_code = 204 uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
412024-11-14T00:30:06.262ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2151 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 74827a84-0484-483d-b883-f1f6670c5693 remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6204b155-7b3f-4ba6-95f6-4c6df961ed83 uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-11-14T00:30:06.262471306Z }
422024-11-14T00:30:06.262ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6204b155-7b3f-4ba6-95f6-4c6df961ed83 roles = RoleSet { roles: {} } uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
432024-11-14T00:30:06.263ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6204b155-7b3f-4ba6-95f6-4c6df961ed83 resource = Database result = Ok(()) uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
442024-11-14T00:30:06.297ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 34690 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6204b155-7b3f-4ba6-95f6-4c6df961ed83 response_code = 204 uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
452024-11-14T00:30:07.295ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = bbfe2af2-e721-4f7d-a3a2-229b87bd5958 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
462024-11-14T00:30:07.295ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = bbfe2af2-e721-4f7d-a3a2-229b87bd5958 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
472024-11-14T00:30:07.297ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2347 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = bbfe2af2-e721-4f7d-a3a2-229b87bd5958 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
482024-11-14T00:30:08.319ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 38f902f5-751d-47ee-9c21-4eb52f581f97 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
492024-11-14T00:30:08.319ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 38f902f5-751d-47ee-9c21-4eb52f581f97 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
502024-11-14T00:30:08.321ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2050 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 38f902f5-751d-47ee-9c21-4eb52f581f97 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
512024-11-14T00:30:09.342ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6d9342a7-60a9-487a-92a5-ad83025d4948 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
522024-11-14T00:30:09.342ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6d9342a7-60a9-487a-92a5-ad83025d4948 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
532024-11-14T00:30:09.344ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2293 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6d9342a7-60a9-487a-92a5-ad83025d4948 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
542024-11-14T00:30:10.365ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 60c90229-99bf-4c1a-8e35-220073260fdb roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
552024-11-14T00:30:10.365ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 60c90229-99bf-4c1a-8e35-220073260fdb resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
562024-11-14T00:30:10.367ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1922 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 60c90229-99bf-4c1a-8e35-220073260fdb response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
572024-11-14T00:30:11.387ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 0e18cd1e-ef40-450d-b003-b873006cd3be roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
582024-11-14T00:30:11.387ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 0e18cd1e-ef40-450d-b003-b873006cd3be resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
592024-11-14T00:30:11.389ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2489 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 0e18cd1e-ef40-450d-b003-b873006cd3be response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
602024-11-14T00:30:12.411ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ae30a311-ee21-4701-b795-cdee50721be0 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
612024-11-14T00:30:12.411ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ae30a311-ee21-4701-b795-cdee50721be0 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
622024-11-14T00:30:12.413ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2164 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ae30a311-ee21-4701-b795-cdee50721be0 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
632024-11-14T00:30:13.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
642024-11-14T00:30:13.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
652024-11-14T00:30:13.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
662024-11-14T00:30:13.140ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
672024-11-14T00:30:13.140ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
682024-11-14T00:30:13.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
692024-11-14T00:30:13.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
702024-11-14T00:30:13.142ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
712024-11-14T00:30:13.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
722024-11-14T00:30:13.144ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.200949ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
732024-11-14T00:30:13.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
742024-11-14T00:30:13.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
752024-11-14T00:30:13.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
762024-11-14T00:30:13.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
772024-11-14T00:30:13.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
782024-11-14T00:30:13.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
792024-11-14T00:30:13.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6178f930-5fa8-46a0-8fe5-b460818f9d6f, lookup_type: ById(6178f930-5fa8-46a0-8fe5-b460818f9d6f) } result = Ok(())
802024-11-14T00:30:13.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
812024-11-14T00:30:13.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
822024-11-14T00:30:13.159ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f original_time_created = 2024-11-14 00:18:47.526379 UTC target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f time_created = 2024-11-14 00:18:47.526379 UTC
832024-11-14T00:30:13.159ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.981511ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"6178f930-5fa8-46a0-8fe5-b460818f9d6f","time_created":"2024-11-14T00:18:47.526379Z"}
842024-11-14T00:30:13.434ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = b64257df-dd3a-4e24-a0e4-6fccc107bf4c roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
852024-11-14T00:30:13.435ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = b64257df-dd3a-4e24-a0e4-6fccc107bf4c resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
862024-11-14T00:30:13.436ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1922 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = b64257df-dd3a-4e24-a0e4-6fccc107bf4c response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
872024-11-14T00:30:14.457ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 121eb835-0ef2-4dcc-8754-2c769a4aad1e roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
882024-11-14T00:30:14.458ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 121eb835-0ef2-4dcc-8754-2c769a4aad1e resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
892024-11-14T00:30:14.459ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1454 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 121eb835-0ef2-4dcc-8754-2c769a4aad1e response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
902024-11-14T00:30:15.130ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2151 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 74827a84-0484-483d-b883-f1f6670c5693 remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 97cff410-5f45-45cc-bc2b-045d722a64bd uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-11-14T00:30:15.130515046Z }
912024-11-14T00:30:15.130ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 97cff410-5f45-45cc-bc2b-045d722a64bd roles = RoleSet { roles: {} } uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
922024-11-14T00:30:15.131ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 97cff410-5f45-45cc-bc2b-045d722a64bd resource = Database result = Ok(()) uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
932024-11-14T00:30:15.140ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 9378 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 97cff410-5f45-45cc-bc2b-045d722a64bd response_code = 204 uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
942024-11-14T00:30:15.480ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = dfb2a854-a69c-414b-9ba8-813bf220f235 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
952024-11-14T00:30:15.481ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = dfb2a854-a69c-414b-9ba8-813bf220f235 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
962024-11-14T00:30:15.482ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2039 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = dfb2a854-a69c-414b-9ba8-813bf220f235 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
972024-11-14T00:30:15.947ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1082 local_addr = [fd00:1122:3344:101::c]:37372 method = GET remote_addr = [fd00:1122:3344:101::d]:55706 req_id = ee4dd183-e3e8-4095-b93e-bd41a2c2329a response_code = 200 uri = /d06da591-b39e-4924-be5a-46bd48b9952b
982024-11-14T00:30:16.300ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = b0828b74-dba5-4d89-99b3-a7659f635b95 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/8?limit=100
992024-11-14T00:30:16.301ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = b0828b74-dba5-4d89-99b3-a7659f635b95 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/8?limit=100
1002024-11-14T00:30:16.303ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2659 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = b0828b74-dba5-4d89-99b3-a7659f635b95 response_code = 200 uri = /nat/ipv4/changeset/8?limit=100
1012024-11-14T00:30:16.304ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = c9d59d2f-66cd-4762-b401-cf325729e23b roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
1022024-11-14T00:30:16.304ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = c9d59d2f-66cd-4762-b401-cf325729e23b resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
1032024-11-14T00:30:16.307ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3015 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = c9d59d2f-66cd-4762-b401-cf325729e23b response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
1042024-11-14T00:30:16.503ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 3085d331-97b2-4940-9e7e-cce5b8b5f9cc roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
1052024-11-14T00:30:16.504ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 3085d331-97b2-4940-9e7e-cce5b8b5f9cc resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
1062024-11-14T00:30:16.505ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1545 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 3085d331-97b2-4940-9e7e-cce5b8b5f9cc response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
1072024-11-14T00:30:17.308ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = a1247014-cc4e-4b9f-bb46-6536cc40fe79 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
1082024-11-14T00:30:17.308ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = a1247014-cc4e-4b9f-bb46-6536cc40fe79 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
1092024-11-14T00:30:17.312ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3815 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = a1247014-cc4e-4b9f-bb46-6536cc40fe79 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
1102024-11-14T00:30:17.532ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6e6bcb4b-5712-49cd-8781-9eeaa631c8a9 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
1112024-11-14T00:30:17.532ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6e6bcb4b-5712-49cd-8781-9eeaa631c8a9 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
1122024-11-14T00:30:17.534ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2113 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6e6bcb4b-5712-49cd-8781-9eeaa631c8a9 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
1132024-11-14T00:30:17.558ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2151 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 74827a84-0484-483d-b883-f1f6670c5693 remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-14T00:30:17.558339600Z }
1142024-11-14T00:30:17.558ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 roles = RoleSet { roles: {} } uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1152024-11-14T00:30:17.558ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 resource = Database result = Ok(()) uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1162024-11-14T00:30:17.602ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = 74827a84-0484-483d-b883-f1f6670c5693 remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693 vmm_needs_update = true
1172024-11-14T00:30:17.602ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 roles = RoleSet { roles: {} } uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1182024-11-14T00:30:17.602ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 resource = Database result = Ok(()) uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1192024-11-14T00:30:17.609ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 roles = RoleSet { roles: {} } uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1202024-11-14T00:30:17.610ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 resource = Database result = Ok(()) uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1212024-11-14T00:30:17.612ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 roles = RoleSet { roles: {} } uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1222024-11-14T00:30:17.612ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 resource = Database result = Ok(()) uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1232024-11-14T00:30:17.617ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 roles = RoleSet { roles: {} } uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1242024-11-14T00:30:17.617ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 resource = Database result = Ok(()) uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1252024-11-14T00:30:17.619ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 roles = RoleSet { roles: {} } uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1262024-11-14T00:30:17.619ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 resource = Database result = Ok(()) uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1272024-11-14T00:30:17.621ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 roles = RoleSet { roles: {} } uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1282024-11-14T00:30:17.621ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 resource = Database result = Ok(()) uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1292024-11-14T00:30:17.622ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1302024-11-14T00:30:17.624ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bf9014ac-edb7-4293-a1ff-4004060e0581, lookup_type: ById(bf9014ac-edb7-4293-a1ff-4004060e0581) }, key: 17b1ded6-f8f1-45cb-8443-dafcf0963e1d, lookup_type: ById(17b1ded6-f8f1-45cb-8443-dafcf0963e1d) }, key: e21ef448-9c04-42c3-96f9-7b85864c1212, lookup_type: ById(e21ef448-9c04-42c3-96f9-7b85864c1212) } result = Ok(()) uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1312024-11-14T00:30:17.632ZINFOnexus (dropshot_internal): starting update saga for e21ef448-9c04-42c3-96f9-7b85864c1212 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1618 instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-14T00:30:17.558339600Z }
1322024-11-14T00:30:17.632ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 73761 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 response_code = 204 uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1332024-11-14T00:30:17.632ZDEBGnexus (dropshot_internal): preparing instance-update saga for e21ef448-9c04-42c3-96f9-7b85864c1212... actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a41e7c1-0770-4a42-acb5-744534b0da32 uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1342024-11-14T00:30:17.632ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update
1352024-11-14T00:30:17.632ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2151 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 74827a84-0484-483d-b883-f1f6670c5693 remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 40ccaf45-556a-4048-9845-0bd70633d666 uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-14T00:30:17.558339600Z }
1362024-11-14T00:30:17.632ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 40ccaf45-556a-4048-9845-0bd70633d666 roles = RoleSet { roles: {} } uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1372024-11-14T00:30:17.633ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 40ccaf45-556a-4048-9845-0bd70633d666 resource = Database result = Ok(()) uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1382024-11-14T00:30:17.634ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"e21ef448-9c04-42c3-96f9-7b85864c1212","lookup_type":{"ById":"e21ef448-9c04-42c3-96f9-7b85864c1212"},"parent":{"key":"17b1ded6-f8f1-45cb-8443-dafcf0963e1d","lookup_type":{"ById":"17b1ded6-f8f1-45cb-8443-dafcf0963e1d"},"parent":{"key":"bf9014ac-edb7-4293-a1ff-4004060e0581","lookup_type":{"ById":"bf9014ac-edb7-4293-a1ff-4004060e0581"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
1392024-11-14T00:30:17.634ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update
1402024-11-14T00:30:17.639ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 6785 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 40ccaf45-556a-4048-9845-0bd70633d666 response_code = 204 uri = /vmms/74827a84-0484-483d-b883-f1f6670c5693
1412024-11-14T00:30:17.676ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update
1422024-11-14T00:30:17.677ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
1432024-11-14T00:30:17.679ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
1442024-11-14T00:30:17.679ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81
1452024-11-14T00:30:17.685ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
1462024-11-14T00:30:17.685ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81
1472024-11-14T00:30:17.690ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
1482024-11-14T00:30:17.690ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81
1492024-11-14T00:30:17.704ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
1502024-11-14T00:30:17.704ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("0236d474-7b87-46b6-874a-465ab3af99e5")) node_id = 0 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81
1512024-11-14T00:30:17.708ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
1522024-11-14T00:30:17.708ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81
1532024-11-14T00:30:17.726ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update
1542024-11-14T00:30:17.732ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1552024-11-14T00:30:17.732ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1562024-11-14T00:30:17.734ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1572024-11-14T00:30:17.734ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1582024-11-14T00:30:17.736ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1592024-11-14T00:30:17.736ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1602024-11-14T00:30:17.738ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1612024-11-14T00:30:17.738ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1622024-11-14T00:30:17.740ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1632024-11-14T00:30:17.740ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1642024-11-14T00:30:17.741ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1652024-11-14T00:30:17.741ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1662024-11-14T00:30:17.742ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1672024-11-14T00:30:17.745ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bf9014ac-edb7-4293-a1ff-4004060e0581, lookup_type: ById(bf9014ac-edb7-4293-a1ff-4004060e0581) }, key: 17b1ded6-f8f1-45cb-8443-dafcf0963e1d, lookup_type: ById(17b1ded6-f8f1-45cb-8443-dafcf0963e1d) }, key: e21ef448-9c04-42c3-96f9-7b85864c1212, lookup_type: ById(e21ef448-9c04-42c3-96f9-7b85864c1212) } result = Ok(()) saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1682024-11-14T00:30:17.745ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance updater_id = 0236d474-7b87-46b6-874a-465ab3af99e5
1692024-11-14T00:30:17.745ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1702024-11-14T00:30:17.745ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1712024-11-14T00:30:17.760ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance updater_id = 0236d474-7b87-46b6-874a-465ab3af99e5
1722024-11-14T00:30:17.760ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1732024-11-14T00:30:17.761ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance
1742024-11-14T00:30:17.773ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 locked_gen = Generation(Generation(2)) saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = LockInstance updater_id = 0236d474-7b87-46b6-874a-465ab3af99e5
1752024-11-14T00:30:17.775ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
1762024-11-14T00:30:17.775ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("0236d474-7b87-46b6-874a-465ab3af99e5")}) node_id = 1 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81
1772024-11-14T00:30:17.779ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
1782024-11-14T00:30:17.779ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81
1792024-11-14T00:30:17.789ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
1802024-11-14T00:30:17.789ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
1812024-11-14T00:30:17.790ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
1822024-11-14T00:30:17.791ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
1832024-11-14T00:30:17.792ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
1842024-11-14T00:30:17.792ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
1852024-11-14T00:30:17.794ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
1862024-11-14T00:30:17.797ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bf9014ac-edb7-4293-a1ff-4004060e0581, lookup_type: ById(bf9014ac-edb7-4293-a1ff-4004060e0581) }, key: 17b1ded6-f8f1-45cb-8443-dafcf0963e1d, lookup_type: ById(17b1ded6-f8f1-45cb-8443-dafcf0963e1d) }, key: e21ef448-9c04-42c3-96f9-7b85864c1212, lookup_type: ById(e21ef448-9c04-42c3-96f9-7b85864c1212) } result = Ok(()) saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
1872024-11-14T00:30:17.797ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
1882024-11-14T00:30:17.797ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
1892024-11-14T00:30:17.807ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: 74827a84-0484-483d-b883-f1f6670c5693, time_created: 2024-11-14T00:29:56.981893Z, time_deleted: None, instance_id: e21ef448-9c04-42c3-96f9-7b85864c1212, sled_id: e211d163-be78-4663-b73f-baf5e9462a10, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2024-11-14T00:30:17.558339Z, gen: Generation(Generation(8)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2024-11-14T00:29:54.960059Z, gen: Generation(Generation(3)), propolis_id: Some(74827a84-0484-483d-b883-f1f6670c5693), dst_propolis_id: None, migration_id: None, nexus_state: Vmm, time_last_auto_restarted: None } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(74827a84-0484-483d-b883-f1f6670c5693 (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2024-11-14T00:30:17.807639808Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None }
1902024-11-14T00:30:17.814ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
1912024-11-14T00:30:17.817ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"e21ef448-9c04-42c3-96f9-7b85864c1212","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"74827a84-0484-483d-b883-f1f6670c5693"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Action":{"action_name":"instance_update.chain_successor_saga","label":"ChainSuccessorSaga","name":"chain_successor_saga"}},{"Start":{"params":{"authz_instance":{"key":"e21ef448-9c04-42c3-96f9-7b85864c1212","lookup_type":{"ById":"e21ef448-9c04-42c3-96f9-7b85864c1212"},"parent":{"key":"17b1ded6-f8f1-45cb-8443-dafcf0963e1d","lookup_type":{"ById":"17b1ded6-f8f1-45cb-8443-dafcf0963e1d"},"parent":{"key":"bf9014ac-edb7-4293-a1ff-4004060e0581","lookup_type":{"ById":"bf9014ac-edb7-4293-a1ff-4004060e0581"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"0236d474-7b87-46b6-874a-465ab3af99e5"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"17b1ded6-f8f1-45cb-8443-dafcf0963e1d","ram_diff":1073741824},"destroy_active_vmm":"74827a84-0484-483d-b883-f1f6670c5693","destroy_target_vmm":null,"network_config":"Delete","new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2024-11-14T00:30:17.807639808Z"}}}}},"End"]},"saga_name":"instance-update","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
1922024-11-14T00:30:17.817ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
1932024-11-14T00:30:17.821ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
1942024-11-14T00:30:17.821ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
1952024-11-14T00:30:17.821ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
1962024-11-14T00:30:17.821ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
1972024-11-14T00:30:17.828ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
1982024-11-14T00:30:17.828ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
1992024-11-14T00:30:17.831ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2002024-11-14T00:30:17.831ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2012024-11-14T00:30:17.834ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2022024-11-14T00:30:17.834ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("afb3571e-1970-478f-a126-e956b18191a8")) node_id = 0 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2032024-11-14T00:30:17.837ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2042024-11-14T00:30:17.837ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2052024-11-14T00:30:17.845ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 parent_lock = UpdaterLock { updater_id: 0236d474-7b87-46b6-874a-465ab3af99e5, locked_gen: Generation(Generation(2)) } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
2062024-11-14T00:30:17.845ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = BecomeUpdater
2072024-11-14T00:30:17.846ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = BecomeUpdater
2082024-11-14T00:30:17.854ZDEBGnexus (SagaExecutor): inherited lock from 0236d474-7b87-46b6-874a-465ab3af99e5 to afb3571e-1970-478f-a126-e956b18191a8 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = 0236d474-7b87-46b6-874a-465ab3af99e5 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = BecomeUpdater updater_id = afb3571e-1970-478f-a126-e956b18191a8
2092024-11-14T00:30:17.854ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:964 instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
2102024-11-14T00:30:17.854ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2112024-11-14T00:30:17.854ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("afb3571e-1970-478f-a126-e956b18191a8")}) node_id = 1 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2122024-11-14T00:30:17.857ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2132024-11-14T00:30:17.857ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2142024-11-14T00:30:17.861ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2152024-11-14T00:30:17.861ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2162024-11-14T00:30:17.864ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2172024-11-14T00:30:17.864ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2182024-11-14T00:30:17.867ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:1017 instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
2192024-11-14T00:30:17.869ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212
2202024-11-14T00:30:17.869ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = UpdateNetworkConfig
2212024-11-14T00:30:17.869ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = UpdateNetworkConfig
2222024-11-14T00:30:17.876ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = UpdateNetworkConfig
2232024-11-14T00:30:17.876ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = UpdateNetworkConfig
2242024-11-14T00:30:17.880ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
2252024-11-14T00:30:17.880ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = UpdateNetworkConfig
2262024-11-14T00:30:17.880ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = UpdateNetworkConfig
2272024-11-14T00:30:17.882ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
2282024-11-14T00:30:17.882ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
2292024-11-14T00:30:17.882ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
2302024-11-14T00:30:17.886ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
2312024-11-14T00:30:17.887ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
2322024-11-14T00:30:17.887ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
2332024-11-14T00:30:17.887ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
2342024-11-14T00:30:17.892ZDEBGnexus: notifying dendrite of updates instance_id = Some(e21ef448-9c04-42c3-96f9-7b85864c1212 (instance)) switch = switch0
2352024-11-14T00:30:17.892ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1717, tv_nsec: 609444165 } })
2362024-11-14T00:30:17.892ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1131
2372024-11-14T00:30:17.923ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
2382024-11-14T00:30:17.923ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2392024-11-14T00:30:17.923ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4362d6d5-a35f-4d61-937e-09b2e565c529", "content-length": "26", "date": "Thu, 14 Nov 2024 00:30:17 GMT"} })
2402024-11-14T00:30:17.923ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2412024-11-14T00:30:17.924ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2422024-11-14T00:30:17.953ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
2432024-11-14T00:30:17.953ZDEBGnexus (DpdClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger", status: 200, headers: {"content-type": "application/json", "x-request-id": "e51b880c-3e0a-4fb8-b519-be4afbc929c2", "content-length": "4", "date": "Thu, 14 Nov 2024 00:30:16 GMT"} })
2442024-11-14T00:30:17.955ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2452024-11-14T00:30:17.955ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2462024-11-14T00:30:17.959ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2472024-11-14T00:30:17.959ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2482024-11-14T00:30:17.964ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
2492024-11-14T00:30:17.965ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
2502024-11-14T00:30:18.018ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1092 instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2024-11-14T00:30:17.967804Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 17b1ded6-f8f1-45cb-8443-dafcf0963e1d, time_modified: Some(2024-11-14T00:30:17.967804Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: bf9014ac-edb7-4293-a1ff-4004060e0581, time_modified: Some(2024-11-14T00:30:17.967804Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
2512024-11-14T00:30:18.018ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2522024-11-14T00:30:18.018ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2532024-11-14T00:30:18.022ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2542024-11-14T00:30:18.022ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2552024-11-14T00:30:18.026ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1127 instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
2562024-11-14T00:30:18.026ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = UnassignOximeterProducer
2572024-11-14T00:30:18.026ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = UnassignOximeterProducer
2582024-11-14T00:30:18.031ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = ad042fde-503a-4314-8c8c-afa2f783d145 producer_id = e21ef448-9c04-42c3-96f9-7b85864c1212 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
2592024-11-14T00:30:18.031ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = UnassignOximeterProducer
2602024-11-14T00:30:18.031ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = UnassignOximeterProducer
2612024-11-14T00:30:18.068ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:262 id = e21ef448-9c04-42c3-96f9-7b85864c1212 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
2622024-11-14T00:30:18.070ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = e21ef448-9c04-42c3-96f9-7b85864c1212 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/e21ef448-9c04-42c3-96f9-7b85864c1212
2632024-11-14T00:30:18.081ZDEBGnexus (SagaExecutor): client response oximeter-collector = e21ef448-9c04-42c3-96f9-7b85864c1212 result = Ok(Response { url: "http://[fd00:1122:3344:101::d]:12223/producers/e21ef448-9c04-42c3-96f9-7b85864c1212", status: 204, headers: {"x-request-id": "19e525ae-238b-471b-94c3-2413580063fe", "date": "Thu, 14 Nov 2024 00:30:18 GMT"} }) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
2642024-11-14T00:30:18.081ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = ad042fde-503a-4314-8c8c-afa2f783d145 producer_id = e21ef448-9c04-42c3-96f9-7b85864c1212 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
2652024-11-14T00:30:18.082ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2662024-11-14T00:30:18.082ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2672024-11-14T00:30:18.085ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2682024-11-14T00:30:18.085ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2692024-11-14T00:30:18.089ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 lock = UpdaterLock { updater_id: afb3571e-1970-478f-a126-e956b18191a8, locked_gen: Generation(Generation(3)) } new_runtime = InstanceRuntimeState { time_updated: 2024-11-14T00:30:17.807639808Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
2702024-11-14T00:30:18.089ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = CommitInstanceUpdates
2712024-11-14T00:30:18.090ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = CommitInstanceUpdates
2722024-11-14T00:30:18.130ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1176 instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 new_runtime = InstanceRuntimeState { time_updated: 2024-11-14T00:30:17.807639808Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
2732024-11-14T00:30:18.130ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2742024-11-14T00:30:18.130ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2752024-11-14T00:30:18.130ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Signaled
2762024-11-14T00:30:18.130ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Signaled
2772024-11-14T00:30:18.130ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
2782024-11-14T00:30:18.130ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2792024-11-14T00:30:18.130ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2802024-11-14T00:30:18.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2812024-11-14T00:30:18.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2822024-11-14T00:30:18.132ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2832024-11-14T00:30:18.133ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2842024-11-14T00:30:18.133ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2852024-11-14T00:30:18.133ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2862024-11-14T00:30:18.133ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2872024-11-14T00:30:18.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2882024-11-14T00:30:18.135ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2892024-11-14T00:30:18.136ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2902024-11-14T00:30:18.136ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("e21ef448-9c04-42c3-96f9-7b85864c1212"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("74827a84-0484-483d-b883-f1f6670c5693")}) node_id = 7 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2912024-11-14T00:30:18.137ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2922024-11-14T00:30:18.137ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2932024-11-14T00:30:18.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2942024-11-14T00:30:18.142ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2952024-11-14T00:30:18.142ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2962024-11-14T00:30:18.145ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2972024-11-14T00:30:18.145ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
2982024-11-14T00:30:18.148ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
2992024-11-14T00:30:18.148ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
3002024-11-14T00:30:18.154ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:83 instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 propolis_id = 74827a84-0484-483d-b883-f1f6670c5693 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
3012024-11-14T00:30:18.154ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = ReleaseSledResources
3022024-11-14T00:30:18.154ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = ReleaseSledResources
3032024-11-14T00:30:18.164ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3042024-11-14T00:30:18.164ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
3052024-11-14T00:30:18.168ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3062024-11-14T00:30:18.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3072024-11-14T00:30:18.168ZINFOnexus (BackgroundTasks): VPC route manager sled e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3082024-11-14T00:30:18.168ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3092024-11-14T00:30:18.169ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6284429a-9be0-44fa-ac17-11e3b4859e8a", "content-length": "372", "date": "Thu, 14 Nov 2024 00:30:18 GMT"} })
3102024-11-14T00:30:18.169ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3112024-11-14T00:30:18.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3122024-11-14T00:30:18.172ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3132024-11-14T00:30:18.172ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
3142024-11-14T00:30:18.175ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:114 instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 propolis_id = 74827a84-0484-483d-b883-f1f6670c5693 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
3152024-11-14T00:30:18.175ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = MarkVmmDeleted
3162024-11-14T00:30:18.175ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = MarkVmmDeleted
3172024-11-14T00:30:18.177ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3182024-11-14T00:30:18.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3192024-11-14T00:30:18.179ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3202024-11-14T00:30:18.181ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3212024-11-14T00:30:18.181ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3222024-11-14T00:30:18.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3232024-11-14T00:30:18.182ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3242024-11-14T00:30:18.183ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3252024-11-14T00:30:18.183ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
3262024-11-14T00:30:18.184ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3272024-11-14T00:30:18.184ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3282024-11-14T00:30:18.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3292024-11-14T00:30:18.187ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3302024-11-14T00:30:18.187ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
3312024-11-14T00:30:18.190ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3322024-11-14T00:30:18.190ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
3332024-11-14T00:30:18.193ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3342024-11-14T00:30:18.193ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
3352024-11-14T00:30:18.196ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = ChainSuccessorSaga
3362024-11-14T00:30:18.196ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = ChainSuccessorSaga
3372024-11-14T00:30:18.197ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = ChainSuccessorSaga
3382024-11-14T00:30:18.198ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = ChainSuccessorSaga
3392024-11-14T00:30:18.199ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = ChainSuccessorSaga
3402024-11-14T00:30:18.199ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = ChainSuccessorSaga
3412024-11-14T00:30:18.201ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = ChainSuccessorSaga
3422024-11-14T00:30:18.204ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bf9014ac-edb7-4293-a1ff-4004060e0581, lookup_type: ById(bf9014ac-edb7-4293-a1ff-4004060e0581) }, key: 17b1ded6-f8f1-45cb-8443-dafcf0963e1d, lookup_type: ById(17b1ded6-f8f1-45cb-8443-dafcf0963e1d) }, key: e21ef448-9c04-42c3-96f9-7b85864c1212, lookup_type: ById(e21ef448-9c04-42c3-96f9-7b85864c1212) } result = Ok(()) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = ChainSuccessorSaga
3432024-11-14T00:30:18.204ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = ChainSuccessorSaga
3442024-11-14T00:30:18.204ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_node = ChainSuccessorSaga
3452024-11-14T00:30:18.209ZDEBGnexus (SagaExecutor): instance update: instance will not reincarnate auto_restart_config = InstanceAutoRestart { policy: None, cooldown: None } instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 karmic_state = WillReincarnate needs_reincarnation = false saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update
3462024-11-14T00:30:18.209ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3472024-11-14T00:30:18.209ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
3482024-11-14T00:30:18.212ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3492024-11-14T00:30:18.212ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
3502024-11-14T00:30:18.215ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3512024-11-14T00:30:18.215ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
3522024-11-14T00:30:18.218ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3532024-11-14T00:30:18.218ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4
3542024-11-14T00:30:18.228ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
3552024-11-14T00:30:18.228ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3562024-11-14T00:30:18.229ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "432dc42d-23fb-45ad-b35b-f997cb86b50b", "date": "Thu, 14 Nov 2024 00:30:18 GMT"} })
3572024-11-14T00:30:18.229ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3582024-11-14T00:30:18.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3592024-11-14T00:30:18.231ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3602024-11-14T00:30:18.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3612024-11-14T00:30:18.234ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3622024-11-14T00:30:18.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3632024-11-14T00:30:18.237ZINFOnexus (BackgroundTasks): VPC route manager sled e211d163-be78-4663-b73f-baf5e9462a10 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3642024-11-14T00:30:18.237ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3652024-11-14T00:30:18.237ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e5844944-f893-4403-8f12-87eb18dd3559", "date": "Thu, 14 Nov 2024 00:30:18 GMT"} })
3662024-11-14T00:30:18.238ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 108.770931ms iteration = 24 status = {}
3672024-11-14T00:30:18.252ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3682024-11-14T00:30:18.252ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 saga_name = instance-update saga_result = SagaResult { saga_id: f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4, saga_log: SagaLog { saga_id: f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(String("afb3571e-1970-478f-a126-e956b18191a8")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("afb3571e-1970-478f-a126-e956b18191a8")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("e21ef448-9c04-42c3-96f9-7b85864c1212"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("74827a84-0484-483d-b883-f1f6670c5693")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "chain_successor_saga": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("e21ef448-9c04-42c3-96f9-7b85864c1212"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("74827a84-0484-483d-b883-f1f6670c5693")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("afb3571e-1970-478f-a126-e956b18191a8"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("afb3571e-1970-478f-a126-e956b18191a8")}} }) }
3692024-11-14T00:30:18.253ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = f9748a40-1cfa-46b3-8c2b-6ec4aa7b6dc4 instance_id = e21ef448-9c04-42c3-96f9-7b85864c1212 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update
3702024-11-14T00:30:18.253ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3712024-11-14T00:30:18.253ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81
3722024-11-14T00:30:18.266ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
3732024-11-14T00:30:18.266ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f61a45ec-afa1-4b31-a980-706a17d8c616", "content-length": "691", "date": "Thu, 14 Nov 2024 00:30:18 GMT"} })
3742024-11-14T00:30:18.266ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
3752024-11-14T00:30:18.266ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
3762024-11-14T00:30:18.266ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
3772024-11-14T00:30:18.267ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 137.621996ms iteration = 24 status = {}
3782024-11-14T00:30:18.269ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3792024-11-14T00:30:18.269ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81
3802024-11-14T00:30:18.273ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3812024-11-14T00:30:18.273ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81
3822024-11-14T00:30:18.276ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3832024-11-14T00:30:18.276ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81
3842024-11-14T00:30:18.308ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update sec_id = d06da591-b39e-4924-be5a-46bd48b9952b
3852024-11-14T00:30:18.308ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 191dd52c-d7c0-4a74-a6f0-bc9683599d81 saga_name = start-instance-update saga_result = SagaResult { saga_id: 191dd52c-d7c0-4a74-a6f0-bc9683599d81, saga_log: SagaLog { saga_id: 191dd52c-d7c0-4a74-a6f0-bc9683599d81, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("0236d474-7b87-46b6-874a-465ab3af99e5")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("0236d474-7b87-46b6-874a-465ab3af99e5")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("0236d474-7b87-46b6-874a-465ab3af99e5"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("0236d474-7b87-46b6-874a-465ab3af99e5")}} }) }
3862024-11-14T00:30:18.313ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = f845547c-7737-4765-a304-e833f7590e8d roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
3872024-11-14T00:30:18.314ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = f845547c-7737-4765-a304-e833f7590e8d resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
3882024-11-14T00:30:18.317ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3221 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = f845547c-7737-4765-a304-e833f7590e8d response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
3892024-11-14T00:30:18.553ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = db506058-23e6-4285-b8c2-913dff12737f roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
3902024-11-14T00:30:18.553ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = db506058-23e6-4285-b8c2-913dff12737f resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
3912024-11-14T00:30:18.555ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2056 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = db506058-23e6-4285-b8c2-913dff12737f response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
3922024-11-14T00:30:19.319ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = f08f33fe-7aad-4ca7-a0ca-be41a18aeb39 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
3932024-11-14T00:30:19.319ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = f08f33fe-7aad-4ca7-a0ca-be41a18aeb39 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
3942024-11-14T00:30:19.321ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2716 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = f08f33fe-7aad-4ca7-a0ca-be41a18aeb39 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
3952024-11-14T00:30:19.577ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = d50e04a9-c319-4bf5-aa49-b2f1099cf85c roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
3962024-11-14T00:30:19.577ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = d50e04a9-c319-4bf5-aa49-b2f1099cf85c resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
3972024-11-14T00:30:19.579ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1817 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = d50e04a9-c319-4bf5-aa49-b2f1099cf85c response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
3982024-11-14T00:30:20.599ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 83dc9191-2957-40eb-a72d-7e89ceeb69c7 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
3992024-11-14T00:30:20.600ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 83dc9191-2957-40eb-a72d-7e89ceeb69c7 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
4002024-11-14T00:30:20.602ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2382 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 83dc9191-2957-40eb-a72d-7e89ceeb69c7 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
4012024-11-14T00:30:20.938ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = 3fb6100c-6c86-43c3-9a7f-6d5624df9af6 roles = RoleSet { roles: {} } uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
4022024-11-14T00:30:20.938ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = 3fb6100c-6c86-43c3-9a7f-6d5624df9af6 resource = Database result = Ok(()) uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
4032024-11-14T00:30:20.942ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 5039 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = 3fb6100c-6c86-43c3-9a7f-6d5624df9af6 response_code = 200 uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
4042024-11-14T00:30:20.943ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = dde74b74-7f13-4aa7-ab43-09a1759a53f8 roles = RoleSet { roles: {} } uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
4052024-11-14T00:30:20.943ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = dde74b74-7f13-4aa7-ab43-09a1759a53f8 resource = Database result = Ok(()) uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
4062024-11-14T00:30:20.946ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3267 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = dde74b74-7f13-4aa7-ab43-09a1759a53f8 response_code = 200 uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
4072024-11-14T00:30:21.623ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 17506a39-5b94-42f6-940e-65cf317d1518 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
4082024-11-14T00:30:21.623ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 17506a39-5b94-42f6-940e-65cf317d1518 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
4092024-11-14T00:30:21.625ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2233 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 17506a39-5b94-42f6-940e-65cf317d1518 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
4102024-11-14T00:30:22.647ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ba506083-4ecf-4b38-8d42-1a3e91c7b464 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
4112024-11-14T00:30:22.647ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ba506083-4ecf-4b38-8d42-1a3e91c7b464 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
4122024-11-14T00:30:22.649ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2061 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ba506083-4ecf-4b38-8d42-1a3e91c7b464 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
4132024-11-14T00:30:23.139ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
4142024-11-14T00:30:23.139ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4152024-11-14T00:30:23.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
4162024-11-14T00:30:23.139ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
4172024-11-14T00:30:23.139ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4182024-11-14T00:30:23.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4192024-11-14T00:30:23.139ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
4202024-11-14T00:30:23.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4212024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4222024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4232024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
4242024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
4252024-11-14T00:30:23.140ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
4262024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4272024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
4282024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4292024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4302024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
4312024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
4322024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
4332024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4342024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4352024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
4362024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4372024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
4382024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
4392024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4402024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
4412024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4422024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4432024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
4442024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
4452024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
4462024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4472024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
4482024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
4492024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
4502024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4512024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4522024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4532024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4542024-11-14T00:30:23.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4552024-11-14T00:30:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4562024-11-14T00:30:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4572024-11-14T00:30:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4582024-11-14T00:30:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4592024-11-14T00:30:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
4602024-11-14T00:30:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4612024-11-14T00:30:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4622024-11-14T00:30:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4632024-11-14T00:30:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
4642024-11-14T00:30:23.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4652024-11-14T00:30:23.142ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1722, tv_nsec: 859699937 } })
4662024-11-14T00:30:23.143ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
4672024-11-14T00:30:23.143ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4682024-11-14T00:30:23.143ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4692024-11-14T00:30:23.143ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.389212ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
4702024-11-14T00:30:23.143ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1722, tv_nsec: 860860133 } })
4712024-11-14T00:30:23.144ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
4722024-11-14T00:30:23.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4732024-11-14T00:30:23.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4742024-11-14T00:30:23.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4752024-11-14T00:30:23.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
4762024-11-14T00:30:23.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
4772024-11-14T00:30:23.146ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4782024-11-14T00:30:23.146ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4792024-11-14T00:30:23.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4802024-11-14T00:30:23.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4812024-11-14T00:30:23.147ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4822024-11-14T00:30:23.148ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4832024-11-14T00:30:23.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4842024-11-14T00:30:23.148ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4852024-11-14T00:30:23.148ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 8.152685ms iteration = 24 status = {"errors":[],"records_set_to_done":[]}
4862024-11-14T00:30:23.148ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4872024-11-14T00:30:23.148ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
4882024-11-14T00:30:23.148ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4892024-11-14T00:30:23.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4902024-11-14T00:30:23.149ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4912024-11-14T00:30:23.149ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4922024-11-14T00:30:23.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4932024-11-14T00:30:23.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4942024-11-14T00:30:23.151ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4952024-11-14T00:30:23.151ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4962024-11-14T00:30:23.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4972024-11-14T00:30:23.153ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 12.318264ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
4982024-11-14T00:30:23.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4992024-11-14T00:30:23.153ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5002024-11-14T00:30:23.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5012024-11-14T00:30:23.159ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 18.794534ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5022024-11-14T00:30:23.159ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1722, tv_nsec: 876872939 } })
5032024-11-14T00:30:23.160ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864
5042024-11-14T00:30:23.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5052024-11-14T00:30:23.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5062024-11-14T00:30:23.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5072024-11-14T00:30:23.165ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5082024-11-14T00:30:23.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5092024-11-14T00:30:23.166ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5102024-11-14T00:30:23.166ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5112024-11-14T00:30:23.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5122024-11-14T00:30:23.167ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
5132024-11-14T00:30:23.167ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 26.850297ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
5142024-11-14T00:30:23.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6178f930-5fa8-46a0-8fe5-b460818f9d6f, lookup_type: ById(6178f930-5fa8-46a0-8fe5-b460818f9d6f) } result = Ok(())
5152024-11-14T00:30:23.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5162024-11-14T00:30:23.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5172024-11-14T00:30:23.168ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5182024-11-14T00:30:23.168ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5192024-11-14T00:30:23.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5202024-11-14T00:30:23.171ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5212024-11-14T00:30:23.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5222024-11-14T00:30:23.173ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5232024-11-14T00:30:23.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5242024-11-14T00:30:23.183ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f original_time_created = 2024-11-14 00:18:47.526379 UTC target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f time_created = 2024-11-14 00:18:47.526379 UTC
5252024-11-14T00:30:23.183ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 43.570183ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"6178f930-5fa8-46a0-8fe5-b460818f9d6f","time_created":"2024-11-14T00:18:47.526379Z"}
5262024-11-14T00:30:23.189ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5272024-11-14T00:30:23.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5282024-11-14T00:30:23.195ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5292024-11-14T00:30:23.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5302024-11-14T00:30:23.196ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5312024-11-14T00:30:23.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5322024-11-14T00:30:23.203ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5332024-11-14T00:30:23.203ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
5342024-11-14T00:30:23.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5352024-11-14T00:30:23.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5362024-11-14T00:30:23.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5372024-11-14T00:30:23.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5382024-11-14T00:30:23.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5392024-11-14T00:30:23.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5402024-11-14T00:30:23.230ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 89.881045ms iteration = 27 status = {"physical_disks_added":0}
5412024-11-14T00:30:23.235ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5422024-11-14T00:30:23.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5432024-11-14T00:30:23.236ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5442024-11-14T00:30:23.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5452024-11-14T00:30:23.242ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 101.67504ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5462024-11-14T00:30:23.247ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
5472024-11-14T00:30:23.247ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5482024-11-14T00:30:23.247ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1dea8fe-4b96-4184-9172-aebd2c441130", "content-length": "26", "date": "Thu, 14 Nov 2024 00:30:23 GMT"} })
5492024-11-14T00:30:23.247ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5502024-11-14T00:30:23.249ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5512024-11-14T00:30:23.257ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
5522024-11-14T00:30:23.257ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5532024-11-14T00:30:23.257ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b2d9675-2a3d-4e8b-875b-bda2c659e8d1", "content-length": "26", "date": "Thu, 14 Nov 2024 00:30:23 GMT"} })
5542024-11-14T00:30:23.257ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5552024-11-14T00:30:23.260ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5562024-11-14T00:30:23.277ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 zone_address = fd00:1122:3344:101::2
5572024-11-14T00:30:23.277ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5582024-11-14T00:30:23.278ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4df62092-9155-464f-9b6d-dd8c5db463c7", "content-length": "26", "date": "Thu, 14 Nov 2024 00:30:23 GMT"} })
5592024-11-14T00:30:23.278ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5602024-11-14T00:30:23.280ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864
5612024-11-14T00:30:23.280ZINFOnexus (BackgroundTasks): VPC route manager sled e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
5622024-11-14T00:30:23.280ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5632024-11-14T00:30:23.281ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b1eacaa-dd8c-42f5-98ff-d05f69bee397", "content-length": "372", "date": "Thu, 14 Nov 2024 00:30:23 GMT"} })
5642024-11-14T00:30:23.281ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5652024-11-14T00:30:23.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5662024-11-14T00:30:23.283ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5672024-11-14T00:30:23.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: e211d163-be78-4663-b73f-baf5e9462a10, lookup_type: ById(e211d163-be78-4663-b73f-baf5e9462a10) } result = Ok(())
5682024-11-14T00:30:23.287ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5692024-11-14T00:30:23.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5702024-11-14T00:30:23.289ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 149.301711ms iteration = 24 status = 0
5712024-11-14T00:30:23.297ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
5722024-11-14T00:30:23.297ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
5732024-11-14T00:30:23.297ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "51afad49-ec13-4f22-8f03-efde13d6e99c", "date": "Thu, 14 Nov 2024 00:30:23 GMT"} })
5742024-11-14T00:30:23.297ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5752024-11-14T00:30:23.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5762024-11-14T00:30:23.299ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5772024-11-14T00:30:23.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5782024-11-14T00:30:23.303ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5792024-11-14T00:30:23.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5802024-11-14T00:30:23.307ZINFOnexus (BackgroundTasks): VPC route manager sled e211d163-be78-4663-b73f-baf5e9462a10 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
5812024-11-14T00:30:23.307ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5822024-11-14T00:30:23.307ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ae75004b-7b19-4d83-bddc-41f09c9b3a0d", "date": "Thu, 14 Nov 2024 00:30:23 GMT"} })
5832024-11-14T00:30:23.309ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 168.734494ms iteration = 25 status = {}
5842024-11-14T00:30:23.314ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 173.44533ms iteration = 24 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
5852024-11-14T00:30:23.314ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 173.40746ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
5862024-11-14T00:30:23.318ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5872024-11-14T00:30:23.318ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
5882024-11-14T00:30:23.318ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 177.792401ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
5892024-11-14T00:30:23.339ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5902024-11-14T00:30:23.339ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "e03b4a30-12aa-4410-853e-8ca2cb7907a8", "content-length": "2", "date": "Thu, 14 Nov 2024 00:30:22 GMT"} })
5912024-11-14T00:30:23.340ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5922024-11-14T00:30:23.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5932024-11-14T00:30:23.342ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 203.403902ms iteration = 24 status = null
5942024-11-14T00:30:23.376ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5952024-11-14T00:30:23.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5962024-11-14T00:30:23.378ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5972024-11-14T00:30:23.380ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5982024-11-14T00:30:23.380ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5992024-11-14T00:30:23.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6002024-11-14T00:30:23.381ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6012024-11-14T00:30:23.383ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6022024-11-14T00:30:23.383ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6032024-11-14T00:30:23.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6042024-11-14T00:30:23.399ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 260.198331ms iteration = 24 status = {}
6052024-11-14T00:30:23.454ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
6062024-11-14T00:30:23.454ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ef9ffa4-c113-475e-9eed-a2acec073191", "content-length": "691", "date": "Thu, 14 Nov 2024 00:30:23 GMT"} })
6072024-11-14T00:30:23.455ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
6082024-11-14T00:30:23.455ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
6092024-11-14T00:30:23.455ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
6102024-11-14T00:30:23.457ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 316.500924ms iteration = 25 status = {}
6112024-11-14T00:30:23.498ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6122024-11-14T00:30:23.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6132024-11-14T00:30:23.500ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1b492b5b-8c3a-4e99-9528-5d52a91581c5, rack_id: fcfbb675-622a-4d07-9c7d-02f1c7317864, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d5fc9a38-ca99-44ad-95a3-d22ca6bdfc5b) } rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 switch_location = Switch0
6142024-11-14T00:30:23.500ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6152024-11-14T00:30:23.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6162024-11-14T00:30:23.511ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
6172024-11-14T00:30:23.511ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b1a26ed-430a-43fb-af47-4e7b8b2a8f66", "content-length": "134", "date": "Thu, 14 Nov 2024 00:30:22 GMT"} })
6182024-11-14T00:30:23.511ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
6192024-11-14T00:30:23.511ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d5fc9a38-ca99-44ad-95a3-d22ca6bdfc5b
6202024-11-14T00:30:23.511ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
6212024-11-14T00:30:23.511ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
6222024-11-14T00:30:23.511ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6232024-11-14T00:30:23.511ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6242024-11-14T00:30:23.511ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6252024-11-14T00:30:23.511ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6262024-11-14T00:30:23.511ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6272024-11-14T00:30:23.511ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6282024-11-14T00:30:23.512ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864
6292024-11-14T00:30:23.512ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6302024-11-14T00:30:23.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6312024-11-14T00:30:23.514ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
6322024-11-14T00:30:23.514ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "abc32f2d-5804-45f5-8447-0435e12512af", "content-length": "2", "date": "Thu, 14 Nov 2024 00:30:22 GMT"} })
6332024-11-14T00:30:23.514ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
6342024-11-14T00:30:23.514ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4fa4559-b258-426d-8b0e-31d48b3491b9", "content-length": "52", "date": "Thu, 14 Nov 2024 00:30:22 GMT"} })
6352024-11-14T00:30:23.514ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 switch_location = Switch0
6362024-11-14T00:30:23.514ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
6372024-11-14T00:30:23.633ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "a0e3eac4-2135-48ac-989f-1695dbd9b7bd", "date": "Thu, 14 Nov 2024 00:30:23 GMT"} })
6382024-11-14T00:30:23.633ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 switch = Switch0
6392024-11-14T00:30:23.633ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
6402024-11-14T00:30:23.633ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "57d959f5-1c9a-4613-b76a-fea67ac534b8", "content-length": "682", "date": "Thu, 14 Nov 2024 00:30:23 GMT"} })
6412024-11-14T00:30:23.633ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6422024-11-14T00:30:23.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6432024-11-14T00:30:23.637ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6442024-11-14T00:30:23.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6452024-11-14T00:30:23.640ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6462024-11-14T00:30:23.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6472024-11-14T00:30:23.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6482024-11-14T00:30:23.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6492024-11-14T00:30:23.649ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 509.002873ms iteration = 24 status = {}
6502024-11-14T00:30:23.673ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 8ef5dd5b-824d-4c9e-8a43-11d66bcf933d roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6512024-11-14T00:30:23.673ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 8ef5dd5b-824d-4c9e-8a43-11d66bcf933d resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6522024-11-14T00:30:23.675ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2272 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 8ef5dd5b-824d-4c9e-8a43-11d66bcf933d response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6532024-11-14T00:30:24.701ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 307c4d21-000a-4879-acb5-5b1ef3c4320d roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6542024-11-14T00:30:24.702ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 307c4d21-000a-4879-acb5-5b1ef3c4320d resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6552024-11-14T00:30:24.703ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1886 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 307c4d21-000a-4879-acb5-5b1ef3c4320d response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6562024-11-14T00:30:25.724ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = d5cb9ae1-b27d-4648-8455-e9f4ae18ebe2 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6572024-11-14T00:30:25.725ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = d5cb9ae1-b27d-4648-8455-e9f4ae18ebe2 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6582024-11-14T00:30:25.726ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2192 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = d5cb9ae1-b27d-4648-8455-e9f4ae18ebe2 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6592024-11-14T00:30:26.746ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = a85c9520-e57a-46db-af29-a2031f8c7e5b roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6602024-11-14T00:30:26.746ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = a85c9520-e57a-46db-af29-a2031f8c7e5b resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6612024-11-14T00:30:26.748ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2469 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = a85c9520-e57a-46db-af29-a2031f8c7e5b response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6622024-11-14T00:30:27.769ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 33702b3a-3694-44bb-8c9f-03fd3e58985c roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6632024-11-14T00:30:27.770ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 33702b3a-3694-44bb-8c9f-03fd3e58985c resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6642024-11-14T00:30:27.772ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3006 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 33702b3a-3694-44bb-8c9f-03fd3e58985c response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6652024-11-14T00:30:28.795ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 61939932-28e2-4b5a-b8f0-edaad771fdd9 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6662024-11-14T00:30:28.795ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 61939932-28e2-4b5a-b8f0-edaad771fdd9 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6672024-11-14T00:30:28.797ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2574 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 61939932-28e2-4b5a-b8f0-edaad771fdd9 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6682024-11-14T00:30:29.818ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e46441ad-8f23-484a-9148-f2dcc07a64e9 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6692024-11-14T00:30:29.819ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e46441ad-8f23-484a-9148-f2dcc07a64e9 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6702024-11-14T00:30:29.821ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2354 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e46441ad-8f23-484a-9148-f2dcc07a64e9 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6712024-11-14T00:30:30.842ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ea07f31e-c0a3-41a8-a5a4-eeb754dd8244 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6722024-11-14T00:30:30.843ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ea07f31e-c0a3-41a8-a5a4-eeb754dd8244 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6732024-11-14T00:30:30.845ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2573 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ea07f31e-c0a3-41a8-a5a4-eeb754dd8244 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6742024-11-14T00:30:30.949ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 624 local_addr = [fd00:1122:3344:101::c]:37372 method = GET remote_addr = [fd00:1122:3344:101::d]:55706 req_id = 0c8b9505-e322-4a6b-a627-d1f81b53ebc5 response_code = 200 uri = /d06da591-b39e-4924-be5a-46bd48b9952b
6752024-11-14T00:30:31.867ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 055cd12e-3fb9-44de-b90b-395c7edd13ce roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6762024-11-14T00:30:31.867ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 055cd12e-3fb9-44de-b90b-395c7edd13ce resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6772024-11-14T00:30:31.870ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2862 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 055cd12e-3fb9-44de-b90b-395c7edd13ce response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6782024-11-14T00:30:32.889ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 5f264a1d-4838-4d20-8c92-ee265f12ac11 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6792024-11-14T00:30:32.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 5f264a1d-4838-4d20-8c92-ee265f12ac11 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6802024-11-14T00:30:32.892ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2574 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 5f264a1d-4838-4d20-8c92-ee265f12ac11 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
6812024-11-14T00:30:33.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
6822024-11-14T00:30:33.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6832024-11-14T00:30:33.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6842024-11-14T00:30:33.140ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
6852024-11-14T00:30:33.140ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6862024-11-14T00:30:33.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6872024-11-14T00:30:33.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6882024-11-14T00:30:33.142ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6892024-11-14T00:30:33.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6902024-11-14T00:30:33.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6912024-11-14T00:30:33.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6922024-11-14T00:30:33.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6932024-11-14T00:30:33.144ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.392852ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6942024-11-14T00:30:33.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6952024-11-14T00:30:33.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6962024-11-14T00:30:33.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6972024-11-14T00:30:33.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6178f930-5fa8-46a0-8fe5-b460818f9d6f, lookup_type: ById(6178f930-5fa8-46a0-8fe5-b460818f9d6f) } result = Ok(())
6982024-11-14T00:30:33.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6992024-11-14T00:30:33.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7002024-11-14T00:30:33.162ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f original_time_created = 2024-11-14 00:18:47.526379 UTC target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f time_created = 2024-11-14 00:18:47.526379 UTC
7012024-11-14T00:30:33.162ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.273368ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"6178f930-5fa8-46a0-8fe5-b460818f9d6f","time_created":"2024-11-14T00:18:47.526379Z"}
7022024-11-14T00:30:33.913ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = a2a10b3d-1c37-49e9-98bb-7fb4611e6551 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7032024-11-14T00:30:33.913ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = a2a10b3d-1c37-49e9-98bb-7fb4611e6551 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7042024-11-14T00:30:33.914ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1837 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = a2a10b3d-1c37-49e9-98bb-7fb4611e6551 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7052024-11-14T00:30:34.936ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = f3a144b4-1d84-40f4-9063-6756290e90cd roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7062024-11-14T00:30:34.936ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = f3a144b4-1d84-40f4-9063-6756290e90cd resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7072024-11-14T00:30:34.938ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2674 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = f3a144b4-1d84-40f4-9063-6756290e90cd response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7082024-11-14T00:30:35.937ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = 4798f21d-e96d-4096-b61d-53919ecf399b roles = RoleSet { roles: {} } uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
7092024-11-14T00:30:35.938ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = 4798f21d-e96d-4096-b61d-53919ecf399b resource = Database result = Ok(()) uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
7102024-11-14T00:30:35.940ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2875 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = 4798f21d-e96d-4096-b61d-53919ecf399b response_code = 200 uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
7112024-11-14T00:30:35.940ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = e7109da2-6caa-4f16-a86d-4fa48544cda2 roles = RoleSet { roles: {} } uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
7122024-11-14T00:30:35.941ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = e7109da2-6caa-4f16-a86d-4fa48544cda2 resource = Database result = Ok(()) uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
7132024-11-14T00:30:35.942ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2066 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = e7109da2-6caa-4f16-a86d-4fa48544cda2 response_code = 200 uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
7142024-11-14T00:30:35.957ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 0d2e999d-8880-41cf-9711-70c40f9cf987 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7152024-11-14T00:30:35.957ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 0d2e999d-8880-41cf-9711-70c40f9cf987 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7162024-11-14T00:30:35.959ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2306 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 0d2e999d-8880-41cf-9711-70c40f9cf987 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7172024-11-14T00:30:36.981ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = b0c41d9f-4043-4a0c-a078-63cc0f511152 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7182024-11-14T00:30:36.981ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = b0c41d9f-4043-4a0c-a078-63cc0f511152 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7192024-11-14T00:30:36.983ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1890 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = b0c41d9f-4043-4a0c-a078-63cc0f511152 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7202024-11-14T00:30:38.005ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ff9113b0-ccc2-423d-b5ab-9003d75984af roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7212024-11-14T00:30:38.005ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ff9113b0-ccc2-423d-b5ab-9003d75984af resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7222024-11-14T00:30:38.007ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2262 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ff9113b0-ccc2-423d-b5ab-9003d75984af response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7232024-11-14T00:30:39.029ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 2e6f8cba-3ee5-4988-854a-a412b734ea80 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7242024-11-14T00:30:39.030ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 2e6f8cba-3ee5-4988-854a-a412b734ea80 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7252024-11-14T00:30:39.032ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2642 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 2e6f8cba-3ee5-4988-854a-a412b734ea80 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7262024-11-14T00:30:40.051ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = abba0641-f037-4b00-89b5-4a65c957846c roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7272024-11-14T00:30:40.052ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = abba0641-f037-4b00-89b5-4a65c957846c resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7282024-11-14T00:30:40.054ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2311 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = abba0641-f037-4b00-89b5-4a65c957846c response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7292024-11-14T00:30:41.074ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 0eaaa93c-7f5c-41c7-bfc3-0916c2295489 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7302024-11-14T00:30:41.075ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 0eaaa93c-7f5c-41c7-bfc3-0916c2295489 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7312024-11-14T00:30:41.077ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2709 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 0eaaa93c-7f5c-41c7-bfc3-0916c2295489 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7322024-11-14T00:30:42.098ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = af837e06-75d6-48b9-8ec1-c1d4423a7307 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7332024-11-14T00:30:42.098ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = af837e06-75d6-48b9-8ec1-c1d4423a7307 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7342024-11-14T00:30:42.100ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2046 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = af837e06-75d6-48b9-8ec1-c1d4423a7307 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7352024-11-14T00:30:43.122ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = dcd36469-20bc-4a2f-a7ee-b1b406717d73 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7362024-11-14T00:30:43.123ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = dcd36469-20bc-4a2f-a7ee-b1b406717d73 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7372024-11-14T00:30:43.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2681 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = dcd36469-20bc-4a2f-a7ee-b1b406717d73 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7382024-11-14T00:30:43.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
7392024-11-14T00:30:43.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7402024-11-14T00:30:43.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7412024-11-14T00:30:43.140ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
7422024-11-14T00:30:43.140ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7432024-11-14T00:30:43.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7442024-11-14T00:30:43.141ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7452024-11-14T00:30:43.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7462024-11-14T00:30:43.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7472024-11-14T00:30:43.143ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.190385ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7482024-11-14T00:30:43.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7492024-11-14T00:30:43.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7502024-11-14T00:30:43.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7512024-11-14T00:30:43.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7522024-11-14T00:30:43.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7532024-11-14T00:30:43.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7542024-11-14T00:30:43.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6178f930-5fa8-46a0-8fe5-b460818f9d6f, lookup_type: ById(6178f930-5fa8-46a0-8fe5-b460818f9d6f) } result = Ok(())
7552024-11-14T00:30:43.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7562024-11-14T00:30:43.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7572024-11-14T00:30:43.159ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f original_time_created = 2024-11-14 00:18:47.526379 UTC target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f time_created = 2024-11-14 00:18:47.526379 UTC
7582024-11-14T00:30:43.159ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.492928ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"6178f930-5fa8-46a0-8fe5-b460818f9d6f","time_created":"2024-11-14T00:18:47.526379Z"}
7592024-11-14T00:30:44.147ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ad834cbf-da84-47de-9f47-5275de0f162b roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7602024-11-14T00:30:44.147ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ad834cbf-da84-47de-9f47-5275de0f162b resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7612024-11-14T00:30:44.149ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2638 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = ad834cbf-da84-47de-9f47-5275de0f162b response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7622024-11-14T00:30:45.170ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 2f04edf3-20b9-4b2f-9c47-38998f7af037 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7632024-11-14T00:30:45.171ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 2f04edf3-20b9-4b2f-9c47-38998f7af037 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7642024-11-14T00:30:45.173ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2518 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 2f04edf3-20b9-4b2f-9c47-38998f7af037 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7652024-11-14T00:30:45.944ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 562 local_addr = [fd00:1122:3344:101::c]:37372 method = GET remote_addr = [fd00:1122:3344:101::d]:55706 req_id = 36b24cd3-0125-4e9d-8de3-0cf8334d6181 response_code = 200 uri = /d06da591-b39e-4924-be5a-46bd48b9952b
7662024-11-14T00:30:46.192ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e31521ba-d8e5-4fb1-9aeb-0b2dc7ae1be3 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7672024-11-14T00:30:46.193ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e31521ba-d8e5-4fb1-9aeb-0b2dc7ae1be3 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7682024-11-14T00:30:46.194ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2224 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e31521ba-d8e5-4fb1-9aeb-0b2dc7ae1be3 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7692024-11-14T00:30:47.215ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6cb817d5-950c-4bd9-a6c2-c20727b028c0 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7702024-11-14T00:30:47.216ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6cb817d5-950c-4bd9-a6c2-c20727b028c0 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7712024-11-14T00:30:47.218ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2518 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6cb817d5-950c-4bd9-a6c2-c20727b028c0 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7722024-11-14T00:30:48.240ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = d11305b8-d41f-47d4-8929-0655aacf73d1 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7732024-11-14T00:30:48.241ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = d11305b8-d41f-47d4-8929-0655aacf73d1 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7742024-11-14T00:30:48.243ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2479 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = d11305b8-d41f-47d4-8929-0655aacf73d1 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7752024-11-14T00:30:49.264ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e2c0152f-4153-42bb-9d8d-e0c68e017840 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7762024-11-14T00:30:49.264ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e2c0152f-4153-42bb-9d8d-e0c68e017840 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7772024-11-14T00:30:49.266ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2386 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e2c0152f-4153-42bb-9d8d-e0c68e017840 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7782024-11-14T00:30:49.352ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = 5fd4ff52-52db-4ab3-9a3c-ce2621654806 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
7792024-11-14T00:30:49.352ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = 5fd4ff52-52db-4ab3-9a3c-ce2621654806 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
7802024-11-14T00:30:49.355ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3703 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = 5fd4ff52-52db-4ab3-9a3c-ce2621654806 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
7812024-11-14T00:30:50.286ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 39d45898-a838-4e3b-9acf-353c0d97691b roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7822024-11-14T00:30:50.287ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 39d45898-a838-4e3b-9acf-353c0d97691b resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7832024-11-14T00:30:50.289ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2306 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 39d45898-a838-4e3b-9acf-353c0d97691b response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7842024-11-14T00:30:50.938ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = f7384a48-d8bd-4daa-b5ee-bfd98e00727f roles = RoleSet { roles: {} } uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
7852024-11-14T00:30:50.938ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = f7384a48-d8bd-4daa-b5ee-bfd98e00727f resource = Database result = Ok(()) uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
7862024-11-14T00:30:50.941ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3096 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = f7384a48-d8bd-4daa-b5ee-bfd98e00727f response_code = 200 uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
7872024-11-14T00:30:50.941ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = e045f061-50d6-4719-b7f9-3231a248b072 roles = RoleSet { roles: {} } uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
7882024-11-14T00:30:50.942ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = e045f061-50d6-4719-b7f9-3231a248b072 resource = Database result = Ok(()) uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
7892024-11-14T00:30:50.944ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2858 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = e045f061-50d6-4719-b7f9-3231a248b072 response_code = 200 uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
7902024-11-14T00:30:51.309ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 7a1cbb69-4de3-4ea4-b290-b47627fa8d19 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7912024-11-14T00:30:51.310ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 7a1cbb69-4de3-4ea4-b290-b47627fa8d19 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7922024-11-14T00:30:51.311ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2013 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 7a1cbb69-4de3-4ea4-b290-b47627fa8d19 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7932024-11-14T00:30:52.332ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6a557129-dc57-4a10-9e2c-03183c2423a2 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7942024-11-14T00:30:52.333ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6a557129-dc57-4a10-9e2c-03183c2423a2 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7952024-11-14T00:30:52.335ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2785 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6a557129-dc57-4a10-9e2c-03183c2423a2 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
7962024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
7972024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
7982024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7992024-11-14T00:30:53.139ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f
8002024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
8012024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 22.401\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"6178f930-5fa8-46a0-8fe5-b460818f9d6f"}
8022024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
8032024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
8042024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
8052024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8062024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
8072024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
8082024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
8092024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
8102024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
8112024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8122024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8132024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
8142024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
8152024-11-14T00:30:53.139ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
8162024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
8172024-11-14T00:30:53.140ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-14 00:20:53.139872826 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
8182024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
8192024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
8202024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8212024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8222024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8232024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
8242024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
8252024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
8262024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
8272024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
8282024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8292024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
8302024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
8312024-11-14T00:30:53.140ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
8322024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8332024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
8342024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
8352024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
8362024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8372024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
8382024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
8392024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8402024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
8412024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
8422024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8432024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8442024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8452024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
8462024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
8472024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
8482024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
8492024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8502024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
8512024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
8522024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
8532024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8542024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
8552024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
8562024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
8572024-11-14T00:30:53.140ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
8582024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
8592024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
8602024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
8612024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8622024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
8632024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
8642024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
8652024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
8662024-11-14T00:30:53.140ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8672024-11-14T00:30:53.140ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ab1c44d-7d9e-4861-bc1e-704f908d5ccd.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("34921995-b6f6-4aad-8964-3692385af9d6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("af9e2ff3-0008-44dc-81fb-c799d2026305.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("af9e2ff3-0008-44dc-81fb-c799d2026305.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1752, tv_nsec: 857349773 } })
8682024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
8692024-11-14T00:30:53.141ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6880e62c-aa82-46ed-8fbf-fcd0d42c3184.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("df51b222-8234-4074-91c0-cdaaebfaa9a4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("df51b222-8234-4074-91c0-cdaaebfaa9a4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1752, tv_nsec: 857389944 } })
8702024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
8712024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
8722024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
8732024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8742024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
8752024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
8762024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8772024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
8782024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8792024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
8802024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8812024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8822024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
8832024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8842024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8852024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8862024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
8872024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
8882024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
8892024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
8902024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
8912024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.643094ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
8922024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
8932024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.623483ms iteration = 13 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
8942024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
8952024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
8962024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
8972024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
8982024-11-14T00:30:53.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
8992024-11-14T00:30:53.142ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9002024-11-14T00:30:53.142ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9012024-11-14T00:30:53.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9022024-11-14T00:30:53.143ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9032024-11-14T00:30:53.143ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1752, tv_nsec: 859910720 } })
9042024-11-14T00:30:53.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9052024-11-14T00:30:53.143ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1752, tv_nsec: 860056802 } })
9062024-11-14T00:30:53.143ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
9072024-11-14T00:30:53.143ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
9082024-11-14T00:30:53.144ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.312641ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
9092024-11-14T00:30:53.144ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
9102024-11-14T00:30:53.144ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9112024-11-14T00:30:53.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9122024-11-14T00:30:53.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9132024-11-14T00:30:53.146ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-14 00:20:53.139872826 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
9142024-11-14T00:30:53.146ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 7.09125ms iteration = 13 status = {"expiration":"2024-11-14T00:20:53.139872826Z","pruned":[]}
9152024-11-14T00:30:53.147ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9162024-11-14T00:30:53.147ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 7.1268ms iteration = 25 status = {"errors":[],"records_set_to_done":[]}
9172024-11-14T00:30:53.148ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
9182024-11-14T00:30:53.148ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
9192024-11-14T00:30:53.148ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 8.123995ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
9202024-11-14T00:30:53.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9212024-11-14T00:30:53.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9222024-11-14T00:30:53.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9232024-11-14T00:30:53.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9242024-11-14T00:30:53.149ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9252024-11-14T00:30:53.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9262024-11-14T00:30:53.151ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 10.996535ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
9272024-11-14T00:30:53.151ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9282024-11-14T00:30:53.152ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9292024-11-14T00:30:53.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9302024-11-14T00:30:53.153ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9312024-11-14T00:30:53.153ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
9322024-11-14T00:30:53.153ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9332024-11-14T00:30:53.153ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9342024-11-14T00:30:53.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9352024-11-14T00:30:53.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9362024-11-14T00:30:53.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
9372024-11-14T00:30:53.154ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9382024-11-14T00:30:53.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9392024-11-14T00:30:53.155ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 15.222984ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
9402024-11-14T00:30:53.156ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9412024-11-14T00:30:53.157ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9422024-11-14T00:30:53.157ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9432024-11-14T00:30:53.157ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9442024-11-14T00:30:53.158ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9452024-11-14T00:30:53.160ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9462024-11-14T00:30:53.160ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9472024-11-14T00:30:53.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
9482024-11-14T00:30:53.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9492024-11-14T00:30:53.160ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9502024-11-14T00:30:53.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9512024-11-14T00:30:53.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
9522024-11-14T00:30:53.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
9532024-11-14T00:30:53.161ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
9542024-11-14T00:30:53.161ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9552024-11-14T00:30:53.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
9562024-11-14T00:30:53.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9572024-11-14T00:30:53.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9582024-11-14T00:30:53.161ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9592024-11-14T00:30:53.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9602024-11-14T00:30:53.163ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9612024-11-14T00:30:53.164ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9622024-11-14T00:30:53.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9632024-11-14T00:30:53.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9642024-11-14T00:30:53.167ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9652024-11-14T00:30:53.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9662024-11-14T00:30:53.167ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9672024-11-14T00:30:53.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9682024-11-14T00:30:53.169ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9692024-11-14T00:30:53.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9702024-11-14T00:30:53.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9712024-11-14T00:30:53.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9722024-11-14T00:30:53.170ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9732024-11-14T00:30:53.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9742024-11-14T00:30:53.170ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9752024-11-14T00:30:53.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9762024-11-14T00:30:53.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9772024-11-14T00:30:53.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9782024-11-14T00:30:53.173ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9792024-11-14T00:30:53.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9802024-11-14T00:30:53.175ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.84257ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9812024-11-14T00:30:53.181ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9822024-11-14T00:30:53.181ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
9832024-11-14T00:30:53.181ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9842024-11-14T00:30:53.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9852024-11-14T00:30:53.182ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1752, tv_nsec: 899428406 } })
9862024-11-14T00:30:53.183ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9872024-11-14T00:30:53.183ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864
9882024-11-14T00:30:53.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9892024-11-14T00:30:53.184ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
9902024-11-14T00:30:53.184ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
9912024-11-14T00:30:53.184ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 43.623863ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
9922024-11-14T00:30:53.184ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9932024-11-14T00:30:53.186ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9942024-11-14T00:30:53.186ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 46.024107ms iteration = 6 status = {"nsuccess":5}
9952024-11-14T00:30:53.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9962024-11-14T00:30:53.187ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9972024-11-14T00:30:53.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9982024-11-14T00:30:53.188ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
9992024-11-14T00:30:53.188ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10002024-11-14T00:30:53.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10012024-11-14T00:30:53.191ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10022024-11-14T00:30:53.191ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10032024-11-14T00:30:53.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10042024-11-14T00:30:53.195ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10052024-11-14T00:30:53.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10062024-11-14T00:30:53.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6178f930-5fa8-46a0-8fe5-b460818f9d6f, lookup_type: ById(6178f930-5fa8-46a0-8fe5-b460818f9d6f) } result = Ok(())
10072024-11-14T00:30:53.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10082024-11-14T00:30:53.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10092024-11-14T00:30:53.200ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 59.571248ms iteration = 25 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
10102024-11-14T00:30:53.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10112024-11-14T00:30:53.205ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10122024-11-14T00:30:53.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10132024-11-14T00:30:53.206ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10142024-11-14T00:30:53.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10152024-11-14T00:30:53.208ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10162024-11-14T00:30:53.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
10172024-11-14T00:30:53.209ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10182024-11-14T00:30:53.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10192024-11-14T00:30:53.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10202024-11-14T00:30:53.209ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f original_time_created = 2024-11-14 00:18:47.526379 UTC target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f time_created = 2024-11-14 00:18:47.526379 UTC
10212024-11-14T00:30:53.209ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 69.956125ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"6178f930-5fa8-46a0-8fe5-b460818f9d6f","time_created":"2024-11-14T00:18:47.526379Z"}
10222024-11-14T00:30:53.211ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 70.32541ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
10232024-11-14T00:30:53.224ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
10242024-11-14T00:30:53.224ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
10252024-11-14T00:30:53.224ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
10262024-11-14T00:30:53.224ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10272024-11-14T00:30:53.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10282024-11-14T00:30:53.227ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10292024-11-14T00:30:53.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10302024-11-14T00:30:53.227ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10312024-11-14T00:30:53.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10322024-11-14T00:30:53.228ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10332024-11-14T00:30:53.229ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10342024-11-14T00:30:53.233ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10352024-11-14T00:30:53.233ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10362024-11-14T00:30:53.233ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
10372024-11-14T00:30:53.233ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10382024-11-14T00:30:53.233ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10392024-11-14T00:30:53.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10402024-11-14T00:30:53.233ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10412024-11-14T00:30:53.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10422024-11-14T00:30:53.233ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
10432024-11-14T00:30:53.233ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
10442024-11-14T00:30:53.233ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 93.279083ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
10452024-11-14T00:30:53.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10462024-11-14T00:30:53.237ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10472024-11-14T00:30:53.238ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10482024-11-14T00:30:53.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10492024-11-14T00:30:53.239ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10502024-11-14T00:30:53.240ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10512024-11-14T00:30:53.241ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
10522024-11-14T00:30:53.241ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10532024-11-14T00:30:53.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10542024-11-14T00:30:53.243ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 102.768685ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
10552024-11-14T00:30:53.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: e211d163-be78-4663-b73f-baf5e9462a10, lookup_type: ById(e211d163-be78-4663-b73f-baf5e9462a10) } result = Ok(())
10562024-11-14T00:30:53.243ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10572024-11-14T00:30:53.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10582024-11-14T00:30:53.244ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10592024-11-14T00:30:53.244ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
10602024-11-14T00:30:53.244ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
10612024-11-14T00:30:53.244ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
10622024-11-14T00:30:53.244ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10632024-11-14T00:30:53.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10642024-11-14T00:30:53.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10652024-11-14T00:30:53.246ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 105.590215ms iteration = 25 status = 0
10662024-11-14T00:30:53.246ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10672024-11-14T00:30:53.246ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10682024-11-14T00:30:53.248ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
10692024-11-14T00:30:53.248ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10702024-11-14T00:30:53.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10712024-11-14T00:30:53.250ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10722024-11-14T00:30:53.250ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10732024-11-14T00:30:53.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10742024-11-14T00:30:53.252ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
10752024-11-14T00:30:53.252ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 2249d3ef-e161-463e-bd03-3f37771eab48 dns_zone_name = oxide.test found_so_far = 0 version = 2
10762024-11-14T00:30:53.252ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10772024-11-14T00:30:53.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10782024-11-14T00:30:53.255ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 115.14531ms iteration = 28 status = {"physical_disks_added":0}
10792024-11-14T00:30:53.262ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
10802024-11-14T00:30:53.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
10812024-11-14T00:30:53.268ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
10822024-11-14T00:30:53.268ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 127.91669ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
10832024-11-14T00:30:53.277ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
10842024-11-14T00:30:53.277ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10852024-11-14T00:30:53.277ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5745509d-06a2-43c6-8a00-2c0b05968477", "content-length": "26", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} })
10862024-11-14T00:30:53.277ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10872024-11-14T00:30:53.279ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
10882024-11-14T00:30:53.279ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 501e9987-47e9-4194-a5b0-10ee954fd76c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
10892024-11-14T00:30:53.279ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10902024-11-14T00:30:53.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10912024-11-14T00:30:53.279ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
10922024-11-14T00:30:53.281ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10932024-11-14T00:30:53.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10942024-11-14T00:30:53.284ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10952024-11-14T00:30:53.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10962024-11-14T00:30:53.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10972024-11-14T00:30:53.287ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 501e9987-47e9-4194-a5b0-10ee954fd76c dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
10982024-11-14T00:30:53.287ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
10992024-11-14T00:30:53.287ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-14 00:18:53.692551 UTC dns_group = internal generation = 1
11002024-11-14T00:30:53.287ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 147.82971ms iteration = 13 status = {"generation":1}
11012024-11-14T00:30:53.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11022024-11-14T00:30:53.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11032024-11-14T00:30:53.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11042024-11-14T00:30:53.293ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-14T00:18:50.724288Z, time_modified: 2024-11-14T00:18:50.724288Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bf9014ac-edb7-4293-a1ff-4004060e0581, db_silo: Silo { identity: SiloIdentity { id: bf9014ac-edb7-4293-a1ff-4004060e0581, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T00:18:55.151722Z, time_modified: 2024-11-14T00:18:55.151722Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "0798a7385384103ed54567d8406210304cdb091736af57f4492a8d16af3f2d52" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bf9014ac-edb7-4293-a1ff-4004060e0581, db_silo: Silo { identity: SiloIdentity { id: bf9014ac-edb7-4293-a1ff-4004060e0581, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-14T00:18:55.151722Z, time_modified: 2024-11-14T00:18:55.151722Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "0798a7385384103ed54567d8406210304cdb091736af57f4492a8d16af3f2d52" }] }) }
11052024-11-14T00:30:53.293ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 152.81444ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"bf9014ac-edb7-4293-a1ff-4004060e0581","tls_certs":["0798a7385384103ed54567d8406210304cdb091736af57f4492a8d16af3f2d52"]}},"default_endpoint":{"silo_id":"bf9014ac-edb7-4293-a1ff-4004060e0581","tls_certs":["0798a7385384103ed54567d8406210304cdb091736af57f4492a8d16af3f2d52"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
11062024-11-14T00:30:53.293ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 2249d3ef-e161-463e-bd03-3f37771eab48 dns_zone_name = oxide.test found_so_far = 1 version = 2
11072024-11-14T00:30:53.293ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
11082024-11-14T00:30:53.293ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-14 00:18:55.211515 UTC dns_group = external generation = 2
11092024-11-14T00:30:53.293ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 152.79659ms iteration = 13 status = {"generation":2}
11102024-11-14T00:30:53.320ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
11112024-11-14T00:30:53.320ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 179.80876ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
11122024-11-14T00:30:53.357ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = fd58ed31-03f7-4917-be8d-2d3a208b159b roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
11132024-11-14T00:30:53.357ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = fd58ed31-03f7-4917-be8d-2d3a208b159b resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
11142024-11-14T00:30:53.359ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2168 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = fd58ed31-03f7-4917-be8d-2d3a208b159b response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
11152024-11-14T00:30:53.459ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
11162024-11-14T00:30:53.460ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "7dd4415b-76ff-4e7e-b4b4-5e7062e20548", "content-length": "2", "date": "Thu, 14 Nov 2024 00:30:52 GMT"} })
11172024-11-14T00:30:53.462ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
11182024-11-14T00:30:53.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
11192024-11-14T00:30:53.465ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 325.320938ms iteration = 25 status = null
11202024-11-14T00:30:53.504ZINFOnexus (BackgroundTasks): VPC route manager sled e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
11212024-11-14T00:30:53.504ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11222024-11-14T00:30:53.505ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc66062b-6bdb-4ad6-ba9d-27289039a9c6", "content-length": "372", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} })
11232024-11-14T00:30:53.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11242024-11-14T00:30:53.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11252024-11-14T00:30:53.524ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
11262024-11-14T00:30:53.525ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c721637a-a47b-4583-86de-fa1460d004fa", "content-length": "691", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} })
11272024-11-14T00:30:53.525ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
11282024-11-14T00:30:53.525ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
11292024-11-14T00:30:53.525ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
11302024-11-14T00:30:53.526ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 385.895ms iteration = 26 status = {}
11312024-11-14T00:30:53.560ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
11322024-11-14T00:30:53.560ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
11332024-11-14T00:30:53.560ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f4e4579f-98d8-4f07-8730-704ddcfa4a5d", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} })
11342024-11-14T00:30:53.560ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11352024-11-14T00:30:53.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11362024-11-14T00:30:53.563ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11372024-11-14T00:30:53.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11382024-11-14T00:30:53.566ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11392024-11-14T00:30:53.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11402024-11-14T00:30:53.569ZINFOnexus (BackgroundTasks): VPC route manager sled e211d163-be78-4663-b73f-baf5e9462a10 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
11412024-11-14T00:30:53.569ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11422024-11-14T00:30:53.569ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7499d583-4c7e-48f2-9456-d6d6874ad898", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} })
11432024-11-14T00:30:53.570ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
11442024-11-14T00:30:53.572ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 431.490912ms iteration = 26 status = {}
11452024-11-14T00:30:53.606ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
11462024-11-14T00:30:53.606ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11472024-11-14T00:30:53.606ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8359fb5d-0228-40b9-871f-860436a6a718", "content-length": "26", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} })
11482024-11-14T00:30:53.606ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11492024-11-14T00:30:53.607ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11502024-11-14T00:30:53.624ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
11512024-11-14T00:30:53.710ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 zone_address = fd00:1122:3344:101::2
11522024-11-14T00:30:53.710ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11532024-11-14T00:30:53.710ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a431f8ab-c105-4a17-bf51-f31c44387ae4", "content-length": "26", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} })
11542024-11-14T00:30:53.710ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11552024-11-14T00:30:53.712ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864
11562024-11-14T00:30:53.739ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 599.911421ms iteration = 25 status = {}
11572024-11-14T00:30:53.837ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
11582024-11-14T00:30:53.838ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "2fcc0c53-b55d-41f5-ba32-bd8703d09364", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11592024-11-14T00:30:53.838ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11602024-11-14T00:30:53.840ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "3ffd0fa6-268d-47e0-ab50-c21fff5ee3b9", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11612024-11-14T00:30:53.840ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11622024-11-14T00:30:53.842ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11632024-11-14T00:30:53.842ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 702.431155ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
11642024-11-14T00:30:53.842ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
11652024-11-14T00:30:53.903ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
11662024-11-14T00:30:53.903ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "c8254f6e-f7cc-4f4e-bcf1-52858c1946fa", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
11672024-11-14T00:30:53.903ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
11682024-11-14T00:30:53.905ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "ae2c41f7-b2d5-4833-b119-8791c32c156a", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
11692024-11-14T00:30:53.905ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
11702024-11-14T00:30:53.906ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "84a90cb3-6394-458c-ada9-584cffe256b6", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
11712024-11-14T00:30:53.906ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
11722024-11-14T00:30:53.908ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
11732024-11-14T00:30:53.908ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 768.676137ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
11742024-11-14T00:30:53.922ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11752024-11-14T00:30:53.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11762024-11-14T00:30:53.925ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1b492b5b-8c3a-4e99-9528-5d52a91581c5, rack_id: fcfbb675-622a-4d07-9c7d-02f1c7317864, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d5fc9a38-ca99-44ad-95a3-d22ca6bdfc5b) } rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 switch_location = Switch0
11772024-11-14T00:30:53.925ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11782024-11-14T00:30:53.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11792024-11-14T00:30:53.935ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
11802024-11-14T00:30:53.935ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "75a162d2-43d5-47a9-ae34-c38edcfe922d", "content-length": "134", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} })
11812024-11-14T00:30:53.935ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
11822024-11-14T00:30:53.935ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d5fc9a38-ca99-44ad-95a3-d22ca6bdfc5b
11832024-11-14T00:30:53.935ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
11842024-11-14T00:30:53.935ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
11852024-11-14T00:30:53.935ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
11862024-11-14T00:30:53.935ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
11872024-11-14T00:30:53.935ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
11882024-11-14T00:30:53.935ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
11892024-11-14T00:30:53.936ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
11902024-11-14T00:30:53.936ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
11912024-11-14T00:30:53.936ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864
11922024-11-14T00:30:53.936ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11932024-11-14T00:30:53.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11942024-11-14T00:30:53.938ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
11952024-11-14T00:30:53.938ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "2124fdc7-fd00-472f-bcb8-455c8f7a10bc", "content-length": "2", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} })
11962024-11-14T00:30:53.938ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
11972024-11-14T00:30:53.938ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "399ade11-0773-4fe6-ac89-d36f00de7e67", "content-length": "52", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} })
11982024-11-14T00:30:53.938ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 switch_location = Switch0
11992024-11-14T00:30:53.938ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
12002024-11-14T00:30:54.090ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "33cf88d5-565e-43a0-a766-d11a96634271", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} })
12012024-11-14T00:30:54.090ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 switch = Switch0
12022024-11-14T00:30:54.090ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
12032024-11-14T00:30:54.090ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad6d4b49-1eb0-4a33-b2d1-cf58d4d12718", "content-length": "682", "date": "Thu, 14 Nov 2024 00:30:53 GMT"} })
12042024-11-14T00:30:54.090ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12052024-11-14T00:30:54.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12062024-11-14T00:30:54.094ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12072024-11-14T00:30:54.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12082024-11-14T00:30:54.097ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12092024-11-14T00:30:54.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12102024-11-14T00:30:54.100ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12112024-11-14T00:30:54.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12122024-11-14T00:30:54.108ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 967.969851ms iteration = 25 status = {}
12132024-11-14T00:30:54.380ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 1bf8eafa-5527-47dd-92a1-ddbfde42869e roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12142024-11-14T00:30:54.381ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 1bf8eafa-5527-47dd-92a1-ddbfde42869e resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12152024-11-14T00:30:54.383ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2650 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 1bf8eafa-5527-47dd-92a1-ddbfde42869e response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12162024-11-14T00:30:55.403ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 5f266c5e-ce23-417b-a352-83f198475057 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12172024-11-14T00:30:55.403ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 5f266c5e-ce23-417b-a352-83f198475057 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12182024-11-14T00:30:55.405ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2350 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 5f266c5e-ce23-417b-a352-83f198475057 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12192024-11-14T00:30:56.426ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 844b7e14-c356-4742-b29d-e2de052e7c86 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12202024-11-14T00:30:56.427ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 844b7e14-c356-4742-b29d-e2de052e7c86 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12212024-11-14T00:30:56.429ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2713 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 844b7e14-c356-4742-b29d-e2de052e7c86 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12222024-11-14T00:30:57.451ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = a4e3b42c-bd6a-4bb6-92a3-37bda5d9cd57 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12232024-11-14T00:30:57.451ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = a4e3b42c-bd6a-4bb6-92a3-37bda5d9cd57 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12242024-11-14T00:30:57.453ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2819 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = a4e3b42c-bd6a-4bb6-92a3-37bda5d9cd57 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12252024-11-14T00:30:58.478ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 07948083-9a44-460e-a7d8-66764b6fccb8 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12262024-11-14T00:30:58.479ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 07948083-9a44-460e-a7d8-66764b6fccb8 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12272024-11-14T00:30:58.481ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2283 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 07948083-9a44-460e-a7d8-66764b6fccb8 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12282024-11-14T00:30:59.503ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 4e5d39fc-8581-4933-a4c3-d1b1dd7479a6 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12292024-11-14T00:30:59.504ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 4e5d39fc-8581-4933-a4c3-d1b1dd7479a6 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12302024-11-14T00:30:59.505ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1963 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 4e5d39fc-8581-4933-a4c3-d1b1dd7479a6 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12312024-11-14T00:31:00.527ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = cca596d7-84dd-43be-8523-eddb762d2060 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12322024-11-14T00:31:00.527ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = cca596d7-84dd-43be-8523-eddb762d2060 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12332024-11-14T00:31:00.529ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2514 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = cca596d7-84dd-43be-8523-eddb762d2060 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12342024-11-14T00:31:00.947ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 607 local_addr = [fd00:1122:3344:101::c]:37372 method = GET remote_addr = [fd00:1122:3344:101::d]:55706 req_id = 3d882509-e52f-4a3c-8cbe-aa0c353546cd response_code = 200 uri = /d06da591-b39e-4924-be5a-46bd48b9952b
12352024-11-14T00:31:01.549ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 4709f575-ea65-45f7-9c72-50bbed498336 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12362024-11-14T00:31:01.550ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 4709f575-ea65-45f7-9c72-50bbed498336 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12372024-11-14T00:31:01.552ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2602 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 4709f575-ea65-45f7-9c72-50bbed498336 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12382024-11-14T00:31:02.574ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e95b3ab4-d61a-48e1-80f4-965a03043dc9 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12392024-11-14T00:31:02.574ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e95b3ab4-d61a-48e1-80f4-965a03043dc9 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12402024-11-14T00:31:02.576ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2376 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e95b3ab4-d61a-48e1-80f4-965a03043dc9 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12412024-11-14T00:31:03.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
12422024-11-14T00:31:03.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12432024-11-14T00:31:03.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12442024-11-14T00:31:03.140ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
12452024-11-14T00:31:03.140ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12462024-11-14T00:31:03.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12472024-11-14T00:31:03.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12482024-11-14T00:31:03.142ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12492024-11-14T00:31:03.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12502024-11-14T00:31:03.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12512024-11-14T00:31:03.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12522024-11-14T00:31:03.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12532024-11-14T00:31:03.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12542024-11-14T00:31:03.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12552024-11-14T00:31:03.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12562024-11-14T00:31:03.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6178f930-5fa8-46a0-8fe5-b460818f9d6f, lookup_type: ById(6178f930-5fa8-46a0-8fe5-b460818f9d6f) } result = Ok(())
12572024-11-14T00:31:03.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12582024-11-14T00:31:03.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12592024-11-14T00:31:03.161ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f original_time_created = 2024-11-14 00:18:47.526379 UTC target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f time_created = 2024-11-14 00:18:47.526379 UTC
12602024-11-14T00:31:03.161ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.174632ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"6178f930-5fa8-46a0-8fe5-b460818f9d6f","time_created":"2024-11-14T00:18:47.526379Z"}
12612024-11-14T00:31:03.171ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.747933ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
12622024-11-14T00:31:03.598ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9ab6ccdd-b00b-4dce-9e8b-c38e7869bc32 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12632024-11-14T00:31:03.599ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9ab6ccdd-b00b-4dce-9e8b-c38e7869bc32 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12642024-11-14T00:31:03.600ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2262 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9ab6ccdd-b00b-4dce-9e8b-c38e7869bc32 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12652024-11-14T00:31:04.621ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = b42f6214-082b-40ee-8696-74dc125b9b5d roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12662024-11-14T00:31:04.621ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = b42f6214-082b-40ee-8696-74dc125b9b5d resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12672024-11-14T00:31:04.623ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2233 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = b42f6214-082b-40ee-8696-74dc125b9b5d response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12682024-11-14T00:31:05.644ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a117e8e-e5a7-498e-a743-6f57a9f4938b roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12692024-11-14T00:31:05.644ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a117e8e-e5a7-498e-a743-6f57a9f4938b resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12702024-11-14T00:31:05.646ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2413 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 9a117e8e-e5a7-498e-a743-6f57a9f4938b response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12712024-11-14T00:31:05.938ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = c33acb86-2c34-4875-a521-76e4c5f8b27a roles = RoleSet { roles: {} } uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
12722024-11-14T00:31:05.938ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = c33acb86-2c34-4875-a521-76e4c5f8b27a resource = Database result = Ok(()) uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
12732024-11-14T00:31:05.941ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3018 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = c33acb86-2c34-4875-a521-76e4c5f8b27a response_code = 200 uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
12742024-11-14T00:31:05.941ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = 754e1434-f714-4790-8fd2-f098a5cc8976 roles = RoleSet { roles: {} } uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
12752024-11-14T00:31:05.941ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = 754e1434-f714-4790-8fd2-f098a5cc8976 resource = Database result = Ok(()) uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
12762024-11-14T00:31:05.944ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2608 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = 754e1434-f714-4790-8fd2-f098a5cc8976 response_code = 200 uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
12772024-11-14T00:31:06.666ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 35c6402c-c69f-45a3-9030-a4a1df2b47fb roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12782024-11-14T00:31:06.667ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 35c6402c-c69f-45a3-9030-a4a1df2b47fb resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12792024-11-14T00:31:06.668ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2188 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 35c6402c-c69f-45a3-9030-a4a1df2b47fb response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12802024-11-14T00:31:07.690ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = b32139e6-f61d-4dca-9945-df841020c081 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12812024-11-14T00:31:07.691ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = b32139e6-f61d-4dca-9945-df841020c081 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12822024-11-14T00:31:07.693ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2631 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = b32139e6-f61d-4dca-9945-df841020c081 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12832024-11-14T00:31:08.714ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 7c92e73a-8666-4fe3-89ee-44c9ef0f1139 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12842024-11-14T00:31:08.714ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 7c92e73a-8666-4fe3-89ee-44c9ef0f1139 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12852024-11-14T00:31:08.716ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1992 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 7c92e73a-8666-4fe3-89ee-44c9ef0f1139 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12862024-11-14T00:31:09.736ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 227394e6-0dac-460e-bea4-f0bc62c58775 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12872024-11-14T00:31:09.737ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 227394e6-0dac-460e-bea4-f0bc62c58775 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12882024-11-14T00:31:09.739ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2505 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 227394e6-0dac-460e-bea4-f0bc62c58775 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12892024-11-14T00:31:10.758ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = f8060f73-92cf-4e82-be05-fd250722b259 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12902024-11-14T00:31:10.759ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = f8060f73-92cf-4e82-be05-fd250722b259 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12912024-11-14T00:31:10.760ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1948 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = f8060f73-92cf-4e82-be05-fd250722b259 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12922024-11-14T00:31:11.783ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 75ef3cf3-71de-40fc-ac7a-3dd28dea82a0 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12932024-11-14T00:31:11.783ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 75ef3cf3-71de-40fc-ac7a-3dd28dea82a0 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12942024-11-14T00:31:11.785ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2330 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 75ef3cf3-71de-40fc-ac7a-3dd28dea82a0 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12952024-11-14T00:31:12.806ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = f728d66a-6793-45df-84b5-d081cbc50a8e roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12962024-11-14T00:31:12.807ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = f728d66a-6793-45df-84b5-d081cbc50a8e resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12972024-11-14T00:31:12.809ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2438 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = f728d66a-6793-45df-84b5-d081cbc50a8e response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
12982024-11-14T00:31:13.140ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
12992024-11-14T00:31:13.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13002024-11-14T00:31:13.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13012024-11-14T00:31:13.140ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
13022024-11-14T00:31:13.140ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13032024-11-14T00:31:13.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13042024-11-14T00:31:13.143ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13052024-11-14T00:31:13.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13062024-11-14T00:31:13.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13072024-11-14T00:31:13.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13082024-11-14T00:31:13.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13092024-11-14T00:31:13.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13102024-11-14T00:31:13.172ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.726456ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
13112024-11-14T00:31:13.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13122024-11-14T00:31:13.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13132024-11-14T00:31:13.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13142024-11-14T00:31:13.177ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6178f930-5fa8-46a0-8fe5-b460818f9d6f, lookup_type: ById(6178f930-5fa8-46a0-8fe5-b460818f9d6f) } result = Ok(())
13152024-11-14T00:31:13.177ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13162024-11-14T00:31:13.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13172024-11-14T00:31:13.188ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f original_time_created = 2024-11-14 00:18:47.526379 UTC target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f time_created = 2024-11-14 00:18:47.526379 UTC
13182024-11-14T00:31:13.188ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 48.33722ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"6178f930-5fa8-46a0-8fe5-b460818f9d6f","time_created":"2024-11-14T00:18:47.526379Z"}
13192024-11-14T00:31:13.830ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = a3ccc415-f68a-48ee-8bec-2acf66c77754 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13202024-11-14T00:31:13.830ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = a3ccc415-f68a-48ee-8bec-2acf66c77754 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13212024-11-14T00:31:13.831ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1763 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = a3ccc415-f68a-48ee-8bec-2acf66c77754 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13222024-11-14T00:31:14.853ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 961f2bec-9085-43c6-9793-1cabd82063d4 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13232024-11-14T00:31:14.853ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 961f2bec-9085-43c6-9793-1cabd82063d4 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13242024-11-14T00:31:14.855ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2119 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 961f2bec-9085-43c6-9793-1cabd82063d4 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13252024-11-14T00:31:15.876ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = d4b66fec-0c01-4cda-b7dd-3c81121377e1 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13262024-11-14T00:31:15.877ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = d4b66fec-0c01-4cda-b7dd-3c81121377e1 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13272024-11-14T00:31:15.878ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2150 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = d4b66fec-0c01-4cda-b7dd-3c81121377e1 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13282024-11-14T00:31:15.946ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 664 local_addr = [fd00:1122:3344:101::c]:37372 method = GET remote_addr = [fd00:1122:3344:101::d]:55706 req_id = 4434e07f-4076-497d-90ba-c8eaae7f4047 response_code = 200 uri = /d06da591-b39e-4924-be5a-46bd48b9952b
13292024-11-14T00:31:16.901ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 83b874f6-9605-482c-a9af-d101b98323dc roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13302024-11-14T00:31:16.901ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 83b874f6-9605-482c-a9af-d101b98323dc resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13312024-11-14T00:31:16.903ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2480 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 83b874f6-9605-482c-a9af-d101b98323dc response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13322024-11-14T00:31:17.924ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 608fe428-6130-4ef0-91ea-14f96ea06c16 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13332024-11-14T00:31:17.924ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 608fe428-6130-4ef0-91ea-14f96ea06c16 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13342024-11-14T00:31:17.926ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2618 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 608fe428-6130-4ef0-91ea-14f96ea06c16 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13352024-11-14T00:31:18.948ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = f2d6f0d5-f903-4ba9-ace0-14cdc137455b roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13362024-11-14T00:31:18.948ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = f2d6f0d5-f903-4ba9-ace0-14cdc137455b resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13372024-11-14T00:31:18.950ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2204 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = f2d6f0d5-f903-4ba9-ace0-14cdc137455b response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13382024-11-14T00:31:19.389ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = 6588c7f8-b527-42f1-9ee3-0b73faa67333 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
13392024-11-14T00:31:19.389ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = 6588c7f8-b527-42f1-9ee3-0b73faa67333 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
13402024-11-14T00:31:19.392ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3705 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:53990 req_id = 6588c7f8-b527-42f1-9ee3-0b73faa67333 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
13412024-11-14T00:31:19.972ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 11c67e0e-e026-40b9-9369-0fabe2075c40 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13422024-11-14T00:31:19.973ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 11c67e0e-e026-40b9-9369-0fabe2075c40 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13432024-11-14T00:31:19.975ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2584 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 11c67e0e-e026-40b9-9369-0fabe2075c40 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13442024-11-14T00:31:20.939ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = e762e04c-2fdf-40da-9117-f6941f943378 roles = RoleSet { roles: {} } uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
13452024-11-14T00:31:20.939ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = e762e04c-2fdf-40da-9117-f6941f943378 resource = Database result = Ok(()) uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
13462024-11-14T00:31:20.945ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 6563 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = e762e04c-2fdf-40da-9117-f6941f943378 response_code = 200 uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?sort_by=id_ascending
13472024-11-14T00:31:20.945ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = 17df8531-8d03-4895-ae06-fd4129b745d6 roles = RoleSet { roles: {} } uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
13482024-11-14T00:31:20.946ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = 17df8531-8d03-4895-ae06-fd4129b745d6 resource = Database result = Ok(()) uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
13492024-11-14T00:31:20.948ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3100 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58473 req_id = 17df8531-8d03-4895-ae06-fd4129b745d6 response_code = 200 uri = /metrics/collectors/ad042fde-503a-4314-8c8c-afa2f783d145/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTRiNmQ3YS03ZmM4LTQxMTAtOTdjYS1kNTE5ZTBiYWQ3OWYifX0%3D
13502024-11-14T00:31:20.997ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 1650455c-f736-444b-922b-6b0ba1239e1b roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13512024-11-14T00:31:20.997ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 1650455c-f736-444b-922b-6b0ba1239e1b resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13522024-11-14T00:31:21.000ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2576 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 1650455c-f736-444b-922b-6b0ba1239e1b response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13532024-11-14T00:31:22.023ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 0230d88d-9c51-4d67-822c-7eef36657a85 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13542024-11-14T00:31:22.023ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 0230d88d-9c51-4d67-822c-7eef36657a85 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13552024-11-14T00:31:22.025ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1806 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 0230d88d-9c51-4d67-822c-7eef36657a85 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13562024-11-14T00:31:23.047ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 441c9857-b410-41d2-9fab-fb0f2055a4dc roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13572024-11-14T00:31:23.047ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 441c9857-b410-41d2-9fab-fb0f2055a4dc resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13582024-11-14T00:31:23.048ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2042 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 441c9857-b410-41d2-9fab-fb0f2055a4dc response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
13592024-11-14T00:31:23.140ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
13602024-11-14T00:31:23.140ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
13612024-11-14T00:31:23.140ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
13622024-11-14T00:31:23.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13632024-11-14T00:31:23.140ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
13642024-11-14T00:31:23.140ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13652024-11-14T00:31:23.140ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
13662024-11-14T00:31:23.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
13672024-11-14T00:31:23.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13682024-11-14T00:31:23.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13692024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
13702024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
13712024-11-14T00:31:23.141ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
13722024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
13732024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13742024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
13752024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13762024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
13772024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
13782024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
13792024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
13802024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
13812024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
13822024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
13832024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
13842024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
13852024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
13862024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
13872024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
13882024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13892024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
13902024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13912024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
13922024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
13932024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13942024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13952024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
13962024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13972024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13982024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
13992024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
14002024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14012024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
14022024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
14032024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
14042024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
14052024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14062024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14072024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
14082024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14092024-11-14T00:31:23.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14102024-11-14T00:31:23.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14112024-11-14T00:31:23.142ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1782, tv_nsec: 859019587 } })
14122024-11-14T00:31:23.143ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
14132024-11-14T00:31:23.143ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14142024-11-14T00:31:23.143ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1782, tv_nsec: 859977581 } })
14152024-11-14T00:31:23.144ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.916665ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
14162024-11-14T00:31:23.144ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
14172024-11-14T00:31:23.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14182024-11-14T00:31:23.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14192024-11-14T00:31:23.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14202024-11-14T00:31:23.145ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
14212024-11-14T00:31:23.145ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14222024-11-14T00:31:23.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14232024-11-14T00:31:23.147ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14242024-11-14T00:31:23.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14252024-11-14T00:31:23.148ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14262024-11-14T00:31:23.148ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14272024-11-14T00:31:23.149ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14282024-11-14T00:31:23.150ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
14292024-11-14T00:31:23.150ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
14302024-11-14T00:31:23.150ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
14312024-11-14T00:31:23.150ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
14322024-11-14T00:31:23.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
14332024-11-14T00:31:23.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
14342024-11-14T00:31:23.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
14352024-11-14T00:31:23.151ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14362024-11-14T00:31:23.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
14372024-11-14T00:31:23.151ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
14382024-11-14T00:31:23.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14392024-11-14T00:31:23.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
14402024-11-14T00:31:23.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
14412024-11-14T00:31:23.152ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14422024-11-14T00:31:23.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14432024-11-14T00:31:23.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14442024-11-14T00:31:23.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14452024-11-14T00:31:23.155ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 14.175ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14462024-11-14T00:31:23.157ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14472024-11-14T00:31:23.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14482024-11-14T00:31:23.158ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14492024-11-14T00:31:23.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14502024-11-14T00:31:23.192ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 51.392154ms iteration = 26 status = {"errors":[],"records_set_to_done":[]}
14512024-11-14T00:31:23.194ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
14522024-11-14T00:31:23.194ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14532024-11-14T00:31:23.195ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee50402b-8daa-48a5-b397-ccde4dfd518b", "content-length": "26", "date": "Thu, 14 Nov 2024 00:31:23 GMT"} })
14542024-11-14T00:31:23.195ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14552024-11-14T00:31:23.196ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14562024-11-14T00:31:23.196ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
14572024-11-14T00:31:23.196ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14582024-11-14T00:31:23.197ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "545ac938-1da9-42ef-a8c4-b1e00b80f8c6", "content-length": "26", "date": "Thu, 14 Nov 2024 00:31:23 GMT"} })
14592024-11-14T00:31:23.197ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14602024-11-14T00:31:23.197ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
14612024-11-14T00:31:23.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
14622024-11-14T00:31:23.199ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14632024-11-14T00:31:23.199ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
14642024-11-14T00:31:23.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
14652024-11-14T00:31:23.201ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
14662024-11-14T00:31:23.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
14672024-11-14T00:31:23.203ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 62.5437ms iteration = 26 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
14682024-11-14T00:31:23.203ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14692024-11-14T00:31:23.204ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 63.535014ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
14702024-11-14T00:31:23.205ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14712024-11-14T00:31:23.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14722024-11-14T00:31:23.206ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14732024-11-14T00:31:23.206ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
14742024-11-14T00:31:23.206ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14752024-11-14T00:31:23.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14762024-11-14T00:31:23.208ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14772024-11-14T00:31:23.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
14782024-11-14T00:31:23.208ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14792024-11-14T00:31:23.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14802024-11-14T00:31:23.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14812024-11-14T00:31:23.209ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14822024-11-14T00:31:23.210ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14832024-11-14T00:31:23.211ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e211d163-be78-4663-b73f-baf5e9462a10.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1782, tv_nsec: 927648453 } })
14842024-11-14T00:31:23.211ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864
14852024-11-14T00:31:23.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
14862024-11-14T00:31:23.212ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14872024-11-14T00:31:23.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14882024-11-14T00:31:23.213ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
14892024-11-14T00:31:23.213ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
14902024-11-14T00:31:23.213ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
14912024-11-14T00:31:23.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
14922024-11-14T00:31:23.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: e211d163-be78-4663-b73f-baf5e9462a10, lookup_type: ById(e211d163-be78-4663-b73f-baf5e9462a10) } result = Ok(())
14932024-11-14T00:31:23.215ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14942024-11-14T00:31:23.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14952024-11-14T00:31:23.217ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 76.359884ms iteration = 26 status = 0
14962024-11-14T00:31:23.218ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
14972024-11-14T00:31:23.218ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 77.217396ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
14982024-11-14T00:31:23.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14992024-11-14T00:31:23.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6178f930-5fa8-46a0-8fe5-b460818f9d6f, lookup_type: ById(6178f930-5fa8-46a0-8fe5-b460818f9d6f) } result = Ok(())
15002024-11-14T00:31:23.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15012024-11-14T00:31:23.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15022024-11-14T00:31:23.224ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 83.057059ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
15032024-11-14T00:31:23.231ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 90.140379ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
15042024-11-14T00:31:23.243ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f original_time_created = 2024-11-14 00:18:47.526379 UTC target_id = 6178f930-5fa8-46a0-8fe5-b460818f9d6f time_created = 2024-11-14 00:18:47.526379 UTC
15052024-11-14T00:31:23.243ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 103.764239ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"6178f930-5fa8-46a0-8fe5-b460818f9d6f","time_created":"2024-11-14T00:18:47.526379Z"}
15062024-11-14T00:31:23.250ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15072024-11-14T00:31:23.250ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
15082024-11-14T00:31:23.250ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 109.245267ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
15092024-11-14T00:31:23.284ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 144.848048ms iteration = 26 status = {}
15102024-11-14T00:31:23.292ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15112024-11-14T00:31:23.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
15122024-11-14T00:31:23.294ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
15132024-11-14T00:31:23.294ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6f575ab-22d4-4590-bb7a-ab6cbb0e129e", "content-length": "2", "date": "Thu, 14 Nov 2024 00:31:22 GMT"} })
15142024-11-14T00:31:23.296ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15152024-11-14T00:31:23.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
15162024-11-14T00:31:23.299ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 159.036868ms iteration = 26 status = null
15172024-11-14T00:31:23.299ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 zone_address = fd00:1122:3344:101::2
15182024-11-14T00:31:23.299ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15192024-11-14T00:31:23.299ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec29a3c3-81f9-4396-b8f0-6d21a6740cad", "content-length": "26", "date": "Thu, 14 Nov 2024 00:31:23 GMT"} })
15202024-11-14T00:31:23.299ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15212024-11-14T00:31:23.301ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864
15222024-11-14T00:31:23.340ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 199.149842ms iteration = 29 status = {"physical_disks_added":0}
15232024-11-14T00:31:23.341ZINFOnexus (BackgroundTasks): VPC route manager sled e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
15242024-11-14T00:31:23.341ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
15252024-11-14T00:31:23.341ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "47aea9be-9fb0-4599-94b5-921c7138d807", "content-length": "372", "date": "Thu, 14 Nov 2024 00:31:23 GMT"} })
15262024-11-14T00:31:23.342ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15272024-11-14T00:31:23.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15282024-11-14T00:31:23.389ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
15292024-11-14T00:31:23.389ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
15302024-11-14T00:31:23.389ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f44d926e-0f38-4250-8a37-518999914387", "date": "Thu, 14 Nov 2024 00:31:23 GMT"} })
15312024-11-14T00:31:23.389ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15322024-11-14T00:31:23.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15332024-11-14T00:31:23.392ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15342024-11-14T00:31:23.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15352024-11-14T00:31:23.393ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15362024-11-14T00:31:23.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
15372024-11-14T00:31:23.394ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15382024-11-14T00:31:23.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15392024-11-14T00:31:23.398ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15402024-11-14T00:31:23.400ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
15412024-11-14T00:31:23.400ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15422024-11-14T00:31:23.401ZINFOnexus (BackgroundTasks): VPC route manager sled e211d163-be78-4663-b73f-baf5e9462a10 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
15432024-11-14T00:31:23.401ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
15442024-11-14T00:31:23.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
15452024-11-14T00:31:23.401ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d2db24b5-b02d-4950-a71b-6573d62fd2ce", "date": "Thu, 14 Nov 2024 00:31:23 GMT"} })
15462024-11-14T00:31:23.402ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15472024-11-14T00:31:23.403ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 261.981846ms iteration = 27 status = {}
15482024-11-14T00:31:23.404ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
15492024-11-14T00:31:23.404ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15502024-11-14T00:31:23.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
15512024-11-14T00:31:23.452ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15522024-11-14T00:31:23.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15532024-11-14T00:31:23.454ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 1b492b5b-8c3a-4e99-9528-5d52a91581c5, rack_id: fcfbb675-622a-4d07-9c7d-02f1c7317864, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d5fc9a38-ca99-44ad-95a3-d22ca6bdfc5b) } rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 switch_location = Switch0
15542024-11-14T00:31:23.455ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15552024-11-14T00:31:23.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15562024-11-14T00:31:23.465ZDEBGnexus (BackgroundTasks): client request SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
15572024-11-14T00:31:23.465ZDEBGnexus (BackgroundTasks): client response SledAgent = e211d163-be78-4663-b73f-baf5e9462a10 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "01060b36-5f09-4293-b889-e90130f3942e", "content-length": "691", "date": "Thu, 14 Nov 2024 00:31:23 GMT"} })
15582024-11-14T00:31:23.465ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
15592024-11-14T00:31:23.465ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
15602024-11-14T00:31:23.465ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
15612024-11-14T00:31:23.475ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
15622024-11-14T00:31:23.475ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 334.237803ms iteration = 27 status = {}
15632024-11-14T00:31:23.475ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "a95c738b-cff1-4f4d-a399-cc7114056c6f", "content-length": "134", "date": "Thu, 14 Nov 2024 00:31:22 GMT"} })
15642024-11-14T00:31:23.475ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1747 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
15652024-11-14T00:31:23.475ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1774 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d5fc9a38-ca99-44ad-95a3-d22ca6bdfc5b
15662024-11-14T00:31:23.476ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
15672024-11-14T00:31:23.476ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
15682024-11-14T00:31:23.476ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15692024-11-14T00:31:23.476ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15702024-11-14T00:31:23.476ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15712024-11-14T00:31:23.476ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1921 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
15722024-11-14T00:31:23.476ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15732024-11-14T00:31:23.476ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1957 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
15742024-11-14T00:31:23.476ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864
15752024-11-14T00:31:23.476ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15762024-11-14T00:31:23.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15772024-11-14T00:31:23.478ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
15782024-11-14T00:31:23.478ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a0a4c85-6dd3-4e99-8a89-1796043dd8a8", "content-length": "2", "date": "Thu, 14 Nov 2024 00:31:22 GMT"} })
15792024-11-14T00:31:23.478ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
15802024-11-14T00:31:23.478ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c3c3284-d1f4-4881-ba86-7a144f09efbd", "content-length": "52", "date": "Thu, 14 Nov 2024 00:31:22 GMT"} })
15812024-11-14T00:31:23.478ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 switch_location = Switch0
15822024-11-14T00:31:23.478ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
15832024-11-14T00:31:23.601ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c7cbbd63-fd8a-4bd0-b3ce-40e33463e98a", "date": "Thu, 14 Nov 2024 00:31:23 GMT"} })
15842024-11-14T00:31:23.601ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 switch = Switch0
15852024-11-14T00:31:23.601ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
15862024-11-14T00:31:23.602ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = fcfbb675-622a-4d07-9c7d-02f1c7317864 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d917d56-34a8-42a9-9d67-e047efa9f552", "content-length": "682", "date": "Thu, 14 Nov 2024 00:31:23 GMT"} })
15872024-11-14T00:31:23.602ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15882024-11-14T00:31:23.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15892024-11-14T00:31:23.606ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15902024-11-14T00:31:23.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15912024-11-14T00:31:23.609ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15922024-11-14T00:31:23.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15932024-11-14T00:31:23.611ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15942024-11-14T00:31:23.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15952024-11-14T00:31:23.618ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 477.55083ms iteration = 26 status = {}
15962024-11-14T00:31:24.069ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 44599918-e62d-4453-bb88-0b7d04bc5afa roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
15972024-11-14T00:31:24.070ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 44599918-e62d-4453-bb88-0b7d04bc5afa resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
15982024-11-14T00:31:24.072ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2172 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 44599918-e62d-4453-bb88-0b7d04bc5afa response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
15992024-11-14T00:31:25.093ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e35bfc62-363b-4395-be83-0e90bd4bf279 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16002024-11-14T00:31:25.093ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e35bfc62-363b-4395-be83-0e90bd4bf279 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16012024-11-14T00:31:25.095ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2306 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e35bfc62-363b-4395-be83-0e90bd4bf279 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16022024-11-14T00:31:26.117ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6e58c6e8-a588-4cf5-a0ab-76958cb6646f roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16032024-11-14T00:31:26.117ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6e58c6e8-a588-4cf5-a0ab-76958cb6646f resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16042024-11-14T00:31:26.119ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2444 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 6e58c6e8-a588-4cf5-a0ab-76958cb6646f response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16052024-11-14T00:31:27.139ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = c6e2fab3-7a97-4bee-8aac-9adfc59df5fb roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16062024-11-14T00:31:27.140ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = c6e2fab3-7a97-4bee-8aac-9adfc59df5fb resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16072024-11-14T00:31:27.141ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2400 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = c6e2fab3-7a97-4bee-8aac-9adfc59df5fb response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16082024-11-14T00:31:28.164ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e7a1f319-9538-4c60-b133-ef12205c4580 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16092024-11-14T00:31:28.164ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e7a1f319-9538-4c60-b133-ef12205c4580 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16102024-11-14T00:31:28.167ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3364 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = e7a1f319-9538-4c60-b133-ef12205c4580 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16112024-11-14T00:31:29.187ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 4da88725-f54c-49f2-8644-9b0648301d3a roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16122024-11-14T00:31:29.188ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 4da88725-f54c-49f2-8644-9b0648301d3a resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16132024-11-14T00:31:29.190ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2498 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 4da88725-f54c-49f2-8644-9b0648301d3a response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16142024-11-14T00:31:30.210ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 8fc066cd-7609-40b3-ad15-1902e2bd26b2 roles = RoleSet { roles: {} } uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16152024-11-14T00:31:30.210ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 8fc066cd-7609-40b3-ad15-1902e2bd26b2 resource = Database result = Ok(()) uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10
16162024-11-14T00:31:30.212ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2118 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:43748 req_id = 8fc066cd-7609-40b3-ad15-1902e2bd26b2 response_code = 200 uri = /probes/e211d163-be78-4663-b73f-baf5e9462a10