Max level shown:
12024-12-07T01:00:07.116ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
22024-12-07T01:00:07.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
32024-12-07T01:00:07.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42024-12-07T01:00:07.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52024-12-07T01:00:07.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62024-12-07T01:00:07.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
72024-12-07T01:00:07.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82024-12-07T01:00:07.119ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
92024-12-07T01:00:07.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
102024-12-07T01:00:07.121ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
112024-12-07T01:00:07.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
122024-12-07T01:00:07.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132024-12-07T01:00:07.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
142024-12-07T01:00:07.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
152024-12-07T01:00:07.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162024-12-07T01:00:07.130ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
172024-12-07T01:00:07.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
182024-12-07T01:00:07.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
192024-12-07T01:00:07.143ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
202024-12-07T01:00:07.143ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.418971ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
212024-12-07T01:00:07.162ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 45.677701ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
222024-12-07T01:00:07.185ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1315 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = 6aec94d3-fb4b-4b1a-829a-3f1bffbbc3df response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
232024-12-07T01:00:12.177ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:34061
242024-12-07T01:00:12.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 15c60972-45c7-4451-b74b-a5fa71cd0bf5 roles = RoleSet { roles: {} } uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
252024-12-07T01:00:12.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 15c60972-45c7-4451-b74b-a5fa71cd0bf5 resource = Database result = Ok(()) uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
262024-12-07T01:00:12.181ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3518 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 15c60972-45c7-4451-b74b-a5fa71cd0bf5 response_code = 200 uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
272024-12-07T01:00:12.181ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 67f87c41-fd17-4b0a-ab50-5cd63b4c2447 roles = RoleSet { roles: {} } uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
282024-12-07T01:00:12.181ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 67f87c41-fd17-4b0a-ab50-5cd63b4c2447 resource = Database result = Ok(()) uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
292024-12-07T01:00:12.183ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2256 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 67f87c41-fd17-4b0a-ab50-5cd63b4c2447 response_code = 200 uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
302024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
312024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
322024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
332024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
342024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
352024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
362024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
372024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
382024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
392024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
402024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
412024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
422024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
432024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
442024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
452024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
462024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
472024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
482024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
492024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
502024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
512024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
522024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
532024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
542024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
552024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
562024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
572024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
582024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
592024-12-07T01:00:17.116ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
602024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
612024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
622024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
632024-12-07T01:00:17.116ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
642024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
652024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
662024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
672024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
682024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
692024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
702024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
712024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
722024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
732024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
742024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
752024-12-07T01:00:17.116ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-07 00:50:17.116939785 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
762024-12-07T01:00:17.116ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
772024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
782024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
792024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
802024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
812024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
822024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
832024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
842024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
852024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
862024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
872024-12-07T01:00:17.117ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70
882024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
892024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
902024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
912024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
922024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.35\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70"}
932024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
942024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
952024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
962024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
972024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
982024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
992024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1002024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1012024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1022024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1032024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1042024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1052024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1062024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1072024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1082024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1092024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1102024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1112024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
1122024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1132024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1142024-12-07T01:00:17.117ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("eda26d3e-18ec-4c21-8ff4-73abfd373bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("fe4d45f4-708c-423b-9b9c-06a074a360b5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("fe4d45f4-708c-423b-9b9c-06a074a360b5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 4480, tv_nsec: 342701961 } })
1152024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
1162024-12-07T01:00:17.117ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5661fac0-67b8-45f8-869a-67dd3e21bd0b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("be11a55e-cb67-4535-ad44-b9a4d4586728.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("fd72061d-6779-40de-8011-8e899981c9e7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("fd72061d-6779-40de-8011-8e899981c9e7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 4480, tv_nsec: 342702661 } })
1172024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1182024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1192024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1202024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1212024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1222024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1232024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1242024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1252024-12-07T01:00:17.117ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1262024-12-07T01:00:17.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1272024-12-07T01:00:17.118ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1282024-12-07T01:00:17.118ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
1292024-12-07T01:00:17.118ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.166033ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
1302024-12-07T01:00:17.118ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
1312024-12-07T01:00:17.118ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.411438ms iteration = 13 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
1322024-12-07T01:00:17.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1332024-12-07T01:00:17.120ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1342024-12-07T01:00:17.121ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.219243ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1352024-12-07T01:00:17.121ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1362024-12-07T01:00:17.121ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1372024-12-07T01:00:17.121ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1382024-12-07T01:00:17.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1392024-12-07T01:00:17.121ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1402024-12-07T01:00:17.122ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1412024-12-07T01:00:17.122ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1422024-12-07T01:00:17.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1432024-12-07T01:00:17.123ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1442024-12-07T01:00:17.123ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1452024-12-07T01:00:17.123ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1462024-12-07T01:00:17.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1472024-12-07T01:00:17.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1482024-12-07T01:00:17.125ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1492024-12-07T01:00:17.127ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1502024-12-07T01:00:17.127ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1512024-12-07T01:00:17.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1522024-12-07T01:00:17.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1532024-12-07T01:00:17.129ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1542024-12-07T01:00:17.130ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 13.246071ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
1552024-12-07T01:00:17.131ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1562024-12-07T01:00:17.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1572024-12-07T01:00:17.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1582024-12-07T01:00:17.132ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1592024-12-07T01:00:17.132ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1602024-12-07T01:00:17.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1612024-12-07T01:00:17.135ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1622024-12-07T01:00:17.137ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1632024-12-07T01:00:17.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1642024-12-07T01:00:17.137ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1652024-12-07T01:00:17.137ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1662024-12-07T01:00:17.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1672024-12-07T01:00:17.138ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-07 00:50:17.116939785 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
1682024-12-07T01:00:17.138ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 21.560155ms iteration = 13 status = {"expiration":"2024-12-07T00:50:17.116939785Z","pruned":[]}
1692024-12-07T01:00:17.138ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4480, tv_nsec: 363677805 } })
1702024-12-07T01:00:17.138ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
1712024-12-07T01:00:17.139ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4480, tv_nsec: 364677575 } })
1722024-12-07T01:00:17.139ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
1732024-12-07T01:00:17.141ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1742024-12-07T01:00:17.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1752024-12-07T01:00:17.144ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1762024-12-07T01:00:17.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1772024-12-07T01:00:17.148ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1782024-12-07T01:00:17.151ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4480, tv_nsec: 376378505 } })
1792024-12-07T01:00:17.151ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
1802024-12-07T01:00:17.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1812024-12-07T01:00:17.153ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1822024-12-07T01:00:17.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1832024-12-07T01:00:17.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1842024-12-07T01:00:17.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1852024-12-07T01:00:17.173ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1862024-12-07T01:00:17.174ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1872024-12-07T01:00:17.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1882024-12-07T01:00:17.175ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 58.293719ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
1892024-12-07T01:00:17.176ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1902024-12-07T01:00:17.176ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1912024-12-07T01:00:17.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1922024-12-07T01:00:17.177ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1932024-12-07T01:00:17.179ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1942024-12-07T01:00:17.179ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1952024-12-07T01:00:17.179ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
1962024-12-07T01:00:17.179ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1972024-12-07T01:00:17.179ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
1982024-12-07T01:00:17.179ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
1992024-12-07T01:00:17.179ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 63.283197ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
2002024-12-07T01:00:17.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2012024-12-07T01:00:17.180ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2022024-12-07T01:00:17.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2032024-12-07T01:00:17.181ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 64.597434ms iteration = 25 status = {"errors":[],"records_set_to_done":[]}
2042024-12-07T01:00:17.182ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
2052024-12-07T01:00:17.182ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2062024-12-07T01:00:17.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
2072024-12-07T01:00:17.182ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2082024-12-07T01:00:17.182ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2092024-12-07T01:00:17.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2102024-12-07T01:00:17.184ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
2112024-12-07T01:00:17.184ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
2122024-12-07T01:00:17.184ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
2132024-12-07T01:00:17.184ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2142024-12-07T01:00:17.184ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2152024-12-07T01:00:17.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2162024-12-07T01:00:17.184ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2172024-12-07T01:00:17.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2182024-12-07T01:00:17.185ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2192024-12-07T01:00:17.186ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1976 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = fedeceda-1a8a-48db-af98-90b8be271e6e response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
2202024-12-07T01:00:17.187ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2212024-12-07T01:00:17.188ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2222024-12-07T01:00:17.188ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2232024-12-07T01:00:17.188ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2242024-12-07T01:00:17.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2252024-12-07T01:00:17.189ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2262024-12-07T01:00:17.189ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2272024-12-07T01:00:17.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2282024-12-07T01:00:17.190ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2292024-12-07T01:00:17.190ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2302024-12-07T01:00:17.191ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
2312024-12-07T01:00:17.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2322024-12-07T01:00:17.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2332024-12-07T01:00:17.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2342024-12-07T01:00:17.191ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2352024-12-07T01:00:17.191ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2362024-12-07T01:00:17.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2372024-12-07T01:00:17.192ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2382024-12-07T01:00:17.192ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2392024-12-07T01:00:17.192ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2402024-12-07T01:00:17.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2412024-12-07T01:00:17.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2422024-12-07T01:00:17.193ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2432024-12-07T01:00:17.194ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2442024-12-07T01:00:17.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2452024-12-07T01:00:17.197ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2462024-12-07T01:00:17.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2472024-12-07T01:00:17.198ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2482024-12-07T01:00:17.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2492024-12-07T01:00:17.199ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
2502024-12-07T01:00:17.199ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2512024-12-07T01:00:17.199ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2522024-12-07T01:00:17.199ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2532024-12-07T01:00:17.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2542024-12-07T01:00:17.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2552024-12-07T01:00:17.200ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2562024-12-07T01:00:17.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2572024-12-07T01:00:17.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2582024-12-07T01:00:17.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2592024-12-07T01:00:17.208ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 91.698598ms iteration = 6 status = {"nsuccess":5}
2602024-12-07T01:00:17.211ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2612024-12-07T01:00:17.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2622024-12-07T01:00:17.216ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2632024-12-07T01:00:17.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2642024-12-07T01:00:17.225ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
2652024-12-07T01:00:17.225ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 108.620972ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
2662024-12-07T01:00:17.226ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 109.650612ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2672024-12-07T01:00:17.228ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2682024-12-07T01:00:17.228ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
2692024-12-07T01:00:17.228ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 111.469338ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
2702024-12-07T01:00:17.230ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
2712024-12-07T01:00:17.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
2722024-12-07T01:00:17.248ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
2732024-12-07T01:00:17.248ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
2742024-12-07T01:00:17.248ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 131.797599ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
2752024-12-07T01:00:17.248ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2762024-12-07T01:00:17.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2772024-12-07T01:00:17.250ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2782024-12-07T01:00:17.250ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 133.628496ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2792024-12-07T01:00:17.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2802024-12-07T01:00:17.253ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2812024-12-07T01:00:17.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2822024-12-07T01:00:17.258ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
2832024-12-07T01:00:17.258ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ae6d4ace-ffdd-4139-9e11-1794ae966042 dns_zone_name = oxide.test found_so_far = 0 version = 2
2842024-12-07T01:00:17.258ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2852024-12-07T01:00:17.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2862024-12-07T01:00:17.260ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2872024-12-07T01:00:17.261ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
2882024-12-07T01:00:17.261ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
2892024-12-07T01:00:17.261ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 52ba8747-d0f6-43d9-9ec0-995aa6df639a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
2902024-12-07T01:00:17.261ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2912024-12-07T01:00:17.262ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2922024-12-07T01:00:17.262ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2932024-12-07T01:00:17.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2942024-12-07T01:00:17.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2952024-12-07T01:00:17.266ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 149.256133ms iteration = 25 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
2962024-12-07T01:00:17.296ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 179.192783ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
2972024-12-07T01:00:17.296ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2982024-12-07T01:00:17.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2992024-12-07T01:00:17.303ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3002024-12-07T01:00:17.303ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3012024-12-07T01:00:17.303ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b080c92f-ff37-4a53-aee4-b84658c129a8", "content-length": "26", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} })
3022024-12-07T01:00:17.303ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3032024-12-07T01:00:17.305ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3042024-12-07T01:00:17.333ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
3052024-12-07T01:00:17.341ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
3062024-12-07T01:00:17.341ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 224.313753ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
3072024-12-07T01:00:17.341ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a zone_address = fd00:1122:3344:101::2
3082024-12-07T01:00:17.341ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3092024-12-07T01:00:17.341ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a7788a7-7a52-45de-ab4c-f9d54bfbdc8b", "content-length": "26", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} })
3102024-12-07T01:00:17.341ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3112024-12-07T01:00:17.343ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
3122024-12-07T01:00:17.456ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
3132024-12-07T01:00:17.456ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3142024-12-07T01:00:17.457ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e27888a4-147f-4288-84ca-e9318c49d14f", "content-length": "26", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} })
3152024-12-07T01:00:17.457ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3162024-12-07T01:00:17.459ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/cdff06af-548d-4d6e-8e27-5f181e4ed221/state
3172024-12-07T01:00:17.460ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3182024-12-07T01:00:17.472ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
3192024-12-07T01:00:17.473ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3202024-12-07T01:00:17.473ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "58a499d1-1c95-4903-87fa-6459f4bd8b98", "content-length": "1", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} })
3212024-12-07T01:00:17.474ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/cdff06af-548d-4d6e-8e27-5f181e4ed221/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "65fe7c75-ffbe-4080-9e2c-8e8c7e69ae97", "content-length": "130", "date": "Sat, 07 Dec 2024 01:00:16 GMT"} })
3222024-12-07T01:00:17.474ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = d4e357bc-806f-4b64-837e-f3433fef287e sled_id = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-12-07T00:59:43.830514997Z }, migration_in: None, migration_out: None } vmm_id = cdff06af-548d-4d6e-8e27-5f181e4ed221
3232024-12-07T01:00:17.474ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2116 instance_id = d4e357bc-806f-4b64-837e-f3433fef287e migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = cdff06af-548d-4d6e-8e27-5f181e4ed221 sled_id = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 vmm_id = cdff06af-548d-4d6e-8e27-5f181e4ed221 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-12-07T00:59:43.830514997Z }
3242024-12-07T01:00:17.474ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = d4e357bc-806f-4b64-837e-f3433fef287e roles = RoleSet { roles: {} } sled_id = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 vmm_id = cdff06af-548d-4d6e-8e27-5f181e4ed221
3252024-12-07T01:00:17.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = d4e357bc-806f-4b64-837e-f3433fef287e resource = Database result = Ok(()) sled_id = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 vmm_id = cdff06af-548d-4d6e-8e27-5f181e4ed221
3262024-12-07T01:00:17.475ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3272024-12-07T01:00:17.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3282024-12-07T01:00:17.480ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3292024-12-07T01:00:17.480ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3302024-12-07T01:00:17.481ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "36b8c3c5-6057-4bd2-8189-89bbfba2c4f7", "content-length": "583", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} })
3312024-12-07T01:00:17.481ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3322024-12-07T01:00:17.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3332024-12-07T01:00:17.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3342024-12-07T01:00:17.483ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3352024-12-07T01:00:17.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3362024-12-07T01:00:17.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3372024-12-07T01:00:17.487ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3382024-12-07T01:00:17.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3392024-12-07T01:00:17.489ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 372.431044ms iteration = 25 status = null
3402024-12-07T01:00:17.490ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3412024-12-07T01:00:17.491ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ae6d4ace-ffdd-4139-9e11-1794ae966042 dns_zone_name = oxide.test found_so_far = 1 version = 2
3422024-12-07T01:00:17.491ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
3432024-12-07T01:00:17.491ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-07 00:48:45.853224 UTC dns_group = external generation = 2
3442024-12-07T01:00:17.491ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 374.114627ms iteration = 13 status = {"generation":2}
3452024-12-07T01:00:17.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3, lookup_type: ById(e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3) } result = Ok(())
3462024-12-07T01:00:17.492ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3472024-12-07T01:00:17.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3482024-12-07T01:00:17.495ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 378.283179ms iteration = 25 status = 0
3492024-12-07T01:00:17.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3502024-12-07T01:00:17.517ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3512024-12-07T01:00:17.520ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3522024-12-07T01:00:17.520ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3532024-12-07T01:00:17.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3542024-12-07T01:00:17.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3552024-12-07T01:00:17.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3562024-12-07T01:00:17.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3572024-12-07T01:00:17.524ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 52ba8747-d0f6-43d9-9ec0-995aa6df639a dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
3582024-12-07T01:00:17.524ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3592024-12-07T01:00:17.524ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-07 00:48:44.409012 UTC dns_group = internal generation = 1
3602024-12-07T01:00:17.524ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 408.017666ms iteration = 13 status = {"generation":1}
3612024-12-07T01:00:17.525ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
3622024-12-07T01:00:17.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "c0b5a599-4585-4285-9895-f665e4e9b136", "date": "Sat, 07 Dec 2024 01:00:16 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3632024-12-07T01:00:17.526ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3642024-12-07T01:00:17.526ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 409.511476ms iteration = 27 status = {"physical_disks_added":0}
3652024-12-07T01:00:17.527ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "c4f1ca94-cee7-4e79-b6a6-ebde331f71f5", "date": "Sat, 07 Dec 2024 01:00:16 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3662024-12-07T01:00:17.527ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3672024-12-07T01:00:17.529ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3682024-12-07T01:00:17.529ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 413.035165ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
3692024-12-07T01:00:17.531ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
3702024-12-07T01:00:17.533ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 416.35325ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"running":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
3712024-12-07T01:00:17.534ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {ad42495b-040b-4b67-a0fa-9d16ddaed7a9: {10.151.2.176: {4f344fc7-2464-4c76-8b6a-425472f23c9f}, 10.151.2.175: {4f344fc7-2464-4c76-8b6a-425472f23c9f}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
3722024-12-07T01:00:17.534ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3732024-12-07T01:00:17.534ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6020b989-8fe3-4f98-9838-7b5ec7daf4af", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} })
3742024-12-07T01:00:17.534ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3752024-12-07T01:00:17.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3762024-12-07T01:00:17.537ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3772024-12-07T01:00:17.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3782024-12-07T01:00:17.541ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3792024-12-07T01:00:17.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3802024-12-07T01:00:17.544ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3812024-12-07T01:00:17.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3822024-12-07T01:00:17.546ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3832024-12-07T01:00:17.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3842024-12-07T01:00:17.550ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3852024-12-07T01:00:17.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3862024-12-07T01:00:17.553ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-07T00:48:09.490271Z, time_modified: 2024-12-07T00:48:09.490271Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, db_silo: Silo { identity: SiloIdentity { id: 3a67a720-d4af-42e7-875a-3abed4955db5, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-07T00:48:45.569718Z, time_modified: 2024-12-07T00:48:45.569718Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6317ba4eb1538be6191eb18b3b3531d70dbee51545c790c2aff24b81494c24f4" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, db_silo: Silo { identity: SiloIdentity { id: 3a67a720-d4af-42e7-875a-3abed4955db5, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-07T00:48:45.569718Z, time_modified: 2024-12-07T00:48:45.569718Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6317ba4eb1538be6191eb18b3b3531d70dbee51545c790c2aff24b81494c24f4" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
3872024-12-07T01:00:17.553ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 437.276593ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"3a67a720-d4af-42e7-875a-3abed4955db5","tls_certs":["6317ba4eb1538be6191eb18b3b3531d70dbee51545c790c2aff24b81494c24f4"]}},"default_endpoint":{"silo_id":"3a67a720-d4af-42e7-875a-3abed4955db5","tls_certs":["6317ba4eb1538be6191eb18b3b3531d70dbee51545c790c2aff24b81494c24f4"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
3882024-12-07T01:00:17.556ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3892024-12-07T01:00:17.556ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3902024-12-07T01:00:17.556ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3912024-12-07T01:00:17.556ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "897f7729-ecbe-40aa-b783-c949a3c316e3", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} })
3922024-12-07T01:00:17.557ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 441.125969ms iteration = 26 status = {}
3932024-12-07T01:00:17.571ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3942024-12-07T01:00:17.572ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "46725535-350c-4a17-bcf9-9f206d7bb5d8", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
3952024-12-07T01:00:17.572ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
3962024-12-07T01:00:17.573ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "1fb15190-1877-43d5-9744-a2f76113410f", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
3972024-12-07T01:00:17.573ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
3982024-12-07T01:00:17.574ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "f9afd976-a9a7-4555-baa5-9ac6fa14698c", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
3992024-12-07T01:00:17.574ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
4002024-12-07T01:00:17.576ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
4012024-12-07T01:00:17.576ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 459.991901ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
4022024-12-07T01:00:17.578ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4032024-12-07T01:00:17.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4042024-12-07T01:00:17.580ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4052024-12-07T01:00:17.581ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4062024-12-07T01:00:17.581ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4072024-12-07T01:00:17.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4082024-12-07T01:00:17.582ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4092024-12-07T01:00:17.584ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4102024-12-07T01:00:17.584ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4112024-12-07T01:00:17.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4122024-12-07T01:00:17.615ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 498.779455ms iteration = 25 status = {}
4132024-12-07T01:00:17.681ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4142024-12-07T01:00:17.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4152024-12-07T01:00:17.684ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 68b8f65f-54d5-4192-ac26-facf91340245, rack_id: 667e4910-296b-4cf9-a580-7c638e7aa45a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(e642095a-6c9c-4557-b65a-98313f61cfeb) } rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0
4162024-12-07T01:00:17.684ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4172024-12-07T01:00:17.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4182024-12-07T01:00:17.692ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4192024-12-07T01:00:17.693ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe0f2533-3fe5-41c4-a200-934bd8d069ab", "content-length": "810", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} })
4202024-12-07T01:00:17.693ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(1098235) }} sled = sock
4212024-12-07T01:00:17.693ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
4222024-12-07T01:00:17.693ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
4232024-12-07T01:00:17.694ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4242024-12-07T01:00:17.694ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2ba1f6f-55e2-43e2-9719-3ad31784cf7e", "content-length": "134", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} })
4252024-12-07T01:00:17.694ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
4262024-12-07T01:00:17.694ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = e642095a-6c9c-4557-b65a-98313f61cfeb
4272024-12-07T01:00:17.695ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
4282024-12-07T01:00:17.695ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
4292024-12-07T01:00:17.695ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4302024-12-07T01:00:17.695ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4312024-12-07T01:00:17.695ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4322024-12-07T01:00:17.695ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4332024-12-07T01:00:17.695ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4342024-12-07T01:00:17.695ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4352024-12-07T01:00:17.695ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 578.474597ms iteration = 26 status = {}
4362024-12-07T01:00:17.695ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
4372024-12-07T01:00:17.695ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4382024-12-07T01:00:17.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4392024-12-07T01:00:17.697ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4402024-12-07T01:00:17.697ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "69c771dd-6c13-4bde-a5f5-513ec591a9d6", "content-length": "2", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} })
4412024-12-07T01:00:17.697ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4422024-12-07T01:00:17.697ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "c17eee68-0d6f-4c12-87ee-a7105a9ac9ab", "content-length": "52", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} })
4432024-12-07T01:00:17.697ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0
4442024-12-07T01:00:17.697ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4452024-12-07T01:00:17.792ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "48903a8b-5064-47fd-a725-75121c1e00c4", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} })
4462024-12-07T01:00:17.792ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch = Switch0
4472024-12-07T01:00:17.792ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4482024-12-07T01:00:17.793ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa6697ea-0f58-4192-9074-c2589befa9c1", "content-length": "682", "date": "Sat, 07 Dec 2024 01:00:17 GMT"} })
4492024-12-07T01:00:17.793ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4502024-12-07T01:00:17.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4512024-12-07T01:00:17.797ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4522024-12-07T01:00:17.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4532024-12-07T01:00:17.801ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4542024-12-07T01:00:17.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4552024-12-07T01:00:17.803ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4562024-12-07T01:00:17.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4572024-12-07T01:00:17.810ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 694.001445ms iteration = 25 status = {}
4582024-12-07T01:00:17.996ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4592024-12-07T01:00:17.996ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4602024-12-07T01:00:17.998ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4612024-12-07T01:00:17.998ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4622024-12-07T01:00:18.000ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
4632024-12-07T01:00:18.001ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("token = \\"309f87fb99ee10b5e9dbf0867a83b04371747aad\\"") } result = Ok(())
4642024-12-07T01:00:18.001ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4652024-12-07T01:00:18.001ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4662024-12-07T01:00:18.006ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4672024-12-07T01:00:18.006ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4682024-12-07T01:00:18.009ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4692024-12-07T01:00:18.009ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4702024-12-07T01:00:18.011ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4712024-12-07T01:00:18.011ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4722024-12-07T01:00:18.013ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
4732024-12-07T01:00:18.014ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
4742024-12-07T01:00:18.014ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4752024-12-07T01:00:18.014ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4762024-12-07T01:00:18.016ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
4772024-12-07T01:00:18.017ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("309f87fb99ee10b5e9dbf0867a83b04371747aad") } result = Ok(())
4782024-12-07T01:00:18.017ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4792024-12-07T01:00:18.017ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4802024-12-07T01:00:18.028ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4812024-12-07T01:00:18.028ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4822024-12-07T01:00:18.030ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4832024-12-07T01:00:18.030ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4842024-12-07T01:00:18.032ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4852024-12-07T01:00:18.032ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4862024-12-07T01:00:18.033ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4872024-12-07T01:00:18.033ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4882024-12-07T01:00:18.035ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
4892024-12-07T01:00:18.036ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
4902024-12-07T01:00:18.036ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4912024-12-07T01:00:18.036ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4922024-12-07T01:00:18.038ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4932024-12-07T01:00:18.038ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4942024-12-07T01:00:18.066ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
4952024-12-07T01:00:18.066ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
4962024-12-07T01:00:18.068ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
4972024-12-07T01:00:18.069ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) } result = Ok(())
4982024-12-07T01:00:18.069ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
4992024-12-07T01:00:18.069ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5002024-12-07T01:00:18.071ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5012024-12-07T01:00:18.071ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5022024-12-07T01:00:18.073ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5032024-12-07T01:00:18.073ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5042024-12-07T01:00:18.076ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5052024-12-07T01:00:18.076ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5062024-12-07T01:00:18.079ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5072024-12-07T01:00:18.079ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5082024-12-07T01:00:18.082ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5092024-12-07T01:00:18.082ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5102024-12-07T01:00:18.085ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5112024-12-07T01:00:18.085ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5122024-12-07T01:00:18.087ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5132024-12-07T01:00:18.090ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ByName("instance-df6ebfd9906") } result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5142024-12-07T01:00:18.090ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5152024-12-07T01:00:18.090ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5162024-12-07T01:00:18.093ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5172024-12-07T01:00:18.093ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5182024-12-07T01:00:18.096ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5192024-12-07T01:00:18.096ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5202024-12-07T01:00:18.098ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5212024-12-07T01:00:18.098ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5222024-12-07T01:00:18.100ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5232024-12-07T01:00:18.103ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ByName("instance-df6ebfd9906") } result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5242024-12-07T01:00:18.103ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5252024-12-07T01:00:18.103ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5262024-12-07T01:00:18.106ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 110056 local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 5beb57bd-a999-4e47-bbba-238ee3fbdbe2 response_code = 200 uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
5272024-12-07T01:00:18.107ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5282024-12-07T01:00:18.107ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5292024-12-07T01:00:18.109ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5302024-12-07T01:00:18.109ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5312024-12-07T01:00:18.110ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
5322024-12-07T01:00:18.112ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("token = \\"309f87fb99ee10b5e9dbf0867a83b04371747aad\\"") } result = Ok(())
5332024-12-07T01:00:18.112ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5342024-12-07T01:00:18.112ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5352024-12-07T01:00:18.113ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5362024-12-07T01:00:18.113ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5372024-12-07T01:00:18.114ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5382024-12-07T01:00:18.115ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5392024-12-07T01:00:18.115ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5402024-12-07T01:00:18.116ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5412024-12-07T01:00:18.117ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
5422024-12-07T01:00:18.118ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
5432024-12-07T01:00:18.118ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5442024-12-07T01:00:18.118ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5452024-12-07T01:00:18.119ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
5462024-12-07T01:00:18.120ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("309f87fb99ee10b5e9dbf0867a83b04371747aad") } result = Ok(())
5472024-12-07T01:00:18.120ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5482024-12-07T01:00:18.120ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5492024-12-07T01:00:18.127ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5502024-12-07T01:00:18.127ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5512024-12-07T01:00:18.129ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5522024-12-07T01:00:18.129ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5532024-12-07T01:00:18.131ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5542024-12-07T01:00:18.131ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5552024-12-07T01:00:18.132ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5562024-12-07T01:00:18.132ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5572024-12-07T01:00:18.133ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
5582024-12-07T01:00:18.135ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
5592024-12-07T01:00:18.135ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5602024-12-07T01:00:18.135ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5612024-12-07T01:00:18.136ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5622024-12-07T01:00:18.136ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5632024-12-07T01:00:18.137ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5642024-12-07T01:00:18.137ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5652024-12-07T01:00:18.138ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
5662024-12-07T01:00:18.139ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) } result = Ok(())
5672024-12-07T01:00:18.139ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5682024-12-07T01:00:18.140ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5692024-12-07T01:00:18.141ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5702024-12-07T01:00:18.141ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5712024-12-07T01:00:18.143ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5722024-12-07T01:00:18.143ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5732024-12-07T01:00:18.144ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5742024-12-07T01:00:18.145ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5752024-12-07T01:00:18.147ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5762024-12-07T01:00:18.147ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5772024-12-07T01:00:18.148ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5782024-12-07T01:00:18.149ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5792024-12-07T01:00:18.150ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5802024-12-07T01:00:18.151ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5812024-12-07T01:00:18.153ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5822024-12-07T01:00:18.156ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ByName("instance-df6ebfd9906") } result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5832024-12-07T01:00:18.156ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5842024-12-07T01:00:18.156ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5852024-12-07T01:00:18.158ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5862024-12-07T01:00:18.158ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5872024-12-07T01:00:18.160ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5882024-12-07T01:00:18.160ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5892024-12-07T01:00:18.162ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5902024-12-07T01:00:18.162ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5912024-12-07T01:00:18.164ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5922024-12-07T01:00:18.167ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ByName("instance-df6ebfd9906") } result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5932024-12-07T01:00:18.167ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5942024-12-07T01:00:18.167ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5952024-12-07T01:00:18.203ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95516 local_addr = 172.30.2.5:443 method = GET remote_addr = 10.151.2.100:44326 req_id = 2c55f29e-f155-43ba-a81b-fd7a26395e43 response_code = 200 uri = /v1/instances/instance-df6ebfd9906/serial-console?max_bytes=1048576&most_recent=1048576&project=proj-391b50e44b9
5962024-12-07T01:00:19.059ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5972024-12-07T01:00:19.059ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
5982024-12-07T01:00:19.061ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
5992024-12-07T01:00:19.061ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6002024-12-07T01:00:19.063ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
6012024-12-07T01:00:19.064ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("token = \\"309f87fb99ee10b5e9dbf0867a83b04371747aad\\"") } result = Ok(())
6022024-12-07T01:00:19.064ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6032024-12-07T01:00:19.064ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6042024-12-07T01:00:19.066ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6052024-12-07T01:00:19.066ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6062024-12-07T01:00:19.068ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6072024-12-07T01:00:19.068ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6082024-12-07T01:00:19.070ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6092024-12-07T01:00:19.070ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6102024-12-07T01:00:19.071ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
6112024-12-07T01:00:19.072ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
6122024-12-07T01:00:19.072ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6132024-12-07T01:00:19.072ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6142024-12-07T01:00:19.074ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
6152024-12-07T01:00:19.075ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("309f87fb99ee10b5e9dbf0867a83b04371747aad") } result = Ok(())
6162024-12-07T01:00:19.075ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6172024-12-07T01:00:19.075ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6182024-12-07T01:00:19.080ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6192024-12-07T01:00:19.080ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6202024-12-07T01:00:19.082ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6212024-12-07T01:00:19.082ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6222024-12-07T01:00:19.083ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6232024-12-07T01:00:19.083ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6242024-12-07T01:00:19.084ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6252024-12-07T01:00:19.085ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6262024-12-07T01:00:19.086ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
6272024-12-07T01:00:19.087ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
6282024-12-07T01:00:19.087ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6292024-12-07T01:00:19.087ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6302024-12-07T01:00:19.089ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6312024-12-07T01:00:19.089ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6322024-12-07T01:00:19.090ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6332024-12-07T01:00:19.090ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6342024-12-07T01:00:19.091ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
6352024-12-07T01:00:19.092ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) } result = Ok(())
6362024-12-07T01:00:19.092ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6372024-12-07T01:00:19.093ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6382024-12-07T01:00:19.094ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6392024-12-07T01:00:19.094ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6402024-12-07T01:00:19.096ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6412024-12-07T01:00:19.096ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6422024-12-07T01:00:19.099ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6432024-12-07T01:00:19.099ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6442024-12-07T01:00:19.101ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6452024-12-07T01:00:19.101ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6462024-12-07T01:00:19.104ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6472024-12-07T01:00:19.104ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6482024-12-07T01:00:19.106ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6492024-12-07T01:00:19.106ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6502024-12-07T01:00:19.109ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6512024-12-07T01:00:19.111ZDEBGnexus (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ByName("instance-df6ebfd9906") } result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6522024-12-07T01:00:19.111ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6532024-12-07T01:00:19.111ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6542024-12-07T01:00:19.113ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6552024-12-07T01:00:19.113ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6562024-12-07T01:00:19.115ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6572024-12-07T01:00:19.115ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6582024-12-07T01:00:19.117ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6592024-12-07T01:00:19.118ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6602024-12-07T01:00:19.120ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6612024-12-07T01:00:19.123ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ByName("instance-df6ebfd9906") } result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6622024-12-07T01:00:19.123ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6632024-12-07T01:00:19.123ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6642024-12-07T01:00:19.125ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
6652024-12-07T01:00:19.126ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
6662024-12-07T01:00:19.129ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
6672024-12-07T01:00:19.130ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
6682024-12-07T01:00:19.131ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
6692024-12-07T01:00:19.132ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3, lookup_type: ById(e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3) } result = Ok(())
6702024-12-07T01:00:19.165ZDEBGnexus: client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vmms/cdff06af-548d-4d6e-8e27-5f181e4ed221/state
6712024-12-07T01:00:19.166ZDEBGnexus: client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/cdff06af-548d-4d6e-8e27-5f181e4ed221/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "83734b14-131f-49b7-a33a-6feffcc65ed3", "content-length": "151", "date": "Sat, 07 Dec 2024 01:00:19 GMT"} })
6722024-12-07T01:00:19.166ZINFOnexus (dropshot_external): received new VMM runtime state from sled agent actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true file = nexus/src/app/instance.rs:2116 local_addr = 172.30.2.5:443 method = POST migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = cdff06af-548d-4d6e-8e27-5f181e4ed221 remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-12-07T01:00:19.166216455Z }
6732024-12-07T01:00:19.166ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6742024-12-07T01:00:19.166ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6752024-12-07T01:00:19.208ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:159 retry_after (ms) = 2 transaction = vmm_and_migration_update_runtime
6762024-12-07T01:00:19.218ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:115 transaction = vmm_and_migration_update_runtime
6772024-12-07T01:00:19.220ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6782024-12-07T01:00:19.220ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6792024-12-07T01:00:19.222ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6802024-12-07T01:00:19.222ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6812024-12-07T01:00:19.225ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6822024-12-07T01:00:19.225ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6832024-12-07T01:00:19.227ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6842024-12-07T01:00:19.227ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6852024-12-07T01:00:19.230ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6862024-12-07T01:00:19.233ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ByName("instance-df6ebfd9906") } result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6872024-12-07T01:00:19.233ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6882024-12-07T01:00:19.233ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6892024-12-07T01:00:19.235ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 178955 local_addr = 172.30.2.5:443 method = POST remote_addr = 10.151.2.100:44326 req_id = c0e7634c-c560-4b70-805b-de05faefe8a1 response_code = 202 uri = /v1/instances/instance-df6ebfd9906/stop?project=proj-391b50e44b9
6902024-12-07T01:00:19.237ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6912024-12-07T01:00:19.237ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6922024-12-07T01:00:19.238ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6932024-12-07T01:00:19.239ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6942024-12-07T01:00:19.240ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
6952024-12-07T01:00:19.241ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("token = \\"309f87fb99ee10b5e9dbf0867a83b04371747aad\\"") } result = Ok(())
6962024-12-07T01:00:19.241ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6972024-12-07T01:00:19.241ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
6982024-12-07T01:00:19.243ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
6992024-12-07T01:00:19.243ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7002024-12-07T01:00:19.245ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7012024-12-07T01:00:19.245ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7022024-12-07T01:00:19.246ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7032024-12-07T01:00:19.246ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7042024-12-07T01:00:19.248ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
7052024-12-07T01:00:19.249ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
7062024-12-07T01:00:19.249ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7072024-12-07T01:00:19.249ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7082024-12-07T01:00:19.250ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
7092024-12-07T01:00:19.251ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("309f87fb99ee10b5e9dbf0867a83b04371747aad") } result = Ok(())
7102024-12-07T01:00:19.251ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7112024-12-07T01:00:19.251ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7122024-12-07T01:00:19.255ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7132024-12-07T01:00:19.256ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7142024-12-07T01:00:19.257ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7152024-12-07T01:00:19.257ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7162024-12-07T01:00:19.258ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7172024-12-07T01:00:19.258ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7182024-12-07T01:00:19.260ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7192024-12-07T01:00:19.260ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7202024-12-07T01:00:19.261ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
7212024-12-07T01:00:19.262ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
7222024-12-07T01:00:19.262ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7232024-12-07T01:00:19.262ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7242024-12-07T01:00:19.264ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7252024-12-07T01:00:19.264ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7262024-12-07T01:00:19.265ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
7272024-12-07T01:00:19.265ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
7282024-12-07T01:00:19.266ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
7292024-12-07T01:00:19.267ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) } result = Ok(())
7302024-12-07T01:00:19.267ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7312024-12-07T01:00:19.267ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7322024-12-07T01:00:19.269ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7332024-12-07T01:00:19.269ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7342024-12-07T01:00:19.270ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7352024-12-07T01:00:19.271ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7362024-12-07T01:00:19.272ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7372024-12-07T01:00:19.273ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7382024-12-07T01:00:19.275ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7392024-12-07T01:00:19.275ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7402024-12-07T01:00:19.277ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7412024-12-07T01:00:19.277ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7422024-12-07T01:00:19.279ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7432024-12-07T01:00:19.279ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7442024-12-07T01:00:19.281ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7452024-12-07T01:00:19.284ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ByName("instance-df6ebfd9906") } result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
7462024-12-07T01:00:19.284ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
7472024-12-07T01:00:19.284ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
7482024-12-07T01:00:19.285ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
7492024-12-07T01:00:19.286ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
7502024-12-07T01:00:19.286ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
7512024-12-07T01:00:19.286ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
7522024-12-07T01:00:19.290ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete
7532024-12-07T01:00:19.290ZINFOnexus (SEC): saga create dag = {"end_node":5,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[4,0,null],[3,5,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"instance_delete.instance_delete_nat","label":"InstanceDeleteNat","name":"no_result4"}},{"Action":{"action_name":"instance_delete.instance_delete_record","label":"InstanceDeleteRecord","name":"no_result1"}},{"Action":{"action_name":"instance_delete.delete_network_interfaces","label":"DeleteNetworkInterfaces","name":"no_result2"}},{"Action":{"action_name":"instance_delete.deallocate_external_ip","label":"DeallocateExternalIp","name":"no_result3"}},{"Start":{"params":{"authz_instance":{"key":"d4e357bc-806f-4b64-837e-f3433fef287e","lookup_type":{"ByName":"instance-df6ebfd9906"},"parent":{"key":"5f29c645-1da0-4c9a-9e97-85eb8bea9c4c","lookup_type":{"ByName":"proj-391b50e44b9"},"parent":{"key":"3a67a720-d4af-42e7-875a-3abed4955db5","lookup_type":{"ById":"3a67a720-d4af-42e7-875a-3abed4955db5"},"parent":null}}},"boundary_switches":["switch0"],"instance":{"auto_restart":{"cooldown":null,"policy":null},"boot_disk_id":"eac94d6b-8176-41fd-8617-455c5f662336","hostname":"localshark","identity":{"description":"","id":"d4e357bc-806f-4b64-837e-f3433fef287e","name":"instance-df6ebfd9906","time_created":"2024-12-07T00:59:32.823938Z","time_deleted":null,"time_modified":"2024-12-07T00:59:32.823938Z"},"memory":1073741824,"ncpus":2,"project_id":"5f29c645-1da0-4c9a-9e97-85eb8bea9c4c","runtime_state":{"dst_propolis_id":null,"gen":3,"migration_id":null,"nexus_state":"Vmm","propolis_id":"cdff06af-548d-4d6e-8e27-5f181e4ed221","time_last_auto_restarted":null,"time_updated":"2024-12-07T00:59:32.823938Z"},"updater_gen":1,"updater_id":null,"user_data":[]},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"3a67a720-d4af-42e7-875a-3abed4955db5","silo_user_id":"86a1f607-6bb4-4083-b5aa-102b2d1b5a27"}}},{"mapped_fleet_roles":{"admin":["admin"]}}]}}}}},"End"]},"saga_name":"instance-delete","start_node":4} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
7542024-12-07T01:00:19.290ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete
7552024-12-07T01:00:19.321ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete
7562024-12-07T01:00:19.321ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
7572024-12-07T01:00:19.321ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
7582024-12-07T01:00:19.321ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f
7592024-12-07T01:00:19.325ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
7602024-12-07T01:00:19.325ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f
7612024-12-07T01:00:19.329ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
7622024-12-07T01:00:19.329ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f
7632024-12-07T01:00:19.335ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7642024-12-07T01:00:19.336ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7652024-12-07T01:00:19.338ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7662024-12-07T01:00:19.338ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7672024-12-07T01:00:19.339ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7682024-12-07T01:00:19.339ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7692024-12-07T01:00:19.341ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7702024-12-07T01:00:19.341ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7712024-12-07T01:00:19.343ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7722024-12-07T01:00:19.343ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7732024-12-07T01:00:19.345ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7742024-12-07T01:00:19.345ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7752024-12-07T01:00:19.348ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7762024-12-07T01:00:19.348ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7772024-12-07T01:00:19.350ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7782024-12-07T01:00:19.353ZDEBGnexus (SagaExecutor): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ById(5f29c645-1da0-4c9a-9e97-85eb8bea9c4c) }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ById(d4e357bc-806f-4b64-837e-f3433fef287e) } result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7792024-12-07T01:00:19.353ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = d4e357bc-806f-4b64-837e-f3433fef287e
7802024-12-07T01:00:19.353ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7812024-12-07T01:00:19.353ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7822024-12-07T01:00:19.356ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7832024-12-07T01:00:19.356ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7842024-12-07T01:00:19.373ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7852024-12-07T01:00:19.373ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteNat
7862024-12-07T01:00:19.380ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
7872024-12-07T01:00:19.380ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
7882024-12-07T01:00:19.381ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
7892024-12-07T01:00:19.383ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
7902024-12-07T01:00:19.383ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
7912024-12-07T01:00:19.383ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
7922024-12-07T01:00:19.388ZDEBGnexus: notifying dendrite of updates instance_id = Some(d4e357bc-806f-4b64-837e-f3433fef287e (instance)) switch = switch0
7932024-12-07T01:00:19.388ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4482, tv_nsec: 613848046 } })
7942024-12-07T01:00:19.389ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1103
7952024-12-07T01:00:19.420ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
7962024-12-07T01:00:19.420ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7972024-12-07T01:00:19.420ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "782ab6f9-bbc6-4662-a1e3-810c1355ce68", "content-length": "26", "date": "Sat, 07 Dec 2024 01:00:19 GMT"} })
7982024-12-07T01:00:19.420ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7992024-12-07T01:00:19.421ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8002024-12-07T01:00:19.453ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
8012024-12-07T01:00:19.453ZDEBGnexus (DpdClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5466cf1-5670-4679-9e3a-92cc738eea17", "content-length": "4", "date": "Sat, 07 Dec 2024 01:00:18 GMT"} })
8022024-12-07T01:00:19.454ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
8032024-12-07T01:00:19.454ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 0 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f
8042024-12-07T01:00:19.457ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
8052024-12-07T01:00:19.457ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f
8062024-12-07T01:00:19.461ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteRecord
8072024-12-07T01:00:19.462ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteRecord
8082024-12-07T01:00:19.464ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteRecord
8092024-12-07T01:00:19.464ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteRecord
8102024-12-07T01:00:19.467ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteRecord
8112024-12-07T01:00:19.467ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteRecord
8122024-12-07T01:00:19.469ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteRecord
8132024-12-07T01:00:19.469ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteRecord
8142024-12-07T01:00:19.472ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteRecord
8152024-12-07T01:00:19.475ZDEBGnexus (SagaExecutor): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ByName("instance-df6ebfd9906") } result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteRecord
8162024-12-07T01:00:19.475ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteRecord
8172024-12-07T01:00:19.475ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_node = InstanceDeleteRecord
8182024-12-07T01:00:19.519ZDEBGnexus (SEC): saga log event new_state = N001 failed sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
8192024-12-07T01:00:19.519ZDEBGnexus (SecStore): recording saga event event_type = Failed(ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} }) node_id = 1 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f
8202024-12-07T01:00:19.526ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Unwinding saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
8212024-12-07T01:00:19.526ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = unwinding saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f
8222024-12-07T01:00:19.561ZDEBGnexus (SEC): saga log event new_state = N000 undo_started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
8232024-12-07T01:00:19.561ZDEBGnexus (SecStore): recording saga event event_type = UndoStarted node_id = 0 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f
8242024-12-07T01:00:19.565ZDEBGnexus (SEC): saga log event new_state = N000 undo_finished sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
8252024-12-07T01:00:19.565ZDEBGnexus (SecStore): recording saga event event_type = UndoFinished node_id = 0 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f
8262024-12-07T01:00:19.569ZDEBGnexus (SEC): saga log event new_state = N004 undo_started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
8272024-12-07T01:00:19.569ZDEBGnexus (SecStore): recording saga event event_type = UndoStarted node_id = 4 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f
8282024-12-07T01:00:19.573ZDEBGnexus (SEC): saga log event new_state = N004 undo_finished sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
8292024-12-07T01:00:19.573ZDEBGnexus (SecStore): recording saga event event_type = UndoFinished node_id = 4 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f
8302024-12-07T01:00:19.577ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
8312024-12-07T01:00:19.577ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f
8322024-12-07T01:00:19.584ZWARNnexus (SEC): saga finished action_error_node_name = "no_result1" action_error_source = ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} } file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1044 result = failure saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633 undo_result = success
8332024-12-07T01:00:19.584ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 100f21e3-6f63-4f58-b804-c983fd0c719f saga_name = instance-delete saga_result = SagaResult { saga_id: 100f21e3-6f63-4f58-b804-c983fd0c719f, saga_log: SagaLog { saga_id: 100f21e3-6f63-4f58-b804-c983fd0c719f, unwinding: true, events: [N004 started, N004 succeeded, N000 started, N000 succeeded, N001 started, N001 failed, N000 undo_started, N000 undo_finished, N004 undo_started, N004 undo_finished], node_status: {0: UndoFinished, 1: Failed(ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} }), 4: UndoFinished} }, kind: Err(SagaResultErr { error_node_name: "no_result1", error_source: ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} }, undo_failure: None }) }
8342024-12-07T01:00:19.584ZINFOnexus (dropshot_external): request completed error_message_external = cannot delete instance: instance is running or has not yet fully stopped error_message_internal = cannot delete instance: instance is running or has not yet fully stopped (with internal context: saga ACTION error at node "no_result1") file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 347268 local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 0e95cca1-e3cb-4b2a-a99f-b6e023ac8582 response_code = 400 uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8352024-12-07T01:00:20.588ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8362024-12-07T01:00:20.588ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8372024-12-07T01:00:20.590ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8382024-12-07T01:00:20.590ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8392024-12-07T01:00:20.592ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
8402024-12-07T01:00:20.593ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("token = \\"309f87fb99ee10b5e9dbf0867a83b04371747aad\\"") } result = Ok(())
8412024-12-07T01:00:20.593ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8422024-12-07T01:00:20.593ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8432024-12-07T01:00:20.595ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8442024-12-07T01:00:20.595ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8452024-12-07T01:00:20.597ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8462024-12-07T01:00:20.597ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8472024-12-07T01:00:20.599ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8482024-12-07T01:00:20.599ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8492024-12-07T01:00:20.600ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
8502024-12-07T01:00:20.601ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
8512024-12-07T01:00:20.601ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8522024-12-07T01:00:20.602ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8532024-12-07T01:00:20.603ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
8542024-12-07T01:00:20.604ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("309f87fb99ee10b5e9dbf0867a83b04371747aad") } result = Ok(())
8552024-12-07T01:00:20.604ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8562024-12-07T01:00:20.604ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8572024-12-07T01:00:20.608ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8582024-12-07T01:00:20.609ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8592024-12-07T01:00:20.610ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8602024-12-07T01:00:20.610ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8612024-12-07T01:00:20.612ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8622024-12-07T01:00:20.612ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8632024-12-07T01:00:20.613ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8642024-12-07T01:00:20.613ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8652024-12-07T01:00:20.615ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
8662024-12-07T01:00:20.616ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
8672024-12-07T01:00:20.616ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8682024-12-07T01:00:20.616ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8692024-12-07T01:00:20.618ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8702024-12-07T01:00:20.618ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8712024-12-07T01:00:20.619ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
8722024-12-07T01:00:20.619ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
8732024-12-07T01:00:20.620ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
8742024-12-07T01:00:20.621ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) } result = Ok(())
8752024-12-07T01:00:20.621ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8762024-12-07T01:00:20.622ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8772024-12-07T01:00:20.623ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8782024-12-07T01:00:20.623ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8792024-12-07T01:00:20.624ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8802024-12-07T01:00:20.624ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8812024-12-07T01:00:20.627ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8822024-12-07T01:00:20.627ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8832024-12-07T01:00:20.628ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8842024-12-07T01:00:20.628ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8852024-12-07T01:00:20.631ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8862024-12-07T01:00:20.631ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8872024-12-07T01:00:20.633ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8882024-12-07T01:00:20.634ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8892024-12-07T01:00:20.636ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8902024-12-07T01:00:20.638ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ByName("instance-df6ebfd9906") } result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
8912024-12-07T01:00:20.638ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
8922024-12-07T01:00:20.639ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
8932024-12-07T01:00:20.640ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
8942024-12-07T01:00:20.641ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
8952024-12-07T01:00:20.641ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
8962024-12-07T01:00:20.641ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
8972024-12-07T01:00:20.646ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete
8982024-12-07T01:00:20.646ZINFOnexus (SEC): saga create dag = {"end_node":5,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[4,0,null],[3,5,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"instance_delete.instance_delete_nat","label":"InstanceDeleteNat","name":"no_result4"}},{"Action":{"action_name":"instance_delete.instance_delete_record","label":"InstanceDeleteRecord","name":"no_result1"}},{"Action":{"action_name":"instance_delete.delete_network_interfaces","label":"DeleteNetworkInterfaces","name":"no_result2"}},{"Action":{"action_name":"instance_delete.deallocate_external_ip","label":"DeallocateExternalIp","name":"no_result3"}},{"Start":{"params":{"authz_instance":{"key":"d4e357bc-806f-4b64-837e-f3433fef287e","lookup_type":{"ByName":"instance-df6ebfd9906"},"parent":{"key":"5f29c645-1da0-4c9a-9e97-85eb8bea9c4c","lookup_type":{"ByName":"proj-391b50e44b9"},"parent":{"key":"3a67a720-d4af-42e7-875a-3abed4955db5","lookup_type":{"ById":"3a67a720-d4af-42e7-875a-3abed4955db5"},"parent":null}}},"boundary_switches":["switch0"],"instance":{"auto_restart":{"cooldown":null,"policy":null},"boot_disk_id":"eac94d6b-8176-41fd-8617-455c5f662336","hostname":"localshark","identity":{"description":"","id":"d4e357bc-806f-4b64-837e-f3433fef287e","name":"instance-df6ebfd9906","time_created":"2024-12-07T00:59:32.823938Z","time_deleted":null,"time_modified":"2024-12-07T00:59:32.823938Z"},"memory":1073741824,"ncpus":2,"project_id":"5f29c645-1da0-4c9a-9e97-85eb8bea9c4c","runtime_state":{"dst_propolis_id":null,"gen":3,"migration_id":null,"nexus_state":"Vmm","propolis_id":"cdff06af-548d-4d6e-8e27-5f181e4ed221","time_last_auto_restarted":null,"time_updated":"2024-12-07T00:59:32.823938Z"},"updater_gen":1,"updater_id":null,"user_data":[]},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"3a67a720-d4af-42e7-875a-3abed4955db5","silo_user_id":"86a1f607-6bb4-4083-b5aa-102b2d1b5a27"}}},{"mapped_fleet_roles":{"admin":["admin"]}}]}}}}},"End"]},"saga_name":"instance-delete","start_node":4} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
8992024-12-07T01:00:20.646ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete
9002024-12-07T01:00:20.650ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete
9012024-12-07T01:00:20.650ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
9022024-12-07T01:00:20.650ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
9032024-12-07T01:00:20.650ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41
9042024-12-07T01:00:20.654ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
9052024-12-07T01:00:20.654ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41
9062024-12-07T01:00:20.658ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
9072024-12-07T01:00:20.658ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41
9082024-12-07T01:00:20.661ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9092024-12-07T01:00:20.662ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9102024-12-07T01:00:20.663ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9112024-12-07T01:00:20.664ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9122024-12-07T01:00:20.665ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9132024-12-07T01:00:20.665ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9142024-12-07T01:00:20.666ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9152024-12-07T01:00:20.667ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9162024-12-07T01:00:20.669ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9172024-12-07T01:00:20.669ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9182024-12-07T01:00:20.671ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9192024-12-07T01:00:20.671ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9202024-12-07T01:00:20.673ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9212024-12-07T01:00:20.673ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9222024-12-07T01:00:20.676ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9232024-12-07T01:00:20.678ZDEBGnexus (SagaExecutor): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ById(5f29c645-1da0-4c9a-9e97-85eb8bea9c4c) }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ById(d4e357bc-806f-4b64-837e-f3433fef287e) } result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9242024-12-07T01:00:20.678ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = d4e357bc-806f-4b64-837e-f3433fef287e
9252024-12-07T01:00:20.678ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9262024-12-07T01:00:20.679ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9272024-12-07T01:00:20.680ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9282024-12-07T01:00:20.681ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9292024-12-07T01:00:20.682ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
9302024-12-07T01:00:20.682ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9312024-12-07T01:00:20.683ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteNat
9322024-12-07T01:00:20.684ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
9332024-12-07T01:00:20.684ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
9342024-12-07T01:00:20.684ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
9352024-12-07T01:00:20.685ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
9362024-12-07T01:00:20.686ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
9372024-12-07T01:00:20.686ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
9382024-12-07T01:00:20.686ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
9392024-12-07T01:00:20.689ZDEBGnexus: notifying dendrite of updates instance_id = Some(d4e357bc-806f-4b64-837e-f3433fef287e (instance)) switch = switch0
9402024-12-07T01:00:20.690ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4483, tv_nsec: 915436602 } })
9412024-12-07T01:00:20.690ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1103
9422024-12-07T01:00:20.721ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
9432024-12-07T01:00:20.721ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9442024-12-07T01:00:20.721ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2ec87b7-9dde-4ae6-9c21-a53a714919f8", "content-length": "26", "date": "Sat, 07 Dec 2024 01:00:20 GMT"} })
9452024-12-07T01:00:20.721ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9462024-12-07T01:00:20.722ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9472024-12-07T01:00:20.751ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
9482024-12-07T01:00:20.751ZDEBGnexus (DpdClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ed2d359-0e6c-46a3-8863-0468f7efabc4", "content-length": "4", "date": "Sat, 07 Dec 2024 01:00:20 GMT"} })
9492024-12-07T01:00:20.752ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
9502024-12-07T01:00:20.752ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 0 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41
9512024-12-07T01:00:20.756ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
9522024-12-07T01:00:20.756ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41
9532024-12-07T01:00:20.759ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteRecord
9542024-12-07T01:00:20.760ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteRecord
9552024-12-07T01:00:20.762ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteRecord
9562024-12-07T01:00:20.762ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteRecord
9572024-12-07T01:00:20.763ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteRecord
9582024-12-07T01:00:20.763ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteRecord
9592024-12-07T01:00:20.765ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteRecord
9602024-12-07T01:00:20.766ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteRecord
9612024-12-07T01:00:20.768ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteRecord
9622024-12-07T01:00:20.770ZDEBGnexus (SagaExecutor): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ByName("instance-df6ebfd9906") } result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteRecord
9632024-12-07T01:00:20.770ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteRecord
9642024-12-07T01:00:20.771ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_node = InstanceDeleteRecord
9652024-12-07T01:00:20.862ZDEBGnexus (SEC): saga log event new_state = N001 failed sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
9662024-12-07T01:00:20.862ZDEBGnexus (SecStore): recording saga event event_type = Failed(ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} }) node_id = 1 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41
9672024-12-07T01:00:20.866ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Unwinding saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
9682024-12-07T01:00:20.866ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = unwinding saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41
9692024-12-07T01:00:20.899ZDEBGnexus (SEC): saga log event new_state = N000 undo_started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
9702024-12-07T01:00:20.899ZDEBGnexus (SecStore): recording saga event event_type = UndoStarted node_id = 0 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41
9712024-12-07T01:00:20.903ZDEBGnexus (SEC): saga log event new_state = N000 undo_finished sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
9722024-12-07T01:00:20.903ZDEBGnexus (SecStore): recording saga event event_type = UndoFinished node_id = 0 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41
9732024-12-07T01:00:20.905ZDEBGnexus (SEC): saga log event new_state = N004 undo_started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
9742024-12-07T01:00:20.905ZDEBGnexus (SecStore): recording saga event event_type = UndoStarted node_id = 4 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41
9752024-12-07T01:00:20.909ZDEBGnexus (SEC): saga log event new_state = N004 undo_finished sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
9762024-12-07T01:00:20.909ZDEBGnexus (SecStore): recording saga event event_type = UndoFinished node_id = 4 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41
9772024-12-07T01:00:20.912ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
9782024-12-07T01:00:20.912ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41
9792024-12-07T01:00:20.919ZWARNnexus (SEC): saga finished action_error_node_name = "no_result1" action_error_source = ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} } file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1044 result = failure saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633 undo_result = success
9802024-12-07T01:00:20.919ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 0df209d0-4cea-4988-9240-f8c7509b9e41 saga_name = instance-delete saga_result = SagaResult { saga_id: 0df209d0-4cea-4988-9240-f8c7509b9e41, saga_log: SagaLog { saga_id: 0df209d0-4cea-4988-9240-f8c7509b9e41, unwinding: true, events: [N004 started, N004 succeeded, N000 started, N000 succeeded, N001 started, N001 failed, N000 undo_started, N000 undo_finished, N004 undo_started, N004 undo_finished], node_status: {0: UndoFinished, 1: Failed(ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} }), 4: UndoFinished} }, kind: Err(SagaResultErr { error_node_name: "no_result1", error_source: ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("cannot delete instance: instance is running or has not yet fully stopped"), "internal_context": String("")}}} }, undo_failure: None }) }
9812024-12-07T01:00:20.919ZINFOnexus (dropshot_external): request completed error_message_external = cannot delete instance: instance is running or has not yet fully stopped error_message_internal = cannot delete instance: instance is running or has not yet fully stopped (with internal context: saga ACTION error at node "no_result1") file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 331520 local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 67a9298a-6fc6-46ba-a619-7d508c3a3447 response_code = 400 uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
9822024-12-07T01:00:21.922ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9832024-12-07T01:00:21.922ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9842024-12-07T01:00:21.923ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9852024-12-07T01:00:21.923ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9862024-12-07T01:00:21.925ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
9872024-12-07T01:00:21.926ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("token = \\"309f87fb99ee10b5e9dbf0867a83b04371747aad\\"") } result = Ok(())
9882024-12-07T01:00:21.926ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9892024-12-07T01:00:21.926ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9902024-12-07T01:00:21.927ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9912024-12-07T01:00:21.927ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9922024-12-07T01:00:21.928ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9932024-12-07T01:00:21.929ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9942024-12-07T01:00:21.930ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9952024-12-07T01:00:21.930ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
9962024-12-07T01:00:21.931ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
9972024-12-07T01:00:21.932ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
9982024-12-07T01:00:21.932ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
9992024-12-07T01:00:21.933ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10002024-12-07T01:00:21.934ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
10012024-12-07T01:00:21.935ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("309f87fb99ee10b5e9dbf0867a83b04371747aad") } result = Ok(())
10022024-12-07T01:00:21.935ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10032024-12-07T01:00:21.935ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10042024-12-07T01:00:21.940ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10052024-12-07T01:00:21.940ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10062024-12-07T01:00:21.942ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10072024-12-07T01:00:21.942ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10082024-12-07T01:00:21.943ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10092024-12-07T01:00:21.943ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10102024-12-07T01:00:21.945ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10112024-12-07T01:00:21.945ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10122024-12-07T01:00:21.946ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
10132024-12-07T01:00:21.947ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
10142024-12-07T01:00:21.947ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10152024-12-07T01:00:21.947ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10162024-12-07T01:00:21.949ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10172024-12-07T01:00:21.949ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10182024-12-07T01:00:21.950ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
10192024-12-07T01:00:21.950ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
10202024-12-07T01:00:21.951ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
10212024-12-07T01:00:21.953ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) } result = Ok(())
10222024-12-07T01:00:21.953ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10232024-12-07T01:00:21.953ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10242024-12-07T01:00:21.954ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10252024-12-07T01:00:21.954ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10262024-12-07T01:00:21.956ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10272024-12-07T01:00:21.957ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10282024-12-07T01:00:21.958ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10292024-12-07T01:00:21.958ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10302024-12-07T01:00:21.961ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10312024-12-07T01:00:21.961ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10322024-12-07T01:00:21.963ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10332024-12-07T01:00:21.963ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10342024-12-07T01:00:21.965ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 roles = RoleSet { roles: {} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10352024-12-07T01:00:21.965ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 resource = Database result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10362024-12-07T01:00:21.968ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10372024-12-07T01:00:21.970ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ByName("instance-df6ebfd9906") } result = Ok(()) uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
10382024-12-07T01:00:21.970ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
10392024-12-07T01:00:21.971ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
10402024-12-07T01:00:21.972ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
10412024-12-07T01:00:21.973ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
10422024-12-07T01:00:21.973ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
10432024-12-07T01:00:21.973ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
10442024-12-07T01:00:21.977ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete
10452024-12-07T01:00:21.977ZINFOnexus (SEC): saga create dag = {"end_node":5,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[4,0,null],[3,5,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"instance_delete.instance_delete_nat","label":"InstanceDeleteNat","name":"no_result4"}},{"Action":{"action_name":"instance_delete.instance_delete_record","label":"InstanceDeleteRecord","name":"no_result1"}},{"Action":{"action_name":"instance_delete.delete_network_interfaces","label":"DeleteNetworkInterfaces","name":"no_result2"}},{"Action":{"action_name":"instance_delete.deallocate_external_ip","label":"DeallocateExternalIp","name":"no_result3"}},{"Start":{"params":{"authz_instance":{"key":"d4e357bc-806f-4b64-837e-f3433fef287e","lookup_type":{"ByName":"instance-df6ebfd9906"},"parent":{"key":"5f29c645-1da0-4c9a-9e97-85eb8bea9c4c","lookup_type":{"ByName":"proj-391b50e44b9"},"parent":{"key":"3a67a720-d4af-42e7-875a-3abed4955db5","lookup_type":{"ById":"3a67a720-d4af-42e7-875a-3abed4955db5"},"parent":null}}},"boundary_switches":["switch0"],"instance":{"auto_restart":{"cooldown":null,"policy":null},"boot_disk_id":"eac94d6b-8176-41fd-8617-455c5f662336","hostname":"localshark","identity":{"description":"","id":"d4e357bc-806f-4b64-837e-f3433fef287e","name":"instance-df6ebfd9906","time_created":"2024-12-07T00:59:32.823938Z","time_deleted":null,"time_modified":"2024-12-07T00:59:32.823938Z"},"memory":1073741824,"ncpus":2,"project_id":"5f29c645-1da0-4c9a-9e97-85eb8bea9c4c","runtime_state":{"dst_propolis_id":null,"gen":3,"migration_id":null,"nexus_state":"Vmm","propolis_id":"cdff06af-548d-4d6e-8e27-5f181e4ed221","time_last_auto_restarted":null,"time_updated":"2024-12-07T00:59:32.823938Z"},"updater_gen":3,"updater_id":"c8bd7767-6848-4e88-b532-ad5850dc4a5e","user_data":[]},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"3a67a720-d4af-42e7-875a-3abed4955db5","silo_user_id":"86a1f607-6bb4-4083-b5aa-102b2d1b5a27"}}},{"mapped_fleet_roles":{"admin":["admin"]}}]}}}}},"End"]},"saga_name":"instance-delete","start_node":4} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
10462024-12-07T01:00:21.977ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete
10472024-12-07T01:00:21.981ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete
10482024-12-07T01:00:21.981ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
10492024-12-07T01:00:21.981ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
10502024-12-07T01:00:21.981ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19
10512024-12-07T01:00:21.984ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
10522024-12-07T01:00:21.984ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19
10532024-12-07T01:00:21.987ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
10542024-12-07T01:00:21.987ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19
10552024-12-07T01:00:21.990ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10562024-12-07T01:00:21.990ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10572024-12-07T01:00:21.992ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10582024-12-07T01:00:21.992ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10592024-12-07T01:00:21.994ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10602024-12-07T01:00:21.994ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10612024-12-07T01:00:21.995ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10622024-12-07T01:00:21.995ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10632024-12-07T01:00:21.998ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10642024-12-07T01:00:21.998ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10652024-12-07T01:00:22.000ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10662024-12-07T01:00:22.000ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10672024-12-07T01:00:22.002ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10682024-12-07T01:00:22.003ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10692024-12-07T01:00:22.005ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10702024-12-07T01:00:22.008ZDEBGnexus (SagaExecutor): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ById(5f29c645-1da0-4c9a-9e97-85eb8bea9c4c) }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ById(d4e357bc-806f-4b64-837e-f3433fef287e) } result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10712024-12-07T01:00:22.008ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = d4e357bc-806f-4b64-837e-f3433fef287e
10722024-12-07T01:00:22.008ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10732024-12-07T01:00:22.008ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10742024-12-07T01:00:22.010ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10752024-12-07T01:00:22.010ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10762024-12-07T01:00:22.013ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
10772024-12-07T01:00:22.013ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10782024-12-07T01:00:22.013ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteNat
10792024-12-07T01:00:22.015ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
10802024-12-07T01:00:22.015ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
10812024-12-07T01:00:22.015ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
10822024-12-07T01:00:22.016ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
10832024-12-07T01:00:22.017ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
10842024-12-07T01:00:22.017ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
10852024-12-07T01:00:22.017ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
10862024-12-07T01:00:22.021ZDEBGnexus: notifying dendrite of updates instance_id = Some(d4e357bc-806f-4b64-837e-f3433fef287e (instance)) switch = switch0
10872024-12-07T01:00:22.022ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4485, tv_nsec: 247564449 } })
10882024-12-07T01:00:22.022ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1103
10892024-12-07T01:00:22.055ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
10902024-12-07T01:00:22.055ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10912024-12-07T01:00:22.056ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4c248d6-db9c-4944-8a1f-8742cdf6928a", "content-length": "26", "date": "Sat, 07 Dec 2024 01:00:22 GMT"} })
10922024-12-07T01:00:22.056ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10932024-12-07T01:00:22.057ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
10942024-12-07T01:00:22.088ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
10952024-12-07T01:00:22.088ZDEBGnexus (DpdClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger", status: 200, headers: {"content-type": "application/json", "x-request-id": "77d40faf-b81c-45c3-b9f4-152def5ecb8d", "content-length": "4", "date": "Sat, 07 Dec 2024 01:00:21 GMT"} })
10962024-12-07T01:00:22.089ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
10972024-12-07T01:00:22.089ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 0 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19
10982024-12-07T01:00:22.093ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
10992024-12-07T01:00:22.093ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19
11002024-12-07T01:00:22.097ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11012024-12-07T01:00:22.097ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11022024-12-07T01:00:22.100ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11032024-12-07T01:00:22.100ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11042024-12-07T01:00:22.102ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11052024-12-07T01:00:22.102ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11062024-12-07T01:00:22.104ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11072024-12-07T01:00:22.104ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11082024-12-07T01:00:22.106ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11092024-12-07T01:00:22.109ZDEBGnexus (SagaExecutor): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ByName("instance-df6ebfd9906") } result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11102024-12-07T01:00:22.109ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11112024-12-07T01:00:22.109ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11122024-12-07T01:00:22.221ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11132024-12-07T01:00:22.221ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11142024-12-07T01:00:22.283ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11152024-12-07T01:00:22.283ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = InstanceDeleteRecord
11162024-12-07T01:00:22.285ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
11172024-12-07T01:00:22.285ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 1 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19
11182024-12-07T01:00:22.289ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
11192024-12-07T01:00:22.289ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19
11202024-12-07T01:00:22.293ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeleteNetworkInterfaces
11212024-12-07T01:00:22.293ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeleteNetworkInterfaces
11222024-12-07T01:00:22.295ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeleteNetworkInterfaces
11232024-12-07T01:00:22.295ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeleteNetworkInterfaces
11242024-12-07T01:00:22.298ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeleteNetworkInterfaces
11252024-12-07T01:00:22.298ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeleteNetworkInterfaces
11262024-12-07T01:00:22.301ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeleteNetworkInterfaces
11272024-12-07T01:00:22.301ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeleteNetworkInterfaces
11282024-12-07T01:00:22.302ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeleteNetworkInterfaces
11292024-12-07T01:00:22.305ZDEBGnexus (SagaExecutor): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: d4e357bc-806f-4b64-837e-f3433fef287e, lookup_type: ByName("instance-df6ebfd9906") } result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeleteNetworkInterfaces
11302024-12-07T01:00:22.305ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeleteNetworkInterfaces
11312024-12-07T01:00:22.305ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeleteNetworkInterfaces
11322024-12-07T01:00:22.339ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
11332024-12-07T01:00:22.339ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19
11342024-12-07T01:00:22.339ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Signaled
11352024-12-07T01:00:22.339ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11362024-12-07T01:00:22.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11372024-12-07T01:00:22.343ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
11382024-12-07T01:00:22.343ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19
11392024-12-07T01:00:22.347ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeallocateExternalIp
11402024-12-07T01:00:22.348ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeallocateExternalIp
11412024-12-07T01:00:22.402ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11422024-12-07T01:00:22.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11432024-12-07T01:00:22.412ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11442024-12-07T01:00:22.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11452024-12-07T01:00:22.414ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11462024-12-07T01:00:22.415ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11472024-12-07T01:00:22.415ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11482024-12-07T01:00:22.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11492024-12-07T01:00:22.417ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11502024-12-07T01:00:22.418ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11512024-12-07T01:00:22.418ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11522024-12-07T01:00:22.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11532024-12-07T01:00:22.425ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeallocateExternalIp
11542024-12-07T01:00:22.425ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_node = DeallocateExternalIp
11552024-12-07T01:00:22.504ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
11562024-12-07T01:00:22.505ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ddb591b-575d-47ea-bb46-43d1b9428921", "content-length": "691", "date": "Sat, 07 Dec 2024 01:00:22 GMT"} })
11572024-12-07T01:00:22.505ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
11582024-12-07T01:00:22.505ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
11592024-12-07T01:00:22.505ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
11602024-12-07T01:00:22.506ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 167.325139ms iteration = 27 status = {}
11612024-12-07T01:00:22.508ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
11622024-12-07T01:00:22.508ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19
11632024-12-07T01:00:22.513ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
11642024-12-07T01:00:22.513ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19
11652024-12-07T01:00:22.516ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
11662024-12-07T01:00:22.516ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19
11672024-12-07T01:00:22.520ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
11682024-12-07T01:00:22.520ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19
11692024-12-07T01:00:22.553ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
11702024-12-07T01:00:22.553ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = a4ba9c71-9523-4900-9435-a4f78f403d19 saga_name = instance-delete saga_result = SagaResult { saga_id: a4ba9c71-9523-4900-9435-a4f78f403d19, saga_log: SagaLog { saga_id: a4ba9c71-9523-4900-9435-a4f78f403d19, unwinding: false, events: [N004 started, N004 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N005 started, N005 succeeded], node_status: {0: Succeeded(Null), 1: Succeeded(Null), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"no_result1": Null, "no_result2": Null, "no_result3": Null, "no_result4": Null} }) }
11712024-12-07T01:00:22.553ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Signaled
11722024-12-07T01:00:22.553ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
11732024-12-07T01:00:22.553ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 631936 local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = f6334d34-ffae-434f-bf98-20b4f79387e6 response_code = 204 uri = /v1/instances/instance-df6ebfd9906?project=proj-391b50e44b9
11742024-12-07T01:00:22.553ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11752024-12-07T01:00:22.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11762024-12-07T01:00:22.555ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11772024-12-07T01:00:22.556ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11782024-12-07T01:00:22.556ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11792024-12-07T01:00:22.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11802024-12-07T01:00:22.557ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
11812024-12-07T01:00:22.558ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
11822024-12-07T01:00:22.558ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11832024-12-07T01:00:22.558ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
11842024-12-07T01:00:22.559ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
11852024-12-07T01:00:22.559ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11862024-12-07T01:00:22.559ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11872024-12-07T01:00:22.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11882024-12-07T01:00:22.560ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
11892024-12-07T01:00:22.561ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("token = \\"309f87fb99ee10b5e9dbf0867a83b04371747aad\\"") } result = Ok(())
11902024-12-07T01:00:22.561ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
11912024-12-07T01:00:22.562ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
11922024-12-07T01:00:22.563ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
11932024-12-07T01:00:22.563ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
11942024-12-07T01:00:22.564ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
11952024-12-07T01:00:22.564ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
11962024-12-07T01:00:22.565ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
11972024-12-07T01:00:22.566ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
11982024-12-07T01:00:22.567ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
11992024-12-07T01:00:22.568ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
12002024-12-07T01:00:22.568ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12012024-12-07T01:00:22.568ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12022024-12-07T01:00:22.592ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
12032024-12-07T01:00:22.592ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
12042024-12-07T01:00:22.592ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "47160368-f691-414c-a4e5-57103edefd04", "content-length": "372", "date": "Sat, 07 Dec 2024 01:00:22 GMT"} })
12052024-12-07T01:00:22.592ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12062024-12-07T01:00:22.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12072024-12-07T01:00:22.619ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
12082024-12-07T01:00:22.620ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("309f87fb99ee10b5e9dbf0867a83b04371747aad") } result = Ok(())
12092024-12-07T01:00:22.620ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12102024-12-07T01:00:22.620ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12112024-12-07T01:00:22.624ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12122024-12-07T01:00:22.624ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12132024-12-07T01:00:22.626ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12142024-12-07T01:00:22.626ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12152024-12-07T01:00:22.627ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12162024-12-07T01:00:22.628ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12172024-12-07T01:00:22.629ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12182024-12-07T01:00:22.629ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12192024-12-07T01:00:22.630ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
12202024-12-07T01:00:22.631ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
12212024-12-07T01:00:22.632ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12222024-12-07T01:00:22.632ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12232024-12-07T01:00:22.633ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12242024-12-07T01:00:22.633ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12252024-12-07T01:00:22.634ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
12262024-12-07T01:00:22.634ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
12272024-12-07T01:00:22.634ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "da49d08b-29a3-4c31-85b4-d0fc5e9c8113", "date": "Sat, 07 Dec 2024 01:00:22 GMT"} })
12282024-12-07T01:00:22.634ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12292024-12-07T01:00:22.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12302024-12-07T01:00:22.634ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
12312024-12-07T01:00:22.635ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
12322024-12-07T01:00:22.636ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
12332024-12-07T01:00:22.636ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12342024-12-07T01:00:22.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12352024-12-07T01:00:22.637ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) } result = Ok(())
12362024-12-07T01:00:22.637ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12372024-12-07T01:00:22.637ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Database result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12382024-12-07T01:00:22.639ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12392024-12-07T01:00:22.639ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Database result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12402024-12-07T01:00:22.639ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12412024-12-07T01:00:22.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12422024-12-07T01:00:22.640ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12432024-12-07T01:00:22.641ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Database result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12442024-12-07T01:00:22.642ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
12452024-12-07T01:00:22.642ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
12462024-12-07T01:00:22.643ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8a8aebd9-c2f0-49f5-847b-52bd6943ef98", "date": "Sat, 07 Dec 2024 01:00:22 GMT"} })
12472024-12-07T01:00:22.643ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12482024-12-07T01:00:22.643ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Database result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12492024-12-07T01:00:22.644ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 90.510894ms iteration = 27 status = {}
12502024-12-07T01:00:22.645ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12512024-12-07T01:00:22.645ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Database result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12522024-12-07T01:00:22.647ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12532024-12-07T01:00:22.647ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Database result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12542024-12-07T01:00:22.650ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12552024-12-07T01:00:22.650ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Database result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12562024-12-07T01:00:22.652ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12572024-12-07T01:00:22.655ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Disk { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: eac94d6b-8176-41fd-8617-455c5f662336, lookup_type: ByName("disk-ea50e9a0169") } result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12582024-12-07T01:00:22.655ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12592024-12-07T01:00:22.655ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Database result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12602024-12-07T01:00:22.656ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12612024-12-07T01:00:22.657ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Database result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12622024-12-07T01:00:22.658ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12632024-12-07T01:00:22.658ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Database result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12642024-12-07T01:00:22.659ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12652024-12-07T01:00:22.660ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Database result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12662024-12-07T01:00:22.662ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12672024-12-07T01:00:22.662ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Database result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12682024-12-07T01:00:22.664ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12692024-12-07T01:00:22.664ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Database result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12702024-12-07T01:00:22.666ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12712024-12-07T01:00:22.667ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Database result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12722024-12-07T01:00:22.668ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12732024-12-07T01:00:22.671ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 resource = Disk { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ById(5f29c645-1da0-4c9a-9e97-85eb8bea9c4c) }, key: eac94d6b-8176-41fd-8617-455c5f662336, lookup_type: ById(eac94d6b-8176-41fd-8617-455c5f662336) } result = Ok(()) uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
12742024-12-07T01:00:22.671ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete
12752024-12-07T01:00:22.671ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[4,5,null],[4,6,null],[6,7,null],[5,7,null],[7,8,null],[8,9,null],[9,10,null],[10,11,null],[3,4,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"disk_delete.delete_disk_record","label":"DeleteDiskRecord","name":"deleted_disk"}},{"Action":{"action_name":"disk_delete.space_account","label":"SpaceAccount","name":"no_result1"}},{"Constant":{"name":"params_for_volume_delete_subsaga","value":{"serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"3a67a720-d4af-42e7-875a-3abed4955db5","silo_user_id":"86a1f607-6bb4-4083-b5aa-102b2d1b5a27"}}},{"mapped_fleet_roles":{"admin":["admin"]}}]}},"volume_id":"5502428b-4a89-4a82-bdce-0e8eb5e181fb"}}},{"SubsagaStart":{"params_node_name":"params_for_volume_delete_subsaga","saga_name":"volume-delete"}},{"Action":{"action_name":"volume_delete.decrease_crucible_resource_count","label":"DecreaseCrucibleResourceCount","name":"crucible_resources_to_delete"}},{"Action":{"action_name":"volume_delete.delete_crucible_regions","label":"DeleteCrucibleRegions","name":"no_result_1"}},{"Action":{"action_name":"volume_delete.delete_crucible_running_snapshots","label":"DeleteCrucibleRunningSnapshots","name":"no_result_2"}},{"Action":{"action_name":"volume_delete.delete_crucible_snapshots","label":"DeleteCrucibleSnapshots","name":"no_result_3"}},{"Action":{"action_name":"volume_delete.delete_crucible_snapshot_records","label":"DeleteCrucibleSnapshotRecords","name":"no_result_4"}},{"Action":{"action_name":"volume_delete.find_freed_crucible_regions","label":"FindFreedCrucibleRegions","name":"freed_crucible_regions"}},{"Action":{"action_name":"volume_delete.delete_freed_crucible_regions","label":"DeleteFreedCrucibleRegions","name":"no_result_5"}},{"Action":{"action_name":"volume_delete.hard_delete_volume_record","label":"HardDeleteVolumeRecord","name":"volume_hard_deleted"}},{"SubsagaEnd":{"name":"volume_delete_subsaga_no_result"}},{"Start":{"params":{"disk_id":"eac94d6b-8176-41fd-8617-455c5f662336","project_id":"5f29c645-1da0-4c9a-9e97-85eb8bea9c4c","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"3a67a720-d4af-42e7-875a-3abed4955db5","silo_user_id":"86a1f607-6bb4-4083-b5aa-102b2d1b5a27"}}},{"mapped_fleet_roles":{"admin":["admin"]}}]}},"volume_id":"5502428b-4a89-4a82-bdce-0e8eb5e181fb"}}},"End"]},"saga_name":"disk-delete","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
12762024-12-07T01:00:22.671ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete
12772024-12-07T01:00:22.675ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete
12782024-12-07T01:00:22.675ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
12792024-12-07T01:00:22.675ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
12802024-12-07T01:00:22.675ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
12812024-12-07T01:00:22.678ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
12822024-12-07T01:00:22.678ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
12832024-12-07T01:00:22.681ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
12842024-12-07T01:00:22.681ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
12852024-12-07T01:00:22.693ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
12862024-12-07T01:00:22.693ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"block_size": String("Traditional"), "create_image_id": String("ea2b2089-389c-41b3-ad11-680626f0348e"), "create_snapshot_id": Null, "identity": Object {"description": String(""), "id": String("eac94d6b-8176-41fd-8617-455c5f662336"), "name": String("disk-ea50e9a0169"), "time_created": String("2024-12-07T00:59:31.433521Z"), "time_deleted": Null, "time_modified": String("2024-12-07T00:59:31.433521Z")}, "pantry_address": Null, "project_id": String("5f29c645-1da0-4c9a-9e97-85eb8bea9c4c"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("detached"), "gen": Number(2), "time_updated": String("2024-12-07T00:59:32.580734Z")}, "size": Number(2147483648), "slot": Null, "volume_id": String("5502428b-4a89-4a82-bdce-0e8eb5e181fb")}) node_id = 0 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
12872024-12-07T01:00:22.697ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
12882024-12-07T01:00:22.697ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
12892024-12-07T01:00:22.700ZDEBGnexus (SagaExecutor): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true roles = RoleSet { roles: {} } saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete saga_node = SpaceAccount
12902024-12-07T01:00:22.700ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true resource = Database result = Ok(()) saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete saga_node = SpaceAccount
12912024-12-07T01:00:22.713ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
12922024-12-07T01:00:22.713ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 1 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
12932024-12-07T01:00:22.717ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
12942024-12-07T01:00:22.717ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
12952024-12-07T01:00:22.720ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
12962024-12-07T01:00:22.720ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("3a67a720-d4af-42e7-875a-3abed4955db5"), "silo_user_id": String("86a1f607-6bb4-4083-b5aa-102b2d1b5a27")}}}, Object {"mapped_fleet_roles": Object {"admin": Array [String("admin")]}}]}}, "volume_id": String("5502428b-4a89-4a82-bdce-0e8eb5e181fb")}) node_id = 2 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
12972024-12-07T01:00:22.723ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
12982024-12-07T01:00:22.723ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
12992024-12-07T01:00:22.726ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13002024-12-07T01:00:22.726ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13012024-12-07T01:00:22.730ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13022024-12-07T01:00:22.730ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13032024-12-07T01:00:22.808ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13042024-12-07T01:00:22.808ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"V3": Object {"region_snapshots": Array [], "regions": Array [String("10ac5fc8-2f2e-401d-aa33-5b4c97f4a4ac"), String("36acfd9d-aaf1-4119-ae54-fdb0d19d8772"), String("51169681-9919-45b8-9b3d-1b2e925110fd")]}}) node_id = 4 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13052024-12-07T01:00:22.811ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13062024-12-07T01:00:22.811ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13072024-12-07T01:00:22.814ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13082024-12-07T01:00:22.814ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13092024-12-07T01:00:22.819ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13102024-12-07T01:00:22.819ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13112024-12-07T01:00:22.851ZINFOnexus (SagaExecutor): Transient(region not deleted yet), trying again in 123.267528ms file = nexus/src/app/crucible.rs:828 region_id = 51169681-9919-45b8-9b3d-1b2e925110fd saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete
13122024-12-07T01:00:22.854ZINFOnexus (SagaExecutor): Transient(region not deleted yet), trying again in 138.38194ms file = nexus/src/app/crucible.rs:828 region_id = 10ac5fc8-2f2e-401d-aa33-5b4c97f4a4ac saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete
13132024-12-07T01:00:22.855ZINFOnexus (SagaExecutor): Transient(region not deleted yet), trying again in 122.706171ms file = nexus/src/app/crucible.rs:828 region_id = 36acfd9d-aaf1-4119-ae54-fdb0d19d8772 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete
13142024-12-07T01:00:22.979ZINFOnexus (SagaExecutor): region deleted file = nexus/src/app/crucible.rs:813 region_id = 51169681-9919-45b8-9b3d-1b2e925110fd saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete
13152024-12-07T01:00:22.982ZINFOnexus (SagaExecutor): region deleted file = nexus/src/app/crucible.rs:813 region_id = 36acfd9d-aaf1-4119-ae54-fdb0d19d8772 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete
13162024-12-07T01:00:22.997ZINFOnexus (SagaExecutor): region deleted file = nexus/src/app/crucible.rs:813 region_id = 10ac5fc8-2f2e-401d-aa33-5b4c97f4a4ac saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete
13172024-12-07T01:00:23.042ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13182024-12-07T01:00:23.042ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13192024-12-07T01:00:23.047ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13202024-12-07T01:00:23.047ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13212024-12-07T01:00:23.053ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13222024-12-07T01:00:23.053ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 7 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13232024-12-07T01:00:23.057ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13242024-12-07T01:00:23.057ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13252024-12-07T01:00:23.068ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13262024-12-07T01:00:23.068ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13272024-12-07T01:00:23.071ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13282024-12-07T01:00:23.071ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13292024-12-07T01:00:23.082ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13302024-12-07T01:00:23.082ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Array []) node_id = 9 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13312024-12-07T01:00:23.085ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13322024-12-07T01:00:23.085ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13332024-12-07T01:00:23.089ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13342024-12-07T01:00:23.089ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13352024-12-07T01:00:23.092ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13362024-12-07T01:00:23.092ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13372024-12-07T01:00:23.098ZINFOnexus (SagaExecutor): allocated regions for 5502428b-4a89-4a82-bdce-0e8eb5e181fb is empty, calling volume_hard_delete file = nexus/src/app/sagas/volume_delete.rs:531 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete
13382024-12-07T01:00:23.101ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13392024-12-07T01:00:23.101ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Bool(true)) node_id = 11 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13402024-12-07T01:00:23.105ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13412024-12-07T01:00:23.105ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13422024-12-07T01:00:23.108ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13432024-12-07T01:00:23.108ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Bool(true)) node_id = 12 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13442024-12-07T01:00:23.111ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13452024-12-07T01:00:23.111ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13462024-12-07T01:00:23.114ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13472024-12-07T01:00:23.114ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13482024-12-07T01:00:23.118ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13492024-12-07T01:00:23.118ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3
13502024-12-07T01:00:23.125ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete sec_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
13512024-12-07T01:00:23.125ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3 saga_name = disk-delete saga_result = SagaResult { saga_id: a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3, saga_log: SagaLog { saga_id: a7f8c1e2-5fa9-4e69-ab91-9c6fe76a0ba3, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N006 started, N006 succeeded, N005 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(Object {"block_size": String("Traditional"), "create_image_id": String("ea2b2089-389c-41b3-ad11-680626f0348e"), "create_snapshot_id": Null, "identity": Object {"description": String(""), "id": String("eac94d6b-8176-41fd-8617-455c5f662336"), "name": String("disk-ea50e9a0169"), "time_created": String("2024-12-07T00:59:31.433521Z"), "time_deleted": Null, "time_modified": String("2024-12-07T00:59:31.433521Z")}, "pantry_address": Null, "project_id": String("5f29c645-1da0-4c9a-9e97-85eb8bea9c4c"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("detached"), "gen": Number(2), "time_updated": String("2024-12-07T00:59:32.580734Z")}, "size": Number(2147483648), "slot": Null, "volume_id": String("5502428b-4a89-4a82-bdce-0e8eb5e181fb")}), 1: Succeeded(Null), 2: Succeeded(Object {"serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("3a67a720-d4af-42e7-875a-3abed4955db5"), "silo_user_id": String("86a1f607-6bb4-4083-b5aa-102b2d1b5a27")}}}, Object {"mapped_fleet_roles": Object {"admin": Array [String("admin")]}}]}}, "volume_id": String("5502428b-4a89-4a82-bdce-0e8eb5e181fb")}), 3: Succeeded(Null), 4: Succeeded(Object {"V3": Object {"region_snapshots": Array [], "regions": Array [String("10ac5fc8-2f2e-401d-aa33-5b4c97f4a4ac"), String("36acfd9d-aaf1-4119-ae54-fdb0d19d8772"), String("51169681-9919-45b8-9b3d-1b2e925110fd")]}}), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Null), 8: Succeeded(Null), 9: Succeeded(Array []), 10: Succeeded(Null), 11: Succeeded(Bool(true)), 12: Succeeded(Bool(true)), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Bool(true), node_outputs: {"crucible_resources_to_delete": Object {"V3": Object {"region_snapshots": Array [], "regions": Array [String("10ac5fc8-2f2e-401d-aa33-5b4c97f4a4ac"), String("36acfd9d-aaf1-4119-ae54-fdb0d19d8772"), String("51169681-9919-45b8-9b3d-1b2e925110fd")]}}, "deleted_disk": Object {"block_size": String("Traditional"), "create_image_id": String("ea2b2089-389c-41b3-ad11-680626f0348e"), "create_snapshot_id": Null, "identity": Object {"description": String(""), "id": String("eac94d6b-8176-41fd-8617-455c5f662336"), "name": String("disk-ea50e9a0169"), "time_created": String("2024-12-07T00:59:31.433521Z"), "time_deleted": Null, "time_modified": String("2024-12-07T00:59:31.433521Z")}, "pantry_address": Null, "project_id": String("5f29c645-1da0-4c9a-9e97-85eb8bea9c4c"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("detached"), "gen": Number(2), "time_updated": String("2024-12-07T00:59:32.580734Z")}, "size": Number(2147483648), "slot": Null, "volume_id": String("5502428b-4a89-4a82-bdce-0e8eb5e181fb")}, "freed_crucible_regions": Array [], "no_result1": Null, "no_result_1": Null, "no_result_2": Null, "no_result_3": Null, "no_result_4": Null, "no_result_5": Null, "params_for_volume_delete_subsaga": Object {"serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("3a67a720-d4af-42e7-875a-3abed4955db5"), "silo_user_id": String("86a1f607-6bb4-4083-b5aa-102b2d1b5a27")}}}, Object {"mapped_fleet_roles": Object {"admin": Array [String("admin")]}}]}}, "volume_id": String("5502428b-4a89-4a82-bdce-0e8eb5e181fb")}, "volume_delete_subsaga_no_result": Bool(true), "volume_hard_deleted": Bool(true)} }) }
13522024-12-07T01:00:23.125ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 570756 local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = e3a174d0-4d48-4701-8adf-3067835c1458 response_code = 204 uri = /v1/disks/disk-ea50e9a0169?project=proj-391b50e44b9
13532024-12-07T01:00:23.136ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
13542024-12-07T01:00:23.136ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
13552024-12-07T01:00:23.137ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
13562024-12-07T01:00:23.137ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
13572024-12-07T01:00:23.138ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
13582024-12-07T01:00:23.140ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("token = \\"309f87fb99ee10b5e9dbf0867a83b04371747aad\\"") } result = Ok(())
13592024-12-07T01:00:23.140ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
13602024-12-07T01:00:23.140ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
13612024-12-07T01:00:23.141ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
13622024-12-07T01:00:23.141ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
13632024-12-07T01:00:23.142ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
13642024-12-07T01:00:23.142ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
13652024-12-07T01:00:23.143ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
13662024-12-07T01:00:23.144ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
13672024-12-07T01:00:23.145ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
13682024-12-07T01:00:23.146ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
13692024-12-07T01:00:23.146ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
13702024-12-07T01:00:23.146ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
13712024-12-07T01:00:23.147ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
13722024-12-07T01:00:23.149ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("309f87fb99ee10b5e9dbf0867a83b04371747aad") } result = Ok(())
13732024-12-07T01:00:23.149ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
13742024-12-07T01:00:23.149ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
13752024-12-07T01:00:23.153ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
13762024-12-07T01:00:23.153ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
13772024-12-07T01:00:23.154ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
13782024-12-07T01:00:23.154ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
13792024-12-07T01:00:23.155ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
13802024-12-07T01:00:23.155ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
13812024-12-07T01:00:23.157ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
13822024-12-07T01:00:23.157ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
13832024-12-07T01:00:23.158ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
13842024-12-07T01:00:23.159ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
13852024-12-07T01:00:23.159ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
13862024-12-07T01:00:23.159ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
13872024-12-07T01:00:23.161ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
13882024-12-07T01:00:23.161ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
13892024-12-07T01:00:23.162ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
13902024-12-07T01:00:23.162ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
13912024-12-07T01:00:23.163ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
13922024-12-07T01:00:23.164ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) } result = Ok(())
13932024-12-07T01:00:23.171ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
13942024-12-07T01:00:23.171ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
13952024-12-07T01:00:23.173ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
13962024-12-07T01:00:23.173ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
13972024-12-07T01:00:23.174ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
13982024-12-07T01:00:23.175ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
13992024-12-07T01:00:23.179ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14002024-12-07T01:00:23.179ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14012024-12-07T01:00:23.181ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14022024-12-07T01:00:23.181ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14032024-12-07T01:00:23.183ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14042024-12-07T01:00:23.183ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14052024-12-07T01:00:23.185ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14062024-12-07T01:00:23.185ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14072024-12-07T01:00:23.187ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14082024-12-07T01:00:23.188ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14092024-12-07T01:00:23.190ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14102024-12-07T01:00:23.193ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: 8f54d72e-cecb-4bc0-bd15-3b4fa8d741cc, lookup_type: ByName("default") }, key: a17a37d3-1159-4987-8909-0fb4b4ffe220, lookup_type: ByName("default") } result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14112024-12-07T01:00:23.193ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14122024-12-07T01:00:23.193ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14132024-12-07T01:00:23.195ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14142024-12-07T01:00:23.195ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14152024-12-07T01:00:23.197ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14162024-12-07T01:00:23.197ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14172024-12-07T01:00:23.199ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14182024-12-07T01:00:23.200ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14192024-12-07T01:00:23.202ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14202024-12-07T01:00:23.205ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: 8f54d72e-cecb-4bc0-bd15-3b4fa8d741cc, lookup_type: ByName("default") }, key: a17a37d3-1159-4987-8909-0fb4b4ffe220, lookup_type: ByName("default") } result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14212024-12-07T01:00:23.205ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14222024-12-07T01:00:23.205ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14232024-12-07T01:00:23.207ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14242024-12-07T01:00:23.207ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14252024-12-07T01:00:23.212ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14262024-12-07T01:00:23.212ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14272024-12-07T01:00:23.249ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed roles = RoleSet { roles: {} } uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14282024-12-07T01:00:23.249ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed resource = Database result = Ok(()) uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14292024-12-07T01:00:23.254ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 127692 local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = a7bc567d-9d1a-4f1d-8178-2ae7867e73ed response_code = 204 uri = /v1/vpc-subnets/default?project=proj-391b50e44b9&vpc=default
14302024-12-07T01:00:23.254ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Signaled
14312024-12-07T01:00:23.254ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
14322024-12-07T01:00:23.254ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14332024-12-07T01:00:23.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14342024-12-07T01:00:23.255ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
14352024-12-07T01:00:23.255ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
14362024-12-07T01:00:23.256ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14372024-12-07T01:00:23.256ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
14382024-12-07T01:00:23.257ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
14392024-12-07T01:00:23.257ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
14402024-12-07T01:00:23.257ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14412024-12-07T01:00:23.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14422024-12-07T01:00:23.258ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
14432024-12-07T01:00:23.259ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14442024-12-07T01:00:23.259ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("token = \\"309f87fb99ee10b5e9dbf0867a83b04371747aad\\"") } result = Ok(())
14452024-12-07T01:00:23.259ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
14462024-12-07T01:00:23.259ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
14472024-12-07T01:00:23.260ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
14482024-12-07T01:00:23.260ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14492024-12-07T01:00:23.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14502024-12-07T01:00:23.261ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
14512024-12-07T01:00:23.261ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
14522024-12-07T01:00:23.262ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
14532024-12-07T01:00:23.262ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
14542024-12-07T01:00:23.264ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
14552024-12-07T01:00:23.264ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
14562024-12-07T01:00:23.265ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
14572024-12-07T01:00:23.266ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
14582024-12-07T01:00:23.266ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
14592024-12-07T01:00:23.266ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
14602024-12-07T01:00:23.267ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
14612024-12-07T01:00:23.268ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("309f87fb99ee10b5e9dbf0867a83b04371747aad") } result = Ok(())
14622024-12-07T01:00:23.268ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
14632024-12-07T01:00:23.269ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
14642024-12-07T01:00:23.272ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
14652024-12-07T01:00:23.272ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
14662024-12-07T01:00:23.273ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
14672024-12-07T01:00:23.273ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
14682024-12-07T01:00:23.275ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
14692024-12-07T01:00:23.275ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
14702024-12-07T01:00:23.276ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
14712024-12-07T01:00:23.276ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
14722024-12-07T01:00:23.277ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
14732024-12-07T01:00:23.278ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
14742024-12-07T01:00:23.278ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
14752024-12-07T01:00:23.278ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
14762024-12-07T01:00:23.280ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
14772024-12-07T01:00:23.280ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
14782024-12-07T01:00:23.281ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
14792024-12-07T01:00:23.281ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
14802024-12-07T01:00:23.282ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
14812024-12-07T01:00:23.283ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) } result = Ok(())
14822024-12-07T01:00:23.283ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
14832024-12-07T01:00:23.284ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
14842024-12-07T01:00:23.285ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
14852024-12-07T01:00:23.285ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
14862024-12-07T01:00:23.287ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
14872024-12-07T01:00:23.287ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
14882024-12-07T01:00:23.288ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
14892024-12-07T01:00:23.288ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
14902024-12-07T01:00:23.291ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
14912024-12-07T01:00:23.291ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
14922024-12-07T01:00:23.293ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
14932024-12-07T01:00:23.293ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
14942024-12-07T01:00:23.295ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
14952024-12-07T01:00:23.295ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
14962024-12-07T01:00:23.297ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/vpcs/default?project=proj-391b50e44b9
14972024-12-07T01:00:23.300ZDEBGnexus (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: 8f54d72e-cecb-4bc0-bd15-3b4fa8d741cc, lookup_type: ByName("default") } result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
14982024-12-07T01:00:23.300ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
14992024-12-07T01:00:23.300ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15002024-12-07T01:00:23.303ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15012024-12-07T01:00:23.303ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15022024-12-07T01:00:23.305ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15032024-12-07T01:00:23.305ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15042024-12-07T01:00:23.307ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15052024-12-07T01:00:23.307ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15062024-12-07T01:00:23.309ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15072024-12-07T01:00:23.312ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: 8f54d72e-cecb-4bc0-bd15-3b4fa8d741cc, lookup_type: ByName("default") } result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15082024-12-07T01:00:23.312ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15092024-12-07T01:00:23.312ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15102024-12-07T01:00:23.313ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15112024-12-07T01:00:23.313ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15122024-12-07T01:00:23.317ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15132024-12-07T01:00:23.318ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15142024-12-07T01:00:23.323ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15152024-12-07T01:00:23.323ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15162024-12-07T01:00:23.325ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15172024-12-07T01:00:23.326ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15182024-12-07T01:00:23.327ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15192024-12-07T01:00:23.327ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15202024-12-07T01:00:23.330ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15212024-12-07T01:00:23.333ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: 8f54d72e-cecb-4bc0-bd15-3b4fa8d741cc, lookup_type: ByName("default") }, key: 5fbbfb9f-4ce3-4d6c-98c6-db4ae05bbe98, lookup_type: ById(5fbbfb9f-4ce3-4d6c-98c6-db4ae05bbe98) } result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15222024-12-07T01:00:23.333ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15232024-12-07T01:00:23.333ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15242024-12-07T01:00:23.347ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
15252024-12-07T01:00:23.347ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
15262024-12-07T01:00:23.347ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e0c4c12-c345-430c-9cb2-b88b15396eed", "content-length": "372", "date": "Sat, 07 Dec 2024 01:00:22 GMT"} })
15272024-12-07T01:00:23.347ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15282024-12-07T01:00:23.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15292024-12-07T01:00:23.352ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15302024-12-07T01:00:23.352ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15312024-12-07T01:00:23.358ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15322024-12-07T01:00:23.359ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15332024-12-07T01:00:23.361ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
15342024-12-07T01:00:23.361ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
15352024-12-07T01:00:23.361ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "034e73a2-4a22-4f6d-8bc5-c635d3385885", "date": "Sat, 07 Dec 2024 01:00:22 GMT"} })
15362024-12-07T01:00:23.361ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15372024-12-07T01:00:23.361ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15382024-12-07T01:00:23.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15392024-12-07T01:00:23.361ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15402024-12-07T01:00:23.363ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15412024-12-07T01:00:23.363ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15422024-12-07T01:00:23.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15432024-12-07T01:00:23.363ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15442024-12-07T01:00:23.366ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15452024-12-07T01:00:23.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15462024-12-07T01:00:23.366ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15472024-12-07T01:00:23.366ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15482024-12-07T01:00:23.369ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
15492024-12-07T01:00:23.369ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
15502024-12-07T01:00:23.369ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15512024-12-07T01:00:23.369ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "99b4c66c-c70d-4725-8c35-25458ad6fcd2", "date": "Sat, 07 Dec 2024 01:00:22 GMT"} })
15522024-12-07T01:00:23.369ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15532024-12-07T01:00:23.371ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 117.093039ms iteration = 28 status = {}
15542024-12-07T01:00:23.371ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15552024-12-07T01:00:23.374ZDEBGnexus (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") }, key: 8f54d72e-cecb-4bc0-bd15-3b4fa8d741cc, lookup_type: ByName("default") } result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15562024-12-07T01:00:23.374ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 roles = RoleSet { roles: {} } uri = /v1/vpcs/default?project=proj-391b50e44b9
15572024-12-07T01:00:23.374ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 resource = Database result = Ok(()) uri = /v1/vpcs/default?project=proj-391b50e44b9
15582024-12-07T01:00:23.379ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 123821 local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 88f9ba63-29bc-4ade-98cc-c176e1ee3d85 response_code = 204 uri = /v1/vpcs/default?project=proj-391b50e44b9
15592024-12-07T01:00:23.380ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
15602024-12-07T01:00:23.380ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
15612024-12-07T01:00:23.381ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
15622024-12-07T01:00:23.381ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
15632024-12-07T01:00:23.382ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
15642024-12-07T01:00:23.383ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("token = \\"309f87fb99ee10b5e9dbf0867a83b04371747aad\\"") } result = Ok(())
15652024-12-07T01:00:23.383ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
15662024-12-07T01:00:23.383ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
15672024-12-07T01:00:23.385ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
15682024-12-07T01:00:23.385ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
15692024-12-07T01:00:23.386ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
15702024-12-07T01:00:23.386ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
15712024-12-07T01:00:23.387ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
15722024-12-07T01:00:23.388ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
15732024-12-07T01:00:23.389ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
15742024-12-07T01:00:23.390ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
15752024-12-07T01:00:23.390ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
15762024-12-07T01:00:23.390ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
15772024-12-07T01:00:23.391ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
15782024-12-07T01:00:23.392ZDEBGnexus (ExternalAuthn): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = ConsoleSession { parent: Fleet, key: "309f87fb99ee10b5e9dbf0867a83b04371747aad", lookup_type: ByCompositeId("309f87fb99ee10b5e9dbf0867a83b04371747aad") } result = Ok(())
15792024-12-07T01:00:23.392ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
15802024-12-07T01:00:23.392ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
15812024-12-07T01:00:23.396ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
15822024-12-07T01:00:23.396ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
15832024-12-07T01:00:23.398ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
15842024-12-07T01:00:23.398ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
15852024-12-07T01:00:23.399ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
15862024-12-07T01:00:23.399ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
15872024-12-07T01:00:23.400ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
15882024-12-07T01:00:23.400ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
15892024-12-07T01:00:23.401ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
15902024-12-07T01:00:23.402ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, lookup_type: ById(86a1f607-6bb4-4083-b5aa-102b2d1b5a27) } result = Ok(())
15912024-12-07T01:00:23.402ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
15922024-12-07T01:00:23.402ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
15932024-12-07T01:00:23.404ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
15942024-12-07T01:00:23.404ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
15952024-12-07T01:00:23.405ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {} }
15962024-12-07T01:00:23.405ZDEBGnexus (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
15972024-12-07T01:00:23.406ZDEBGnexus (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
15982024-12-07T01:00:23.407ZDEBGnexus (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) } result = Ok(())
15992024-12-07T01:00:23.407ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16002024-12-07T01:00:23.408ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16012024-12-07T01:00:23.409ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16022024-12-07T01:00:23.409ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16032024-12-07T01:00:23.410ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16042024-12-07T01:00:23.411ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16052024-12-07T01:00:23.413ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16062024-12-07T01:00:23.413ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16072024-12-07T01:00:23.415ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16082024-12-07T01:00:23.415ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16092024-12-07T01:00:23.417ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16102024-12-07T01:00:23.417ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16112024-12-07T01:00:23.419ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/projects/proj-391b50e44b9
16122024-12-07T01:00:23.422ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") } result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16132024-12-07T01:00:23.422ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16142024-12-07T01:00:23.422ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16152024-12-07T01:00:23.423ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16162024-12-07T01:00:23.423ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16172024-12-07T01:00:23.426ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16182024-12-07T01:00:23.426ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16192024-12-07T01:00:23.428ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16202024-12-07T01:00:23.428ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16212024-12-07T01:00:23.430ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {(Silo, 3a67a720-d4af-42e7-875a-3abed4955db5, "admin")} } uri = /v1/projects/proj-391b50e44b9
16222024-12-07T01:00:23.432ZDEBGnexus (dropshot_external): authorize result action = Delete actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Project { parent: Silo { parent: Fleet, key: 3a67a720-d4af-42e7-875a-3abed4955db5, lookup_type: ById(3a67a720-d4af-42e7-875a-3abed4955db5) }, key: 5f29c645-1da0-4c9a-9e97-85eb8bea9c4c, lookup_type: ByName("proj-391b50e44b9") } result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16232024-12-07T01:00:23.432ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16242024-12-07T01:00:23.433ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16252024-12-07T01:00:23.434ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16262024-12-07T01:00:23.434ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16272024-12-07T01:00:23.441ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16282024-12-07T01:00:23.441ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16292024-12-07T01:00:23.444ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16302024-12-07T01:00:23.444ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16312024-12-07T01:00:23.446ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16322024-12-07T01:00:23.446ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16332024-12-07T01:00:23.448ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16342024-12-07T01:00:23.448ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16352024-12-07T01:00:23.450ZDEBGnexus (dropshot_external): roles actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 roles = RoleSet { roles: {} } uri = /v1/projects/proj-391b50e44b9
16362024-12-07T01:00:23.450ZDEBGnexus (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 86a1f607-6bb4-4083-b5aa-102b2d1b5a27, silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, .. }) actor_id = 86a1f607-6bb4-4083-b5aa-102b2d1b5a27 authenticated = true local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 resource = Database result = Ok(()) uri = /v1/projects/proj-391b50e44b9
16372024-12-07T01:00:23.459ZINFOnexus (dropshot_external): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79139 local_addr = 172.30.2.5:443 method = DELETE remote_addr = 10.151.2.100:44326 req_id = 187b2d5b-f65e-4b7e-99fd-7ac3ce678ee0 response_code = 204 uri = /v1/projects/proj-391b50e44b9
16382024-12-07T01:00:27.116ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
16392024-12-07T01:00:27.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
16402024-12-07T01:00:27.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16412024-12-07T01:00:27.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16422024-12-07T01:00:27.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16432024-12-07T01:00:27.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16442024-12-07T01:00:27.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16452024-12-07T01:00:27.122ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16462024-12-07T01:00:27.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16472024-12-07T01:00:27.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16482024-12-07T01:00:27.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16492024-12-07T01:00:27.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16502024-12-07T01:00:27.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16512024-12-07T01:00:27.128ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
16522024-12-07T01:00:27.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16532024-12-07T01:00:27.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16542024-12-07T01:00:27.141ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
16552024-12-07T01:00:27.141ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.161836ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
16562024-12-07T01:00:27.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16572024-12-07T01:00:27.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16582024-12-07T01:00:27.174ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.250299ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16592024-12-07T01:00:27.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 47e40cd8-de4c-4812-aa98-eeb6ab844176 roles = RoleSet { roles: {} } uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
16602024-12-07T01:00:27.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 47e40cd8-de4c-4812-aa98-eeb6ab844176 resource = Database result = Ok(()) uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
16612024-12-07T01:00:27.179ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2230 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 47e40cd8-de4c-4812-aa98-eeb6ab844176 response_code = 200 uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
16622024-12-07T01:00:27.179ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 72bdb03d-d172-484e-b77c-e3bb78fe4b14 roles = RoleSet { roles: {} } uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
16632024-12-07T01:00:27.179ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 72bdb03d-d172-484e-b77c-e3bb78fe4b14 resource = Database result = Ok(()) uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
16642024-12-07T01:00:27.181ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2239 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 72bdb03d-d172-484e-b77c-e3bb78fe4b14 response_code = 200 uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
16652024-12-07T01:00:27.186ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1598 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = 6ae15916-ef58-4aac-9834-c3a292df3c12 response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
16662024-12-07T01:00:37.116ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
16672024-12-07T01:00:37.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
16682024-12-07T01:00:37.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16692024-12-07T01:00:37.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16702024-12-07T01:00:37.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16712024-12-07T01:00:37.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16722024-12-07T01:00:37.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16732024-12-07T01:00:37.121ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16742024-12-07T01:00:37.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16752024-12-07T01:00:37.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16762024-12-07T01:00:37.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16772024-12-07T01:00:37.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16782024-12-07T01:00:37.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16792024-12-07T01:00:37.127ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
16802024-12-07T01:00:37.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16812024-12-07T01:00:37.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16822024-12-07T01:00:37.139ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
16832024-12-07T01:00:37.139ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.128226ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
16842024-12-07T01:00:37.147ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16852024-12-07T01:00:37.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16862024-12-07T01:00:37.174ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 58.071895ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16872024-12-07T01:00:37.185ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1853 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = 6268ab54-e431-4216-86ac-381e6d3e397b response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
16882024-12-07T01:00:42.176ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = c61fc7b0-e8ec-4fab-b827-e343307d8fa8 roles = RoleSet { roles: {} } uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
16892024-12-07T01:00:42.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = c61fc7b0-e8ec-4fab-b827-e343307d8fa8 resource = Database result = Ok(()) uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
16902024-12-07T01:00:42.179ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2926 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = c61fc7b0-e8ec-4fab-b827-e343307d8fa8 response_code = 200 uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
16912024-12-07T01:00:42.179ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 2bb12a60-1854-4b73-90c7-8a8e895f2b21 roles = RoleSet { roles: {} } uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
16922024-12-07T01:00:42.180ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 2bb12a60-1854-4b73-90c7-8a8e895f2b21 resource = Database result = Ok(()) uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
16932024-12-07T01:00:42.182ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2505 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 2bb12a60-1854-4b73-90c7-8a8e895f2b21 response_code = 200 uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
16942024-12-07T01:00:47.116ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
16952024-12-07T01:00:47.116ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
16962024-12-07T01:00:47.116ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
16972024-12-07T01:00:47.116ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16982024-12-07T01:00:47.116ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
16992024-12-07T01:00:47.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
17002024-12-07T01:00:47.116ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
17012024-12-07T01:00:47.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
17022024-12-07T01:00:47.116ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
17032024-12-07T01:00:47.116ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
17042024-12-07T01:00:47.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
17052024-12-07T01:00:47.116ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
17062024-12-07T01:00:47.116ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17072024-12-07T01:00:47.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17082024-12-07T01:00:47.116ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
17092024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
17102024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
17112024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
17122024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Timeout
17132024-12-07T01:00:47.117ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
17142024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
17152024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17162024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
17172024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17182024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
17192024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
17202024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
17212024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
17222024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
17232024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17242024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
17252024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17262024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
17272024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
17282024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
17292024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
17302024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17312024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
17322024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
17332024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
17342024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
17352024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17362024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17372024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17382024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17392024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
17402024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17412024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17422024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
17432024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
17442024-12-07T01:00:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
17452024-12-07T01:00:47.119ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17462024-12-07T01:00:47.119ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4510, tv_nsec: 345074492 } })
17472024-12-07T01:00:47.120ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17482024-12-07T01:00:47.120ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
17492024-12-07T01:00:47.120ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.421658ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
17502024-12-07T01:00:47.121ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
17512024-12-07T01:00:47.121ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17522024-12-07T01:00:47.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17532024-12-07T01:00:47.122ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
17542024-12-07T01:00:47.122ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
17552024-12-07T01:00:47.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
17562024-12-07T01:00:47.124ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17572024-12-07T01:00:47.125ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17582024-12-07T01:00:47.125ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17592024-12-07T01:00:47.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17602024-12-07T01:00:47.125ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 9.018218ms iteration = 26 status = {"errors":[],"records_set_to_done":[]}
17612024-12-07T01:00:47.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17622024-12-07T01:00:47.126ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17632024-12-07T01:00:47.126ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17642024-12-07T01:00:47.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17652024-12-07T01:00:47.127ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
17662024-12-07T01:00:47.127ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17672024-12-07T01:00:47.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17682024-12-07T01:00:47.129ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17692024-12-07T01:00:47.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17702024-12-07T01:00:47.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17712024-12-07T01:00:47.130ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17722024-12-07T01:00:47.132ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4510, tv_nsec: 357885705 } })
17732024-12-07T01:00:47.133ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
17742024-12-07T01:00:47.133ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
17752024-12-07T01:00:47.133ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17762024-12-07T01:00:47.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17772024-12-07T01:00:47.134ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4510, tv_nsec: 359503837 } })
17782024-12-07T01:00:47.134ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
17792024-12-07T01:00:47.135ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 18.310631ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
17802024-12-07T01:00:47.136ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17812024-12-07T01:00:47.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17822024-12-07T01:00:47.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17832024-12-07T01:00:47.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17842024-12-07T01:00:47.137ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17852024-12-07T01:00:47.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17862024-12-07T01:00:47.141ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17872024-12-07T01:00:47.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17882024-12-07T01:00:47.153ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 36.951189ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
17892024-12-07T01:00:47.155ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
17902024-12-07T01:00:47.155ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
17912024-12-07T01:00:47.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
17922024-12-07T01:00:47.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
17932024-12-07T01:00:47.157ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
17942024-12-07T01:00:47.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
17952024-12-07T01:00:47.161ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
17962024-12-07T01:00:47.161ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
17972024-12-07T01:00:47.161ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
17982024-12-07T01:00:47.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
17992024-12-07T01:00:47.163ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
18002024-12-07T01:00:47.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
18012024-12-07T01:00:47.164ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
18022024-12-07T01:00:47.164ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18032024-12-07T01:00:47.164ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
18042024-12-07T01:00:47.164ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
18052024-12-07T01:00:47.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
18062024-12-07T01:00:47.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18072024-12-07T01:00:47.167ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18082024-12-07T01:00:47.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18092024-12-07T01:00:47.169ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18102024-12-07T01:00:47.171ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18112024-12-07T01:00:47.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3, lookup_type: ById(e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3) } result = Ok(())
18122024-12-07T01:00:47.172ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18132024-12-07T01:00:47.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18142024-12-07T01:00:47.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18152024-12-07T01:00:47.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
18162024-12-07T01:00:47.173ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18172024-12-07T01:00:47.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
18182024-12-07T01:00:47.173ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 56.713548ms iteration = 26 status = 0
18192024-12-07T01:00:47.176ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
18202024-12-07T01:00:47.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18212024-12-07T01:00:47.176ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18222024-12-07T01:00:47.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18232024-12-07T01:00:47.177ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
18242024-12-07T01:00:47.177ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 60.631005ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
18252024-12-07T01:00:47.178ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
18262024-12-07T01:00:47.178ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18272024-12-07T01:00:47.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18282024-12-07T01:00:47.185ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1901 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = 6a270fca-9556-48d5-9d9b-b3e53b4a48db response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
18292024-12-07T01:00:47.189ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
18302024-12-07T01:00:47.189ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18312024-12-07T01:00:47.189ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e64338b-95f4-46b5-98a8-4f72c46e8bdb", "content-length": "26", "date": "Sat, 07 Dec 2024 01:00:47 GMT"} })
18322024-12-07T01:00:47.189ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18332024-12-07T01:00:47.192ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18342024-12-07T01:00:47.193ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
18352024-12-07T01:00:47.193ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 76.856796ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
18362024-12-07T01:00:47.195ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 78.175732ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
18372024-12-07T01:00:47.197ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 81.093179ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
18382024-12-07T01:00:47.200ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 83.291192ms iteration = 26 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
18392024-12-07T01:00:47.202ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
18402024-12-07T01:00:47.202ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
18412024-12-07T01:00:47.202ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 85.836423ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
18422024-12-07T01:00:47.218ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
18432024-12-07T01:00:47.218ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18442024-12-07T01:00:47.218ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a2eda36-6098-47f1-9495-8fd8d7268ec8", "content-length": "26", "date": "Sat, 07 Dec 2024 01:00:47 GMT"} })
18452024-12-07T01:00:47.218ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18462024-12-07T01:00:47.219ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18472024-12-07T01:00:47.220ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a zone_address = fd00:1122:3344:101::2
18482024-12-07T01:00:47.220ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18492024-12-07T01:00:47.221ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "51602a19-c30b-4bbf-92f8-b5da8d84a454", "content-length": "26", "date": "Sat, 07 Dec 2024 01:00:47 GMT"} })
18502024-12-07T01:00:47.221ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18512024-12-07T01:00:47.223ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
18522024-12-07T01:00:47.228ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 111.399836ms iteration = 28 status = {"physical_disks_added":0}
18532024-12-07T01:00:47.254ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18542024-12-07T01:00:47.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18552024-12-07T01:00:47.257ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18562024-12-07T01:00:47.258ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
18572024-12-07T01:00:47.258ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18582024-12-07T01:00:47.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18592024-12-07T01:00:47.276ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
18602024-12-07T01:00:47.276ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fe2826a-c002-469a-8d7a-f43eeaf2352f", "content-length": "2", "date": "Sat, 07 Dec 2024 01:00:47 GMT"} })
18612024-12-07T01:00:47.278ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
18622024-12-07T01:00:47.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
18632024-12-07T01:00:47.374ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 257.769813ms iteration = 26 status = {}
18642024-12-07T01:00:47.385ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
18652024-12-07T01:00:47.385ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
18662024-12-07T01:00:47.385ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c0ef183-f71f-4fe4-a0eb-fd6385e333a8", "content-length": "372", "date": "Sat, 07 Dec 2024 01:00:47 GMT"} })
18672024-12-07T01:00:47.385ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18682024-12-07T01:00:47.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18692024-12-07T01:00:47.387ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18702024-12-07T01:00:47.387ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 271.100986ms iteration = 26 status = null
18712024-12-07T01:00:47.388ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
18722024-12-07T01:00:47.388ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18732024-12-07T01:00:47.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18742024-12-07T01:00:47.424ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18752024-12-07T01:00:47.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18762024-12-07T01:00:47.426ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 68b8f65f-54d5-4192-ac26-facf91340245, rack_id: 667e4910-296b-4cf9-a580-7c638e7aa45a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(e642095a-6c9c-4557-b65a-98313f61cfeb) } rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0
18772024-12-07T01:00:47.426ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18782024-12-07T01:00:47.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18792024-12-07T01:00:47.429ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
18802024-12-07T01:00:47.429ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
18812024-12-07T01:00:47.430ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4e5a45a5-2937-4d23-b4b6-871e8a7b818b", "date": "Sat, 07 Dec 2024 01:00:47 GMT"} })
18822024-12-07T01:00:47.430ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18832024-12-07T01:00:47.430ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
18842024-12-07T01:00:47.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18852024-12-07T01:00:47.430ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2b412e4-52dd-46bf-bf4c-a7c9c1595401", "content-length": "691", "date": "Sat, 07 Dec 2024 01:00:47 GMT"} })
18862024-12-07T01:00:47.430ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
18872024-12-07T01:00:47.430ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
18882024-12-07T01:00:47.430ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
18892024-12-07T01:00:47.432ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18902024-12-07T01:00:47.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18912024-12-07T01:00:47.432ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 316.016932ms iteration = 28 status = {}
18922024-12-07T01:00:47.434ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
18932024-12-07T01:00:47.434ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "517130eb-98d4-4874-8542-ce65ae8f7071", "content-length": "134", "date": "Sat, 07 Dec 2024 01:00:47 GMT"} })
18942024-12-07T01:00:47.434ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
18952024-12-07T01:00:47.434ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = e642095a-6c9c-4557-b65a-98313f61cfeb
18962024-12-07T01:00:47.435ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
18972024-12-07T01:00:47.435ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
18982024-12-07T01:00:47.435ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
18992024-12-07T01:00:47.435ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
19002024-12-07T01:00:47.435ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
19012024-12-07T01:00:47.435ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
19022024-12-07T01:00:47.435ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
19032024-12-07T01:00:47.435ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
19042024-12-07T01:00:47.435ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19052024-12-07T01:00:47.435ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
19062024-12-07T01:00:47.435ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19072024-12-07T01:00:47.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19082024-12-07T01:00:47.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19092024-12-07T01:00:47.437ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
19102024-12-07T01:00:47.437ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "e474cfbd-fad2-425d-9682-ca78595e0bba", "content-length": "2", "date": "Sat, 07 Dec 2024 01:00:47 GMT"} })
19112024-12-07T01:00:47.437ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
19122024-12-07T01:00:47.437ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "53cde366-317b-45cd-aaa5-065c02f899ec", "content-length": "52", "date": "Sat, 07 Dec 2024 01:00:47 GMT"} })
19132024-12-07T01:00:47.437ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0
19142024-12-07T01:00:47.437ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
19152024-12-07T01:00:47.439ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
19162024-12-07T01:00:47.439ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
19172024-12-07T01:00:47.517ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "38d8ac5e-971d-4d9e-b045-8888dee66e3a", "date": "Sat, 07 Dec 2024 01:00:47 GMT"} })
19182024-12-07T01:00:47.517ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch = Switch0
19192024-12-07T01:00:47.517ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
19202024-12-07T01:00:47.518ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b05f91fc-a070-4e44-98ba-e391b5dbdd10", "content-length": "682", "date": "Sat, 07 Dec 2024 01:00:46 GMT"} })
19212024-12-07T01:00:47.518ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19222024-12-07T01:00:47.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19232024-12-07T01:00:47.518ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "12b428a1-1395-4a3a-bf8c-873e06ada50f", "date": "Sat, 07 Dec 2024 01:00:47 GMT"} })
19242024-12-07T01:00:47.520ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 403.893025ms iteration = 29 status = {}
19252024-12-07T01:00:47.521ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19262024-12-07T01:00:47.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19272024-12-07T01:00:47.525ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19282024-12-07T01:00:47.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19292024-12-07T01:00:47.528ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19302024-12-07T01:00:47.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19312024-12-07T01:00:47.534ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 418.086825ms iteration = 26 status = {}
19322024-12-07T01:00:57.116ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
19332024-12-07T01:00:57.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
19342024-12-07T01:00:57.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19352024-12-07T01:00:57.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19362024-12-07T01:00:57.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19372024-12-07T01:00:57.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19382024-12-07T01:00:57.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19392024-12-07T01:00:57.121ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19402024-12-07T01:00:57.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19412024-12-07T01:00:57.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19422024-12-07T01:00:57.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19432024-12-07T01:00:57.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19442024-12-07T01:00:57.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19452024-12-07T01:00:57.127ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
19462024-12-07T01:00:57.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19472024-12-07T01:00:57.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19482024-12-07T01:00:57.136ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
19492024-12-07T01:00:57.136ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.2717ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
19502024-12-07T01:00:57.146ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19512024-12-07T01:00:57.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19522024-12-07T01:00:57.147ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.360218ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
19532024-12-07T01:00:57.185ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1569 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = 9b12ad95-eb50-40ee-9009-cd39c5d32ed8 response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
19542024-12-07T01:01:07.117ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
19552024-12-07T01:01:07.117ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
19562024-12-07T01:01:07.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19572024-12-07T01:01:07.117ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19582024-12-07T01:01:07.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19592024-12-07T01:01:07.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19602024-12-07T01:01:07.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19612024-12-07T01:01:07.119ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19622024-12-07T01:01:07.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19632024-12-07T01:01:07.121ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19642024-12-07T01:01:07.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19652024-12-07T01:01:07.121ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.864146ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
19662024-12-07T01:01:07.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19672024-12-07T01:01:07.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19682024-12-07T01:01:07.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19692024-12-07T01:01:07.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19702024-12-07T01:01:07.127ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
19712024-12-07T01:01:07.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19722024-12-07T01:01:07.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19732024-12-07T01:01:07.136ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
19742024-12-07T01:01:07.136ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.983964ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
19752024-12-07T01:01:07.186ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1611 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = 5500ba9e-45ed-4abb-b00f-426114efc3ae response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
19762024-12-07T01:01:12.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 41e4fcac-8378-490a-a4f5-7549b39c9200 roles = RoleSet { roles: {} } uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
19772024-12-07T01:01:12.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 41e4fcac-8378-490a-a4f5-7549b39c9200 resource = Database result = Ok(()) uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
19782024-12-07T01:01:12.180ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3133 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 41e4fcac-8378-490a-a4f5-7549b39c9200 response_code = 200 uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
19792024-12-07T01:01:12.180ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 717a5881-08cf-41fd-8412-13a5060d204d roles = RoleSet { roles: {} } uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
19802024-12-07T01:01:12.181ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 717a5881-08cf-41fd-8412-13a5060d204d resource = Database result = Ok(()) uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
19812024-12-07T01:01:12.183ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2819 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 717a5881-08cf-41fd-8412-13a5060d204d response_code = 200 uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
19822024-12-07T01:01:17.116ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
19832024-12-07T01:01:17.116ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
19842024-12-07T01:01:17.116ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
19852024-12-07T01:01:17.116ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19862024-12-07T01:01:17.116ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
19872024-12-07T01:01:17.116ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 29 reason = Timeout
19882024-12-07T01:01:17.116ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
19892024-12-07T01:01:17.116ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
19902024-12-07T01:01:17.116ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19912024-12-07T01:01:17.116ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
19922024-12-07T01:01:17.116ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
19932024-12-07T01:01:17.116ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19942024-12-07T01:01:17.116ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
19952024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19962024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
19972024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
19982024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19992024-12-07T01:01:17.117ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
20002024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
20012024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
20022024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
20032024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20042024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
20052024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
20062024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
20072024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
20082024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
20092024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
20102024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20112024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
20122024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
20132024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
20142024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20152024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
20162024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 30 reason = Timeout
20172024-12-07T01:01:17.117ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
20182024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20192024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
20202024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
20212024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
20222024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20232024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
20242024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
20252024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
20262024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
20272024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
20282024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
20292024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
20302024-12-07T01:01:17.117ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-07 00:51:17.117196531 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
20312024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
20322024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
20332024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
20342024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
20352024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
20362024-12-07T01:01:17.117ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70
20372024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20382024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
20392024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.96\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70"}
20402024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
20412024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
20422024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
20432024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20442024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20452024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20462024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
20472024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20482024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
20492024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20502024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20512024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20522024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
20532024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20542024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20552024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
20562024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
20572024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20582024-12-07T01:01:17.117ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5661fac0-67b8-45f8-869a-67dd3e21bd0b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("be11a55e-cb67-4535-ad44-b9a4d4586728.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("fd72061d-6779-40de-8011-8e899981c9e7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("fd72061d-6779-40de-8011-8e899981c9e7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 4540, tv_nsec: 342852563 } })
20592024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20602024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
20612024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
20622024-12-07T01:01:17.117ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("eda26d3e-18ec-4c21-8ff4-73abfd373bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("fe4d45f4-708c-423b-9b9c-06a074a360b5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("fe4d45f4-708c-423b-9b9c-06a074a360b5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 4540, tv_nsec: 342916174 } })
20632024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
20642024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
20652024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20662024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20672024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
20682024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
20692024-12-07T01:01:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
20702024-12-07T01:01:17.118ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
20712024-12-07T01:01:17.118ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.220925ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
20722024-12-07T01:01:17.118ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
20732024-12-07T01:01:17.118ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.430308ms iteration = 14 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
20742024-12-07T01:01:17.118ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20752024-12-07T01:01:17.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20762024-12-07T01:01:17.119ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20772024-12-07T01:01:17.119ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20782024-12-07T01:01:17.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20792024-12-07T01:01:17.120ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20802024-12-07T01:01:17.121ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.748014ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
20812024-12-07T01:01:17.122ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
20822024-12-07T01:01:17.122ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20832024-12-07T01:01:17.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20842024-12-07T01:01:17.122ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
20852024-12-07T01:01:17.122ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
20862024-12-07T01:01:17.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
20872024-12-07T01:01:17.125ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20882024-12-07T01:01:17.125ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20892024-12-07T01:01:17.125ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20902024-12-07T01:01:17.127ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
20912024-12-07T01:01:17.127ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20922024-12-07T01:01:17.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20932024-12-07T01:01:17.127ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
20942024-12-07T01:01:17.128ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20952024-12-07T01:01:17.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20962024-12-07T01:01:17.128ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
20972024-12-07T01:01:17.128ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20982024-12-07T01:01:17.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20992024-12-07T01:01:17.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21002024-12-07T01:01:17.135ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21012024-12-07T01:01:17.135ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21022024-12-07T01:01:17.135ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21032024-12-07T01:01:17.135ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 18.243019ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
21042024-12-07T01:01:17.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21052024-12-07T01:01:17.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21062024-12-07T01:01:17.137ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4540, tv_nsec: 362313947 } })
21072024-12-07T01:01:17.137ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
21082024-12-07T01:01:17.137ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
21092024-12-07T01:01:17.137ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
21102024-12-07T01:01:17.137ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 20.517695ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
21112024-12-07T01:01:17.137ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
21122024-12-07T01:01:17.137ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21132024-12-07T01:01:17.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
21142024-12-07T01:01:17.139ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-07 00:51:17.117196531 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
21152024-12-07T01:01:17.139ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 22.399231ms iteration = 14 status = {"expiration":"2024-12-07T00:51:17.117196531Z","pruned":[]}
21162024-12-07T01:01:17.139ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4540, tv_nsec: 364812946 } })
21172024-12-07T01:01:17.139ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
21182024-12-07T01:01:17.139ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21192024-12-07T01:01:17.139ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21202024-12-07T01:01:17.139ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
21212024-12-07T01:01:17.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21222024-12-07T01:01:17.142ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21232024-12-07T01:01:17.144ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
21242024-12-07T01:01:17.144ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
21252024-12-07T01:01:17.144ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
21262024-12-07T01:01:17.144ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21272024-12-07T01:01:17.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21282024-12-07T01:01:17.144ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21292024-12-07T01:01:17.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21302024-12-07T01:01:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21312024-12-07T01:01:17.146ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21322024-12-07T01:01:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21332024-12-07T01:01:17.146ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4540, tv_nsec: 372169462 } })
21342024-12-07T01:01:17.147ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
21352024-12-07T01:01:17.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
21362024-12-07T01:01:17.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21372024-12-07T01:01:17.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21382024-12-07T01:01:17.152ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21392024-12-07T01:01:17.153ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21402024-12-07T01:01:17.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21412024-12-07T01:01:17.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21422024-12-07T01:01:17.157ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21432024-12-07T01:01:17.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21442024-12-07T01:01:17.158ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
21452024-12-07T01:01:17.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
21462024-12-07T01:01:17.159ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21472024-12-07T01:01:17.162ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
21482024-12-07T01:01:17.162ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21492024-12-07T01:01:17.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21502024-12-07T01:01:17.167ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
21512024-12-07T01:01:17.167ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
21522024-12-07T01:01:17.167ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
21532024-12-07T01:01:17.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
21542024-12-07T01:01:17.168ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
21552024-12-07T01:01:17.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
21562024-12-07T01:01:17.170ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 53.356722ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
21572024-12-07T01:01:17.170ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
21582024-12-07T01:01:17.170ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 53.628547ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
21592024-12-07T01:01:17.172ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
21602024-12-07T01:01:17.172ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
21612024-12-07T01:01:17.172ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
21622024-12-07T01:01:17.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
21632024-12-07T01:01:17.174ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21642024-12-07T01:01:17.174ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 57.30305ms iteration = 27 status = {"errors":[],"records_set_to_done":[]}
21652024-12-07T01:01:17.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21662024-12-07T01:01:17.175ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
21672024-12-07T01:01:17.175ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
21682024-12-07T01:01:17.175ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 58.073415ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
21692024-12-07T01:01:17.176ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
21702024-12-07T01:01:17.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
21712024-12-07T01:01:17.178ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 61.250988ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
21722024-12-07T01:01:17.185ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21732024-12-07T01:01:17.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21742024-12-07T01:01:17.185ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1982 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = 58ff9686-3d22-425b-822b-1c14b1ff5e23 response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
21752024-12-07T01:01:17.186ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21762024-12-07T01:01:17.186ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21772024-12-07T01:01:17.186ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 69.620773ms iteration = 29 status = {"physical_disks_added":0}
21782024-12-07T01:01:17.187ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21792024-12-07T01:01:17.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21802024-12-07T01:01:17.188ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
21812024-12-07T01:01:17.188ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
21822024-12-07T01:01:17.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
21832024-12-07T01:01:17.189ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
21842024-12-07T01:01:17.189ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21852024-12-07T01:01:17.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21862024-12-07T01:01:17.190ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21872024-12-07T01:01:17.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3, lookup_type: ById(e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3) } result = Ok(())
21882024-12-07T01:01:17.193ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21892024-12-07T01:01:17.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21902024-12-07T01:01:17.193ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21912024-12-07T01:01:17.194ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
21922024-12-07T01:01:17.194ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
21932024-12-07T01:01:17.194ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 77.317024ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
21942024-12-07T01:01:17.194ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 77.798774ms iteration = 27 status = 0
21952024-12-07T01:01:17.196ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 78.981247ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21962024-12-07T01:01:17.196ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
21972024-12-07T01:01:17.196ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 79.711592ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
21982024-12-07T01:01:17.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
21992024-12-07T01:01:17.196ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22002024-12-07T01:01:17.196ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
22012024-12-07T01:01:17.196ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ae6d4ace-ffdd-4139-9e11-1794ae966042 dns_zone_name = oxide.test found_so_far = 0 version = 2
22022024-12-07T01:01:17.196ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22032024-12-07T01:01:17.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22042024-12-07T01:01:17.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22052024-12-07T01:01:17.198ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22062024-12-07T01:01:17.198ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22072024-12-07T01:01:17.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22082024-12-07T01:01:17.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22092024-12-07T01:01:17.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22102024-12-07T01:01:17.201ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22112024-12-07T01:01:17.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22122024-12-07T01:01:17.201ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
22132024-12-07T01:01:17.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
22142024-12-07T01:01:17.202ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
22152024-12-07T01:01:17.202ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
22162024-12-07T01:01:17.202ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
22172024-12-07T01:01:17.202ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22182024-12-07T01:01:17.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22192024-12-07T01:01:17.203ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22202024-12-07T01:01:17.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
22212024-12-07T01:01:17.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22222024-12-07T01:01:17.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22232024-12-07T01:01:17.204ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
22242024-12-07T01:01:17.204ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22252024-12-07T01:01:17.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22262024-12-07T01:01:17.204ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 88.009355ms iteration = 27 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
22272024-12-07T01:01:17.236ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
22282024-12-07T01:01:17.248ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
22292024-12-07T01:01:17.273ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22302024-12-07T01:01:17.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22312024-12-07T01:01:17.305ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
22322024-12-07T01:01:17.305ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22332024-12-07T01:01:17.305ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "df01af19-06c3-4038-9138-e8bac7c248e6", "content-length": "26", "date": "Sat, 07 Dec 2024 01:01:16 GMT"} })
22342024-12-07T01:01:17.305ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22352024-12-07T01:01:17.307ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22362024-12-07T01:01:17.390ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
22372024-12-07T01:01:17.390ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22382024-12-07T01:01:17.393ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ae6d4ace-ffdd-4139-9e11-1794ae966042 dns_zone_name = oxide.test found_so_far = 1 version = 2
22392024-12-07T01:01:17.393ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
22402024-12-07T01:01:17.393ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-07 00:48:45.853224 UTC dns_group = external generation = 2
22412024-12-07T01:01:17.393ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 276.432541ms iteration = 14 status = {"generation":2}
22422024-12-07T01:01:17.393ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22432024-12-07T01:01:17.396ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22442024-12-07T01:01:17.396ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22452024-12-07T01:01:17.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22462024-12-07T01:01:17.397ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "80a80869-a799-4303-b8e5-1c124f2937ac", "content-length": "372", "date": "Sat, 07 Dec 2024 01:01:16 GMT"} })
22472024-12-07T01:01:17.397ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22482024-12-07T01:01:17.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22492024-12-07T01:01:17.404ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
22502024-12-07T01:01:17.404ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e3ba3312-50bd-489c-aada-a2269d4850f8", "date": "Sat, 07 Dec 2024 01:01:16 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22512024-12-07T01:01:17.404ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22522024-12-07T01:01:17.404ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
22532024-12-07T01:01:17.404ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 287.346896ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
22542024-12-07T01:01:17.405ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
22552024-12-07T01:01:17.405ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22562024-12-07T01:01:17.405ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4822f90d-af9b-438f-b618-cd240f620009", "content-length": "26", "date": "Sat, 07 Dec 2024 01:01:16 GMT"} })
22572024-12-07T01:01:17.405ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22582024-12-07T01:01:17.406ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "30a99743-84ab-4e1b-afd1-e903c1468d1f", "date": "Sat, 07 Dec 2024 01:01:16 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22592024-12-07T01:01:17.406ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22602024-12-07T01:01:17.408ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22612024-12-07T01:01:17.409ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22622024-12-07T01:01:17.409ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 292.948347ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
22632024-12-07T01:01:17.420ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
22642024-12-07T01:01:17.422ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 304.872462ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
22652024-12-07T01:01:17.426ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
22662024-12-07T01:01:17.426ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 52ba8747-d0f6-43d9-9ec0-995aa6df639a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
22672024-12-07T01:01:17.426ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22682024-12-07T01:01:17.427ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22692024-12-07T01:01:17.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22702024-12-07T01:01:17.428ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a zone_address = fd00:1122:3344:101::2
22712024-12-07T01:01:17.428ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22722024-12-07T01:01:17.428ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "50039595-ca4a-4e73-9a6a-3800ec89b381", "content-length": "26", "date": "Sat, 07 Dec 2024 01:01:16 GMT"} })
22732024-12-07T01:01:17.428ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22742024-12-07T01:01:17.428ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-07T00:48:09.490271Z, time_modified: 2024-12-07T00:48:09.490271Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, db_silo: Silo { identity: SiloIdentity { id: 3a67a720-d4af-42e7-875a-3abed4955db5, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-07T00:48:45.569718Z, time_modified: 2024-12-07T00:48:45.569718Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6317ba4eb1538be6191eb18b3b3531d70dbee51545c790c2aff24b81494c24f4" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, db_silo: Silo { identity: SiloIdentity { id: 3a67a720-d4af-42e7-875a-3abed4955db5, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-07T00:48:45.569718Z, time_modified: 2024-12-07T00:48:45.569718Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6317ba4eb1538be6191eb18b3b3531d70dbee51545c790c2aff24b81494c24f4" }] }) }
22752024-12-07T01:01:17.428ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 311.742978ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"3a67a720-d4af-42e7-875a-3abed4955db5","tls_certs":["6317ba4eb1538be6191eb18b3b3531d70dbee51545c790c2aff24b81494c24f4"]}},"default_endpoint":{"silo_id":"3a67a720-d4af-42e7-875a-3abed4955db5","tls_certs":["6317ba4eb1538be6191eb18b3b3531d70dbee51545c790c2aff24b81494c24f4"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
22762024-12-07T01:01:17.429ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22772024-12-07T01:01:17.429ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22782024-12-07T01:01:17.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22792024-12-07T01:01:17.430ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22802024-12-07T01:01:17.430ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
22812024-12-07T01:01:17.433ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22822024-12-07T01:01:17.433ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22832024-12-07T01:01:17.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22842024-12-07T01:01:17.435ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 52ba8747-d0f6-43d9-9ec0-995aa6df639a dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
22852024-12-07T01:01:17.435ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
22862024-12-07T01:01:17.435ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-07 00:48:44.409012 UTC dns_group = internal generation = 1
22872024-12-07T01:01:17.435ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 318.690524ms iteration = 14 status = {"generation":1}
22882024-12-07T01:01:17.447ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
22892024-12-07T01:01:17.447ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
22902024-12-07T01:01:17.448ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "680e8346-07e5-445a-88d5-7a14642f9731", "date": "Sat, 07 Dec 2024 01:01:16 GMT"} })
22912024-12-07T01:01:17.448ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22922024-12-07T01:01:17.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22932024-12-07T01:01:17.451ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22942024-12-07T01:01:17.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22952024-12-07T01:01:17.454ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22962024-12-07T01:01:17.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22972024-12-07T01:01:17.458ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
22982024-12-07T01:01:17.458ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22992024-12-07T01:01:17.458ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "fac827e3-7041-491a-bec9-8ff809c4c047", "date": "Sat, 07 Dec 2024 01:01:16 GMT"} })
23002024-12-07T01:01:17.460ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 343.186418ms iteration = 30 status = {}
23012024-12-07T01:01:17.460ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23022024-12-07T01:01:17.462ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
23032024-12-07T01:01:17.462ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23042024-12-07T01:01:17.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23052024-12-07T01:01:17.553ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
23062024-12-07T01:01:17.553ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b004379-b504-46aa-b1b3-6bb57e9dfe2d", "content-length": "2", "date": "Sat, 07 Dec 2024 01:01:17 GMT"} })
23072024-12-07T01:01:17.553ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
23082024-12-07T01:01:17.554ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "6a1ed0b1-be6b-4061-89ff-0464f70f525a", "date": "Sat, 07 Dec 2024 01:01:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
23092024-12-07T01:01:17.554ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
23102024-12-07T01:01:17.556ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
23112024-12-07T01:01:17.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
23122024-12-07T01:01:17.557ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "47509b66-72a0-42e4-9960-4a5ae3baba83", "date": "Sat, 07 Dec 2024 01:01:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
23132024-12-07T01:01:17.557ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
23142024-12-07T01:01:17.558ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 440.907344ms iteration = 27 status = null
23152024-12-07T01:01:17.559ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "a7c74c27-558a-4dfc-a7fd-3f4cb685dc74", "date": "Sat, 07 Dec 2024 01:01:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
23162024-12-07T01:01:17.559ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
23172024-12-07T01:01:17.561ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
23182024-12-07T01:01:17.561ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 445.118747ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
23192024-12-07T01:01:17.593ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 475.828223ms iteration = 27 status = {}
23202024-12-07T01:01:17.617ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
23212024-12-07T01:01:17.618ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "12eee931-945b-43cc-8cda-69b8c2c659bb", "content-length": "691", "date": "Sat, 07 Dec 2024 01:01:17 GMT"} })
23222024-12-07T01:01:17.618ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
23232024-12-07T01:01:17.618ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
23242024-12-07T01:01:17.618ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
23252024-12-07T01:01:17.620ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 503.381766ms iteration = 29 status = {}
23262024-12-07T01:01:17.656ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23272024-12-07T01:01:17.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23282024-12-07T01:01:17.659ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 68b8f65f-54d5-4192-ac26-facf91340245, rack_id: 667e4910-296b-4cf9-a580-7c638e7aa45a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(e642095a-6c9c-4557-b65a-98313f61cfeb) } rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0
23292024-12-07T01:01:17.659ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23302024-12-07T01:01:17.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23312024-12-07T01:01:17.669ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
23322024-12-07T01:01:17.669ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "e181d097-8575-49d1-863a-3d75a5a7c8f2", "content-length": "134", "date": "Sat, 07 Dec 2024 01:01:17 GMT"} })
23332024-12-07T01:01:17.669ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
23342024-12-07T01:01:17.669ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = e642095a-6c9c-4557-b65a-98313f61cfeb
23352024-12-07T01:01:17.669ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
23362024-12-07T01:01:17.669ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
23372024-12-07T01:01:17.669ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23382024-12-07T01:01:17.669ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23392024-12-07T01:01:17.669ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23402024-12-07T01:01:17.669ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
23412024-12-07T01:01:17.669ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23422024-12-07T01:01:17.669ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
23432024-12-07T01:01:17.670ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
23442024-12-07T01:01:17.670ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23452024-12-07T01:01:17.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23462024-12-07T01:01:17.672ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
23472024-12-07T01:01:17.672ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "b79bf5b4-a1a6-4d5b-9789-2484c3e51a1e", "content-length": "2", "date": "Sat, 07 Dec 2024 01:01:17 GMT"} })
23482024-12-07T01:01:17.672ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
23492024-12-07T01:01:17.672ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "37ac4b5f-3884-475e-9e8d-1b28977864f2", "content-length": "52", "date": "Sat, 07 Dec 2024 01:01:17 GMT"} })
23502024-12-07T01:01:17.672ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0
23512024-12-07T01:01:17.672ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
23522024-12-07T01:01:17.764ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "610709a2-c5a0-4fc6-9000-a6b6a3d0a553", "date": "Sat, 07 Dec 2024 01:01:17 GMT"} })
23532024-12-07T01:01:17.764ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch = Switch0
23542024-12-07T01:01:17.764ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
23552024-12-07T01:01:17.764ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "892f037b-92ed-41f6-87df-f330825f35b8", "content-length": "682", "date": "Sat, 07 Dec 2024 01:01:17 GMT"} })
23562024-12-07T01:01:17.764ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23572024-12-07T01:01:17.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23582024-12-07T01:01:17.768ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23592024-12-07T01:01:17.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23602024-12-07T01:01:17.772ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23612024-12-07T01:01:17.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23622024-12-07T01:01:17.774ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23632024-12-07T01:01:17.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23642024-12-07T01:01:17.782ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 665.630396ms iteration = 27 status = {}
23652024-12-07T01:01:27.116ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
23662024-12-07T01:01:27.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
23672024-12-07T01:01:27.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23682024-12-07T01:01:27.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23692024-12-07T01:01:27.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23702024-12-07T01:01:27.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23712024-12-07T01:01:27.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23722024-12-07T01:01:27.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23732024-12-07T01:01:27.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23742024-12-07T01:01:27.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23752024-12-07T01:01:27.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23762024-12-07T01:01:27.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23772024-12-07T01:01:27.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23782024-12-07T01:01:27.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23792024-12-07T01:01:27.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23802024-12-07T01:01:27.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
23812024-12-07T01:01:27.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23822024-12-07T01:01:27.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23832024-12-07T01:01:27.165ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
23842024-12-07T01:01:27.165ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.437595ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
23852024-12-07T01:01:27.174ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 58.431742ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
23862024-12-07T01:01:27.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 7f5152f2-8d0a-493f-b3a3-ea1e920a5bd5 roles = RoleSet { roles: {} } uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
23872024-12-07T01:01:27.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 7f5152f2-8d0a-493f-b3a3-ea1e920a5bd5 resource = Database result = Ok(()) uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
23882024-12-07T01:01:27.179ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2837 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 7f5152f2-8d0a-493f-b3a3-ea1e920a5bd5 response_code = 200 uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
23892024-12-07T01:01:27.180ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 1a26adc1-f781-4fc7-acb6-a53da46414a7 roles = RoleSet { roles: {} } uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
23902024-12-07T01:01:27.180ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 1a26adc1-f781-4fc7-acb6-a53da46414a7 resource = Database result = Ok(()) uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
23912024-12-07T01:01:27.182ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1921 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 1a26adc1-f781-4fc7-acb6-a53da46414a7 response_code = 200 uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
23922024-12-07T01:01:27.187ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2671 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = 701ee358-69fd-494c-b487-ddb0644db48e response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
23932024-12-07T01:01:37.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
23942024-12-07T01:01:37.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23952024-12-07T01:01:37.116ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
23962024-12-07T01:01:37.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23972024-12-07T01:01:37.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23982024-12-07T01:01:37.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23992024-12-07T01:01:37.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24002024-12-07T01:01:37.122ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24012024-12-07T01:01:37.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24022024-12-07T01:01:37.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24032024-12-07T01:01:37.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24042024-12-07T01:01:37.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24052024-12-07T01:01:37.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24062024-12-07T01:01:37.129ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
24072024-12-07T01:01:37.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24082024-12-07T01:01:37.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24092024-12-07T01:01:37.141ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
24102024-12-07T01:01:37.141ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.645836ms iteration = 81 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
24112024-12-07T01:01:37.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24122024-12-07T01:01:37.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24132024-12-07T01:01:37.173ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.024785ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
24142024-12-07T01:01:37.185ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1635 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = ad737470-7943-414b-8694-06e4f871e300 response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
24152024-12-07T01:01:47.116ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
24162024-12-07T01:01:47.116ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24172024-12-07T01:01:47.116ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 28 reason = Timeout
24182024-12-07T01:01:47.116ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
24192024-12-07T01:01:47.116ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 30 reason = Timeout
24202024-12-07T01:01:47.116ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
24212024-12-07T01:01:47.116ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24222024-12-07T01:01:47.116ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
24232024-12-07T01:01:47.116ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
24242024-12-07T01:01:47.116ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
24252024-12-07T01:01:47.116ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24262024-12-07T01:01:47.116ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
24272024-12-07T01:01:47.116ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
24282024-12-07T01:01:47.116ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
24292024-12-07T01:01:47.116ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
24302024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
24312024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
24322024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
24332024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 31 reason = Timeout
24342024-12-07T01:01:47.117ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
24352024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
24362024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24372024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
24382024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
24392024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24402024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
24412024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
24422024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
24432024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
24442024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24452024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
24462024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24472024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
24482024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
24492024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
24502024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24512024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
24522024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
24532024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24542024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24552024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
24562024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
24572024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
24582024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24592024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
24602024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
24612024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24622024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
24632024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
24642024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24652024-12-07T01:01:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24662024-12-07T01:01:47.119ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24672024-12-07T01:01:47.120ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.869816ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
24682024-12-07T01:01:47.121ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24692024-12-07T01:01:47.121ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4570, tv_nsec: 346327236 } })
24702024-12-07T01:01:47.121ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
24712024-12-07T01:01:47.121ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4570, tv_nsec: 346976039 } })
24722024-12-07T01:01:47.121ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
24732024-12-07T01:01:47.122ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
24742024-12-07T01:01:47.122ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
24752024-12-07T01:01:47.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
24762024-12-07T01:01:47.122ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
24772024-12-07T01:01:47.122ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24782024-12-07T01:01:47.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24792024-12-07T01:01:47.124ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
24802024-12-07T01:01:47.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
24812024-12-07T01:01:47.127ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24822024-12-07T01:01:47.127ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24832024-12-07T01:01:47.129ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
24842024-12-07T01:01:47.129ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24852024-12-07T01:01:47.130ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
24862024-12-07T01:01:47.130ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24872024-12-07T01:01:47.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24882024-12-07T01:01:47.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24892024-12-07T01:01:47.132ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24902024-12-07T01:01:47.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24912024-12-07T01:01:47.137ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24922024-12-07T01:01:47.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24932024-12-07T01:01:47.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24942024-12-07T01:01:47.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24952024-12-07T01:01:47.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24962024-12-07T01:01:47.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24972024-12-07T01:01:47.161ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24982024-12-07T01:01:47.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24992024-12-07T01:01:47.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25002024-12-07T01:01:47.162ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 45.864215ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
25012024-12-07T01:01:47.163ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
25022024-12-07T01:01:47.163ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25032024-12-07T01:01:47.163ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc397ec6-2cc9-424e-a322-d11a678c7648", "content-length": "26", "date": "Sat, 07 Dec 2024 01:01:46 GMT"} })
25042024-12-07T01:01:47.163ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25052024-12-07T01:01:47.163ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
25062024-12-07T01:01:47.163ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25072024-12-07T01:01:47.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25082024-12-07T01:01:47.164ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
25092024-12-07T01:01:47.164ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 47.66634ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
25102024-12-07T01:01:47.165ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
25112024-12-07T01:01:47.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25122024-12-07T01:01:47.165ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 48.840623ms iteration = 28 status = {"errors":[],"records_set_to_done":[]}
25132024-12-07T01:01:47.166ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4570, tv_nsec: 391843513 } })
25142024-12-07T01:01:47.166ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
25152024-12-07T01:01:47.166ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
25162024-12-07T01:01:47.166ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
25172024-12-07T01:01:47.166ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
25182024-12-07T01:01:47.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
25192024-12-07T01:01:47.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
25202024-12-07T01:01:47.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25212024-12-07T01:01:47.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25222024-12-07T01:01:47.168ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
25232024-12-07T01:01:47.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
25242024-12-07T01:01:47.169ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25252024-12-07T01:01:47.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25262024-12-07T01:01:47.169ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25272024-12-07T01:01:47.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25282024-12-07T01:01:47.169ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
25292024-12-07T01:01:47.169ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
25302024-12-07T01:01:47.169ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
25312024-12-07T01:01:47.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
25322024-12-07T01:01:47.170ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25332024-12-07T01:01:47.171ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 53.994854ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
25342024-12-07T01:01:47.171ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25352024-12-07T01:01:47.171ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
25362024-12-07T01:01:47.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25372024-12-07T01:01:47.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
25382024-12-07T01:01:47.172ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
25392024-12-07T01:01:47.172ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25402024-12-07T01:01:47.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25412024-12-07T01:01:47.175ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25422024-12-07T01:01:47.176ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
25432024-12-07T01:01:47.177ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25442024-12-07T01:01:47.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25452024-12-07T01:01:47.182ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
25462024-12-07T01:01:47.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
25472024-12-07T01:01:47.186ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2045 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = f9271ad4-7d52-4cc3-bb18-a49b36a56ed3 response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
25482024-12-07T01:01:47.189ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
25492024-12-07T01:01:47.189ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 72.578721ms iteration = 82 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
25502024-12-07T01:01:47.194ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
25512024-12-07T01:01:47.194ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25522024-12-07T01:01:47.194ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6604b78c-fb2a-4668-bb4f-7576474fabc9", "content-length": "26", "date": "Sat, 07 Dec 2024 01:01:47 GMT"} })
25532024-12-07T01:01:47.194ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25542024-12-07T01:01:47.196ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
25552024-12-07T01:01:47.209ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 92.378011ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
25562024-12-07T01:01:47.209ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
25572024-12-07T01:01:47.209ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
25582024-12-07T01:01:47.209ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 92.488734ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
25592024-12-07T01:01:47.209ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 92.762349ms iteration = 28 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
25602024-12-07T01:01:47.210ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25612024-12-07T01:01:47.213ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 96.224388ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
25622024-12-07T01:01:47.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3, lookup_type: ById(e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3) } result = Ok(())
25632024-12-07T01:01:47.213ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25642024-12-07T01:01:47.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25652024-12-07T01:01:47.215ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 98.321939ms iteration = 28 status = 0
25662024-12-07T01:01:47.230ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25672024-12-07T01:01:47.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25682024-12-07T01:01:47.248ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
25692024-12-07T01:01:47.248ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd51ab26-7c38-4a53-9d8e-94afc05fa69f", "content-length": "2", "date": "Sat, 07 Dec 2024 01:01:47 GMT"} })
25702024-12-07T01:01:47.250ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
25712024-12-07T01:01:47.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
25722024-12-07T01:01:47.254ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 137.702125ms iteration = 28 status = null
25732024-12-07T01:01:47.259ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a zone_address = fd00:1122:3344:101::2
25742024-12-07T01:01:47.259ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25752024-12-07T01:01:47.260ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "56d93a43-e582-4e95-bbc0-7e18eef4bfde", "content-length": "26", "date": "Sat, 07 Dec 2024 01:01:47 GMT"} })
25762024-12-07T01:01:47.260ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25772024-12-07T01:01:47.262ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
25782024-12-07T01:01:47.280ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 163.882871ms iteration = 28 status = {}
25792024-12-07T01:01:47.284ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 167.427142ms iteration = 30 status = {"physical_disks_added":0}
25802024-12-07T01:01:47.313ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
25812024-12-07T01:01:47.313ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
25822024-12-07T01:01:47.314ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1dc9ca6a-1469-4c9f-8c6b-d6571e625613", "content-length": "372", "date": "Sat, 07 Dec 2024 01:01:46 GMT"} })
25832024-12-07T01:01:47.314ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25842024-12-07T01:01:47.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25852024-12-07T01:01:47.325ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25862024-12-07T01:01:47.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25872024-12-07T01:01:47.326ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25882024-12-07T01:01:47.328ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
25892024-12-07T01:01:47.328ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25902024-12-07T01:01:47.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25912024-12-07T01:01:47.330ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25922024-12-07T01:01:47.331ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
25932024-12-07T01:01:47.331ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25942024-12-07T01:01:47.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25952024-12-07T01:01:47.365ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
25962024-12-07T01:01:47.365ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
25972024-12-07T01:01:47.365ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "31f6c1aa-d51e-42f8-9744-2de2274f860a", "date": "Sat, 07 Dec 2024 01:01:47 GMT"} })
25982024-12-07T01:01:47.365ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25992024-12-07T01:01:47.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26002024-12-07T01:01:47.368ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26012024-12-07T01:01:47.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26022024-12-07T01:01:47.371ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26032024-12-07T01:01:47.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26042024-12-07T01:01:47.374ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
26052024-12-07T01:01:47.374ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
26062024-12-07T01:01:47.374ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8a5274e8-bce8-49dc-bbad-e3b537e85214", "date": "Sat, 07 Dec 2024 01:01:47 GMT"} })
26072024-12-07T01:01:47.376ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 259.807743ms iteration = 31 status = {}
26082024-12-07T01:01:47.376ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
26092024-12-07T01:01:47.377ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c801a59-e5ec-47d1-9989-f107589c342b", "content-length": "691", "date": "Sat, 07 Dec 2024 01:01:47 GMT"} })
26102024-12-07T01:01:47.377ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
26112024-12-07T01:01:47.377ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
26122024-12-07T01:01:47.377ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
26132024-12-07T01:01:47.378ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 262.081548ms iteration = 30 status = {}
26142024-12-07T01:01:47.393ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26152024-12-07T01:01:47.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26162024-12-07T01:01:47.395ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 68b8f65f-54d5-4192-ac26-facf91340245, rack_id: 667e4910-296b-4cf9-a580-7c638e7aa45a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(e642095a-6c9c-4557-b65a-98313f61cfeb) } rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0
26172024-12-07T01:01:47.395ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26182024-12-07T01:01:47.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26192024-12-07T01:01:47.404ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
26202024-12-07T01:01:47.404ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "afd46a6a-0fd1-4845-a2ec-ab52f22cd7a3", "content-length": "134", "date": "Sat, 07 Dec 2024 01:01:47 GMT"} })
26212024-12-07T01:01:47.404ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
26222024-12-07T01:01:47.404ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = e642095a-6c9c-4557-b65a-98313f61cfeb
26232024-12-07T01:01:47.405ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
26242024-12-07T01:01:47.405ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
26252024-12-07T01:01:47.405ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26262024-12-07T01:01:47.405ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26272024-12-07T01:01:47.405ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26282024-12-07T01:01:47.405ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
26292024-12-07T01:01:47.405ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26302024-12-07T01:01:47.405ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
26312024-12-07T01:01:47.405ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
26322024-12-07T01:01:47.405ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26332024-12-07T01:01:47.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26342024-12-07T01:01:47.407ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
26352024-12-07T01:01:47.407ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd94ea54-77b0-45f0-a1aa-a34c95bbcd1e", "content-length": "2", "date": "Sat, 07 Dec 2024 01:01:47 GMT"} })
26362024-12-07T01:01:47.407ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
26372024-12-07T01:01:47.407ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3c8943d-ca85-4287-b4fc-fcc806352774", "content-length": "52", "date": "Sat, 07 Dec 2024 01:01:47 GMT"} })
26382024-12-07T01:01:47.407ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0
26392024-12-07T01:01:47.407ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
26402024-12-07T01:01:47.499ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "eb3a444b-ae50-48fd-8856-b753a1b0b172", "date": "Sat, 07 Dec 2024 01:01:47 GMT"} })
26412024-12-07T01:01:47.499ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch = Switch0
26422024-12-07T01:01:47.499ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
26432024-12-07T01:01:47.499ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "48053d03-fca2-4d87-9708-63cf540d6a56", "content-length": "682", "date": "Sat, 07 Dec 2024 01:01:47 GMT"} })
26442024-12-07T01:01:47.499ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26452024-12-07T01:01:47.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26462024-12-07T01:01:47.503ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26472024-12-07T01:01:47.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26482024-12-07T01:01:47.507ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26492024-12-07T01:01:47.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26502024-12-07T01:01:47.509ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26512024-12-07T01:01:47.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26522024-12-07T01:01:47.517ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 400.401535ms iteration = 28 status = {}
26532024-12-07T01:01:47.895ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
26542024-12-07T01:01:47.895ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::a]:12221
26552024-12-07T01:01:47.926ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
26562024-12-07T01:01:47.926ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:42346
26572024-12-07T01:01:47.927ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:42346 req_id = ed5c8027-9c10-428d-a6dd-3f680b802323 roles = RoleSet { roles: {} } uri = /metrics/producers
26582024-12-07T01:01:47.927ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:42346 req_id = ed5c8027-9c10-428d-a6dd-3f680b802323 resource = Database result = Ok(()) uri = /metrics/producers
26592024-12-07T01:01:47.964ZINFOnexus: assigned collector to new producer collector_id = 0c426bb3-2016-46d9-ae49-782d763f4e64 file = nexus/src/app/oximeter.rs:76 producer_id = 046d7edc-b85d-4313-a8cb-1398fbeee633
26602024-12-07T01:01:47.964ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37070 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:42346 req_id = ed5c8027-9c10-428d-a6dd-3f680b802323 response_code = 201 uri = /metrics/producers
26612024-12-07T01:01:47.964ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "ed5c8027-9c10-428d-a6dd-3f680b802323", "content-length": "41", "date": "Sat, 07 Dec 2024 01:01:47 GMT"} })
26622024-12-07T01:01:47.966ZDEBGnexus (producer-registration-task): registered with nexus successfully
26632024-12-07T01:01:47.966ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
26642024-12-07T01:01:57.116ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 83 reason = Timeout
26652024-12-07T01:01:57.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 83 reason = Timeout
26662024-12-07T01:01:57.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26672024-12-07T01:01:57.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26682024-12-07T01:01:57.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26692024-12-07T01:01:57.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26702024-12-07T01:01:57.118ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26712024-12-07T01:01:57.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26722024-12-07T01:01:57.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26732024-12-07T01:01:57.120ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.885346ms iteration = 83 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
26742024-12-07T01:01:57.121ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26752024-12-07T01:01:57.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26762024-12-07T01:01:57.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26772024-12-07T01:01:57.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26782024-12-07T01:01:57.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26792024-12-07T01:01:57.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26802024-12-07T01:01:57.128ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
26812024-12-07T01:01:57.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26822024-12-07T01:01:57.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26832024-12-07T01:01:57.138ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
26842024-12-07T01:01:57.138ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.80653ms iteration = 83 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
26852024-12-07T01:01:57.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 028f688d-fce2-46fa-932d-e44dc8d71ce6 roles = RoleSet { roles: {} } uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
26862024-12-07T01:01:57.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 028f688d-fce2-46fa-932d-e44dc8d71ce6 resource = Database result = Ok(()) uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
26872024-12-07T01:01:57.179ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2565 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 028f688d-fce2-46fa-932d-e44dc8d71ce6 response_code = 200 uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
26882024-12-07T01:01:57.179ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = b5587a38-de53-4785-b703-643341088ad5 roles = RoleSet { roles: {} } uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
26892024-12-07T01:01:57.180ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = b5587a38-de53-4785-b703-643341088ad5 resource = Database result = Ok(()) uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
26902024-12-07T01:01:57.181ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1950 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = b5587a38-de53-4785-b703-643341088ad5 response_code = 200 uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
26912024-12-07T01:01:57.186ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1840 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = 80a711e0-db4e-4e03-94b9-3b00ab6e458b response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
26922024-12-07T01:02:07.116ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 84 reason = Timeout
26932024-12-07T01:02:07.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 84 reason = Timeout
26942024-12-07T01:02:07.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26952024-12-07T01:02:07.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26962024-12-07T01:02:07.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26972024-12-07T01:02:07.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26982024-12-07T01:02:07.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26992024-12-07T01:02:07.119ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27002024-12-07T01:02:07.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27012024-12-07T01:02:07.121ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27022024-12-07T01:02:07.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27032024-12-07T01:02:07.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27042024-12-07T01:02:07.121ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.08015ms iteration = 84 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27052024-12-07T01:02:07.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27062024-12-07T01:02:07.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27072024-12-07T01:02:07.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27082024-12-07T01:02:07.127ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
27092024-12-07T01:02:07.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27102024-12-07T01:02:07.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27112024-12-07T01:02:07.139ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
27122024-12-07T01:02:07.139ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.546174ms iteration = 84 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
27132024-12-07T01:02:07.186ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2196 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = 2f233239-4a04-4928-96b4-d5229f20cdf4 response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
27142024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 17 reason = Timeout
27152024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 15 reason = Timeout
27162024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
27172024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
27182024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 31 reason = Timeout
27192024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27202024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 29 reason = Timeout
27212024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 29 reason = Timeout
27222024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
27232024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 32 reason = Timeout
27242024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27252024-12-07T01:02:17.116ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
27262024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27272024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 15 reason = Timeout
27282024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 29 reason = Timeout
27292024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
27302024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 15 reason = Timeout
27312024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 85 reason = Timeout
27322024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
27332024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 15 reason = Timeout
27342024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27352024-12-07T01:02:17.116ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-12-07 00:52:17.116623396 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
27362024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
27372024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 29 reason = Timeout
27382024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 29 reason = Timeout
27392024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
27402024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 15 reason = Timeout
27412024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
27422024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
27432024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 29 reason = Timeout
27442024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 29 reason = Timeout
27452024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
27462024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
27472024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
27482024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 15 reason = Timeout
27492024-12-07T01:02:17.116ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
27502024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
27512024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 29 reason = Timeout
27522024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
27532024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 29 reason = Timeout
27542024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
27552024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 16 reason = Timeout
27562024-12-07T01:02:17.116ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70
27572024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.22\u{b5}s iteration = 16 status = {"enabled":false,"target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70"}
27582024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 29 reason = Timeout
27592024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
27602024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 29 reason = Timeout
27612024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Timeout
27622024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27632024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 15 reason = Timeout
27642024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 15 reason = Timeout
27652024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 85 reason = Timeout
27662024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27672024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
27682024-12-07T01:02:17.116ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 17 reason = Timeout
27692024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 15 reason = Timeout
27702024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
27712024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
27722024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 15 reason = Timeout
27732024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
27742024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 29 reason = Timeout
27752024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27762024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
27772024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
27782024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27792024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
27802024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27812024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27822024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
27832024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
27842024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
27852024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
27862024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
27872024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
27882024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27892024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
27902024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
27912024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
27922024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
27932024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
27942024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27952024-12-07T01:02:17.117ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5661fac0-67b8-45f8-869a-67dd3e21bd0b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("be11a55e-cb67-4535-ad44-b9a4d4586728.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("fd72061d-6779-40de-8011-8e899981c9e7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("fd72061d-6779-40de-8011-8e899981c9e7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 4600, tv_nsec: 342662756 } })
27962024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
27972024-12-07T01:02:17.117ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("eda26d3e-18ec-4c21-8ff4-73abfd373bec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("fe4d45f4-708c-423b-9b9c-06a074a360b5.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("fe4d45f4-708c-423b-9b9c-06a074a360b5.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 4600, tv_nsec: 342669406 } })
27982024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
27992024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28002024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28012024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
28022024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
28032024-12-07T01:02:17.117ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.420168ms iteration = 15 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
28042024-12-07T01:02:17.118ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
28052024-12-07T01:02:17.118ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.213254ms iteration = 15 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
28062024-12-07T01:02:17.118ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
28072024-12-07T01:02:17.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
28082024-12-07T01:02:17.120ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.076261ms iteration = 29 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
28092024-12-07T01:02:17.122ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28102024-12-07T01:02:17.123ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-12-07 00:52:17.116623396 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
28112024-12-07T01:02:17.123ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 6.960127ms iteration = 15 status = {"expiration":"2024-12-07T00:52:17.116623396Z","pruned":[]}
28122024-12-07T01:02:17.124ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
28132024-12-07T01:02:17.124ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28142024-12-07T01:02:17.124ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4600, tv_nsec: 349442689 } })
28152024-12-07T01:02:17.124ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28162024-12-07T01:02:17.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28172024-12-07T01:02:17.124ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
28182024-12-07T01:02:17.125ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 9.296243ms iteration = 15 status = {"errors":[],"found_port_ok":[]}
28192024-12-07T01:02:17.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28202024-12-07T01:02:17.126ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
28212024-12-07T01:02:17.126ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
28222024-12-07T01:02:17.126ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4600, tv_nsec: 352182543 } })
28232024-12-07T01:02:17.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
28242024-12-07T01:02:17.127ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
28252024-12-07T01:02:17.129ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28262024-12-07T01:02:17.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28272024-12-07T01:02:17.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28282024-12-07T01:02:17.130ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28292024-12-07T01:02:17.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28302024-12-07T01:02:17.131ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
28312024-12-07T01:02:17.131ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28322024-12-07T01:02:17.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
28332024-12-07T01:02:17.131ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
28342024-12-07T01:02:17.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
28352024-12-07T01:02:17.133ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
28362024-12-07T01:02:17.133ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28372024-12-07T01:02:17.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28382024-12-07T01:02:17.134ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28392024-12-07T01:02:17.135ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
28402024-12-07T01:02:17.135ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
28412024-12-07T01:02:17.135ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 18.453314ms iteration = 15 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
28422024-12-07T01:02:17.135ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28432024-12-07T01:02:17.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28442024-12-07T01:02:17.136ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28452024-12-07T01:02:17.137ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
28462024-12-07T01:02:17.137ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
28472024-12-07T01:02:17.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
28482024-12-07T01:02:17.138ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
28492024-12-07T01:02:17.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28502024-12-07T01:02:17.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28512024-12-07T01:02:17.141ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28522024-12-07T01:02:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
28532024-12-07T01:02:17.149ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
28542024-12-07T01:02:17.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28552024-12-07T01:02:17.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
28562024-12-07T01:02:17.149ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 33.030331ms iteration = 29 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
28572024-12-07T01:02:17.150ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 33.633433ms iteration = 29 status = {"errors":[],"records_set_to_done":[]}
28582024-12-07T01:02:17.150ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28592024-12-07T01:02:17.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28602024-12-07T01:02:17.150ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28612024-12-07T01:02:17.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28622024-12-07T01:02:17.151ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28632024-12-07T01:02:17.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
28642024-12-07T01:02:17.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28652024-12-07T01:02:17.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28662024-12-07T01:02:17.152ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28672024-12-07T01:02:17.153ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28682024-12-07T01:02:17.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
28692024-12-07T01:02:17.154ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28702024-12-07T01:02:17.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28712024-12-07T01:02:17.155ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
28722024-12-07T01:02:17.155ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
28732024-12-07T01:02:17.155ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
28742024-12-07T01:02:17.155ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28752024-12-07T01:02:17.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
28762024-12-07T01:02:17.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28772024-12-07T01:02:17.158ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28782024-12-07T01:02:17.160ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
28792024-12-07T01:02:17.160ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28802024-12-07T01:02:17.160ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28812024-12-07T01:02:17.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28822024-12-07T01:02:17.162ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28832024-12-07T01:02:17.163ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28842024-12-07T01:02:17.163ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
28852024-12-07T01:02:17.163ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
28862024-12-07T01:02:17.163ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
28872024-12-07T01:02:17.163ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
28882024-12-07T01:02:17.163ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28892024-12-07T01:02:17.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
28902024-12-07T01:02:17.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28912024-12-07T01:02:17.164ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 47.859864ms iteration = 29 status = {"errors":[],"garbage_collect_requested":[]}
28922024-12-07T01:02:17.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
28932024-12-07T01:02:17.165ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
28942024-12-07T01:02:17.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
28952024-12-07T01:02:17.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
28962024-12-07T01:02:17.165ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28972024-12-07T01:02:17.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28982024-12-07T01:02:17.168ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4600, tv_nsec: 394059889 } })
28992024-12-07T01:02:17.169ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
29002024-12-07T01:02:17.171ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29012024-12-07T01:02:17.171ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29022024-12-07T01:02:17.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29032024-12-07T01:02:17.172ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
29042024-12-07T01:02:17.172ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 56.079456ms iteration = 85 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
29052024-12-07T01:02:17.172ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
29062024-12-07T01:02:17.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
29072024-12-07T01:02:17.173ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
29082024-12-07T01:02:17.173ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29092024-12-07T01:02:17.173ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
29102024-12-07T01:02:17.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29112024-12-07T01:02:17.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
29122024-12-07T01:02:17.175ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
29132024-12-07T01:02:17.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
29142024-12-07T01:02:17.176ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 59.37422ms iteration = 29 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
29152024-12-07T01:02:17.186ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1900 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = d0efd40e-86e3-421d-95e2-0069df752b79 response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
29162024-12-07T01:02:17.187ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
29172024-12-07T01:02:17.187ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 71.089492ms iteration = 29 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
29182024-12-07T01:02:17.187ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
29192024-12-07T01:02:17.187ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
29202024-12-07T01:02:17.187ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
29212024-12-07T01:02:17.187ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29222024-12-07T01:02:17.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29232024-12-07T01:02:17.190ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29242024-12-07T01:02:17.192ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
29252024-12-07T01:02:17.192ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29262024-12-07T01:02:17.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29272024-12-07T01:02:17.193ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 76.235783ms iteration = 85 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
29282024-12-07T01:02:17.193ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
29292024-12-07T01:02:17.193ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
29302024-12-07T01:02:17.193ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
29312024-12-07T01:02:17.193ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29322024-12-07T01:02:17.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29332024-12-07T01:02:17.196ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29342024-12-07T01:02:17.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29352024-12-07T01:02:17.200ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
29362024-12-07T01:02:17.200ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
29372024-12-07T01:02:17.200ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
29382024-12-07T01:02:17.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
29392024-12-07T01:02:17.202ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29402024-12-07T01:02:17.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29412024-12-07T01:02:17.207ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 90.830931ms iteration = 29 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
29422024-12-07T01:02:17.211ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
29432024-12-07T01:02:17.212ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
29442024-12-07T01:02:17.212ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
29452024-12-07T01:02:17.212ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 95.3189ms iteration = 15 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
29462024-12-07T01:02:17.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
29472024-12-07T01:02:17.230ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
29482024-12-07T01:02:17.231ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
29492024-12-07T01:02:17.238ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
29502024-12-07T01:02:17.238ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29512024-12-07T01:02:17.239ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b273e4b-b511-4e55-82ec-0e3d87c4a91d", "content-length": "26", "date": "Sat, 07 Dec 2024 01:02:16 GMT"} })
29522024-12-07T01:02:17.239ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29532024-12-07T01:02:17.240ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29542024-12-07T01:02:17.271ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 154.373535ms iteration = 31 status = {"physical_disks_added":0}
29552024-12-07T01:02:17.302ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
29562024-12-07T01:02:17.302ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29572024-12-07T01:02:17.302ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ae2d8f9-f15d-4f58-a4a4-a2155e819be8", "content-length": "26", "date": "Sat, 07 Dec 2024 01:02:16 GMT"} })
29582024-12-07T01:02:17.302ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29592024-12-07T01:02:17.306ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29602024-12-07T01:02:17.316ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a zone_address = fd00:1122:3344:101::2
29612024-12-07T01:02:17.316ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29622024-12-07T01:02:17.316ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "51e61968-65a3-4ba8-b061-dae60ebc5d8e", "content-length": "26", "date": "Sat, 07 Dec 2024 01:02:16 GMT"} })
29632024-12-07T01:02:17.316ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29642024-12-07T01:02:17.318ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
29652024-12-07T01:02:17.367ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
29662024-12-07T01:02:17.367ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
29672024-12-07T01:02:17.369ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29682024-12-07T01:02:17.369ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
29692024-12-07T01:02:17.369ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 52ba8747-d0f6-43d9-9ec0-995aa6df639a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
29702024-12-07T01:02:17.369ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29712024-12-07T01:02:17.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29722024-12-07T01:02:17.370ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29732024-12-07T01:02:17.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29742024-12-07T01:02:17.372ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
29752024-12-07T01:02:17.372ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29762024-12-07T01:02:17.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29772024-12-07T01:02:17.401ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29782024-12-07T01:02:17.401ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29792024-12-07T01:02:17.403ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
29802024-12-07T01:02:17.403ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29812024-12-07T01:02:17.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29822024-12-07T01:02:17.404ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
29832024-12-07T01:02:17.404ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29842024-12-07T01:02:17.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29852024-12-07T01:02:17.405ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29862024-12-07T01:02:17.407ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 52ba8747-d0f6-43d9-9ec0-995aa6df639a dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
29872024-12-07T01:02:17.407ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
29882024-12-07T01:02:17.407ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-12-07 00:48:44.409012 UTC dns_group = internal generation = 1
29892024-12-07T01:02:17.407ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 290.640711ms iteration = 15 status = {"generation":1}
29902024-12-07T01:02:17.407ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3, lookup_type: ById(e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3) } result = Ok(())
29912024-12-07T01:02:17.407ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29922024-12-07T01:02:17.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29932024-12-07T01:02:17.408ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
29942024-12-07T01:02:17.408ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ae6d4ace-ffdd-4139-9e11-1794ae966042 dns_zone_name = oxide.test found_so_far = 0 version = 2
29952024-12-07T01:02:17.408ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29962024-12-07T01:02:17.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29972024-12-07T01:02:17.409ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
29982024-12-07T01:02:17.409ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "de157f88-46d1-467f-bdae-a02bed995af4", "content-length": "2", "date": "Sat, 07 Dec 2024 01:02:17 GMT"} })
29992024-12-07T01:02:17.410ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 293.176912ms iteration = 29 status = 0
30002024-12-07T01:02:17.410ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30012024-12-07T01:02:17.411ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
30022024-12-07T01:02:17.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
30032024-12-07T01:02:17.412ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
30042024-12-07T01:02:17.412ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
30052024-12-07T01:02:17.412ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 295.441545ms iteration = 29 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
30062024-12-07T01:02:17.412ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
30072024-12-07T01:02:17.412ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30082024-12-07T01:02:17.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30092024-12-07T01:02:17.414ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 298.087427ms iteration = 29 status = null
30102024-12-07T01:02:17.415ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ae6d4ace-ffdd-4139-9e11-1794ae966042 dns_zone_name = oxide.test found_so_far = 1 version = 2
30112024-12-07T01:02:17.415ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
30122024-12-07T01:02:17.415ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-12-07 00:48:45.853224 UTC dns_group = external generation = 2
30132024-12-07T01:02:17.415ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 299.20152ms iteration = 15 status = {"generation":2}
30142024-12-07T01:02:17.432ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
30152024-12-07T01:02:17.435ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
30162024-12-07T01:02:17.435ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "4da64ac3-6673-4bfa-b014-81aa4123986c", "date": "Sat, 07 Dec 2024 01:02:16 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30172024-12-07T01:02:17.435ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30182024-12-07T01:02:17.437ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "5e4161f2-45b4-456e-891e-4cb891c06f46", "date": "Sat, 07 Dec 2024 01:02:16 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30192024-12-07T01:02:17.437ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30202024-12-07T01:02:17.437ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b12877ea-3b1b-4c68-a704-16a043b20220", "content-length": "372", "date": "Sat, 07 Dec 2024 01:02:16 GMT"} })
30212024-12-07T01:02:17.437ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30222024-12-07T01:02:17.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30232024-12-07T01:02:17.439ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30242024-12-07T01:02:17.439ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 322.747625ms iteration = 17 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
30252024-12-07T01:02:17.449ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-12-07T00:48:09.490271Z, time_modified: 2024-12-07T00:48:09.490271Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, db_silo: Silo { identity: SiloIdentity { id: 3a67a720-d4af-42e7-875a-3abed4955db5, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-07T00:48:45.569718Z, time_modified: 2024-12-07T00:48:45.569718Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6317ba4eb1538be6191eb18b3b3531d70dbee51545c790c2aff24b81494c24f4" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 3a67a720-d4af-42e7-875a-3abed4955db5, db_silo: Silo { identity: SiloIdentity { id: 3a67a720-d4af-42e7-875a-3abed4955db5, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-12-07T00:48:45.569718Z, time_modified: 2024-12-07T00:48:45.569718Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6317ba4eb1538be6191eb18b3b3531d70dbee51545c790c2aff24b81494c24f4" }] }) }
30262024-12-07T01:02:17.449ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 332.487896ms iteration = 15 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"3a67a720-d4af-42e7-875a-3abed4955db5","tls_certs":["6317ba4eb1538be6191eb18b3b3531d70dbee51545c790c2aff24b81494c24f4"]}},"default_endpoint":{"silo_id":"3a67a720-d4af-42e7-875a-3abed4955db5","tls_certs":["6317ba4eb1538be6191eb18b3b3531d70dbee51545c790c2aff24b81494c24f4"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
30272024-12-07T01:02:17.452ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 335.468045ms iteration = 29 status = {}
30282024-12-07T01:02:17.455ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
30292024-12-07T01:02:17.455ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
30302024-12-07T01:02:17.455ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "de63ee23-6d8c-4b65-9307-2dd30fd3cf82", "date": "Sat, 07 Dec 2024 01:02:17 GMT"} })
30312024-12-07T01:02:17.455ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30322024-12-07T01:02:17.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30332024-12-07T01:02:17.458ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30342024-12-07T01:02:17.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30352024-12-07T01:02:17.461ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30362024-12-07T01:02:17.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30372024-12-07T01:02:17.464ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
30382024-12-07T01:02:17.464ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30392024-12-07T01:02:17.464ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "0eb8de75-4e76-402b-b6c8-4bc4b09d7cda", "date": "Sat, 07 Dec 2024 01:02:17 GMT"} })
30402024-12-07T01:02:17.466ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 350.389139ms iteration = 32 status = {}
30412024-12-07T01:02:17.478ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
30422024-12-07T01:02:17.478ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 361.778564ms iteration = 15 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
30432024-12-07T01:02:17.519ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
30442024-12-07T01:02:17.519ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "68088710-fea5-43a7-b7df-c28c98bd819a", "content-length": "691", "date": "Sat, 07 Dec 2024 01:02:17 GMT"} })
30452024-12-07T01:02:17.519ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
30462024-12-07T01:02:17.519ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
30472024-12-07T01:02:17.519ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
30482024-12-07T01:02:17.521ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 404.831943ms iteration = 31 status = {}
30492024-12-07T01:02:17.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
30502024-12-07T01:02:17.523ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "e04837bd-6de9-4c97-8a05-f8eaaa110a0d", "date": "Sat, 07 Dec 2024 01:02:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
30512024-12-07T01:02:17.523ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
30522024-12-07T01:02:17.524ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "b7d1c953-ef29-4991-bb31-4526aaba6571", "date": "Sat, 07 Dec 2024 01:02:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
30532024-12-07T01:02:17.524ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
30542024-12-07T01:02:17.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "7bc36e3b-8b0e-4b30-816f-bff7050964bf", "date": "Sat, 07 Dec 2024 01:02:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
30552024-12-07T01:02:17.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
30562024-12-07T01:02:17.528ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
30572024-12-07T01:02:17.528ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 412.173528ms iteration = 17 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
30582024-12-07T01:02:17.587ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30592024-12-07T01:02:17.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30602024-12-07T01:02:17.589ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 68b8f65f-54d5-4192-ac26-facf91340245, rack_id: 667e4910-296b-4cf9-a580-7c638e7aa45a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(e642095a-6c9c-4557-b65a-98313f61cfeb) } rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0
30612024-12-07T01:02:17.589ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30622024-12-07T01:02:17.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30632024-12-07T01:02:17.599ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
30642024-12-07T01:02:17.599ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbd7f43b-0b41-46df-aca3-dc12d12c8643", "content-length": "134", "date": "Sat, 07 Dec 2024 01:02:17 GMT"} })
30652024-12-07T01:02:17.599ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
30662024-12-07T01:02:17.599ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = e642095a-6c9c-4557-b65a-98313f61cfeb
30672024-12-07T01:02:17.599ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
30682024-12-07T01:02:17.599ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
30692024-12-07T01:02:17.599ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30702024-12-07T01:02:17.599ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30712024-12-07T01:02:17.599ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30722024-12-07T01:02:17.599ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
30732024-12-07T01:02:17.599ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30742024-12-07T01:02:17.599ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
30752024-12-07T01:02:17.600ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
30762024-12-07T01:02:17.600ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30772024-12-07T01:02:17.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30782024-12-07T01:02:17.602ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
30792024-12-07T01:02:17.602ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f630bd5-441b-4801-b68b-487497502f74", "content-length": "2", "date": "Sat, 07 Dec 2024 01:02:17 GMT"} })
30802024-12-07T01:02:17.602ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
30812024-12-07T01:02:17.602ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb44df94-aaef-423b-a084-af6587c784c5", "content-length": "52", "date": "Sat, 07 Dec 2024 01:02:17 GMT"} })
30822024-12-07T01:02:17.602ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0
30832024-12-07T01:02:17.602ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
30842024-12-07T01:02:17.693ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8b0537fc-e110-4eba-948d-70e52f7b3c03", "date": "Sat, 07 Dec 2024 01:02:17 GMT"} })
30852024-12-07T01:02:17.693ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch = Switch0
30862024-12-07T01:02:17.693ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
30872024-12-07T01:02:17.693ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e69a68cc-9b0f-4ecb-bc11-8c2698743a81", "content-length": "682", "date": "Sat, 07 Dec 2024 01:02:17 GMT"} })
30882024-12-07T01:02:17.693ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30892024-12-07T01:02:17.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30902024-12-07T01:02:17.697ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30912024-12-07T01:02:17.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30922024-12-07T01:02:17.700ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30932024-12-07T01:02:17.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30942024-12-07T01:02:17.702ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30952024-12-07T01:02:17.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30962024-12-07T01:02:17.709ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 593.078325ms iteration = 29 status = {}
30972024-12-07T01:02:27.116ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 86 reason = Timeout
30982024-12-07T01:02:27.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 86 reason = Timeout
30992024-12-07T01:02:27.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31002024-12-07T01:02:27.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31012024-12-07T01:02:27.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31022024-12-07T01:02:27.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31032024-12-07T01:02:27.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31042024-12-07T01:02:27.121ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31052024-12-07T01:02:27.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31062024-12-07T01:02:27.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31072024-12-07T01:02:27.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31082024-12-07T01:02:27.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31092024-12-07T01:02:27.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31102024-12-07T01:02:27.129ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
31112024-12-07T01:02:27.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31122024-12-07T01:02:27.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31132024-12-07T01:02:27.141ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
31142024-12-07T01:02:27.141ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.543084ms iteration = 86 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
31152024-12-07T01:02:27.151ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31162024-12-07T01:02:27.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31172024-12-07T01:02:27.153ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.51812ms iteration = 86 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31182024-12-07T01:02:27.176ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 2669584e-e436-4c5e-b2e0-40abb946d293 roles = RoleSet { roles: {} } uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
31192024-12-07T01:02:27.176ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 2669584e-e436-4c5e-b2e0-40abb946d293 resource = Database result = Ok(()) uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
31202024-12-07T01:02:27.178ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2240 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 2669584e-e436-4c5e-b2e0-40abb946d293 response_code = 200 uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?sort_by=id_ascending
31212024-12-07T01:02:27.179ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 7b8b165d-98aa-418d-8276-9ee0bcb7481c roles = RoleSet { roles: {} } uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
31222024-12-07T01:02:27.179ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 7b8b165d-98aa-418d-8276-9ee0bcb7481c resource = Database result = Ok(()) uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
31232024-12-07T01:02:27.181ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2187 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34061 req_id = 7b8b165d-98aa-418d-8276-9ee0bcb7481c response_code = 200 uri = /metrics/collectors/0c426bb3-2016-46d9-ae49-782d763f4e64/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOTYyZWNlZS0zMmM4LTRjZTUtYjQ5NS1iMmE5ZTRjZjE2MWEifX0%3D
31242024-12-07T01:02:27.185ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1728 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = 2aebee92-cea0-4e3f-a29c-ae5c0a6cc771 response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
31252024-12-07T01:02:37.116ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 87 reason = Timeout
31262024-12-07T01:02:37.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 87 reason = Timeout
31272024-12-07T01:02:37.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31282024-12-07T01:02:37.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31292024-12-07T01:02:37.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31302024-12-07T01:02:37.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31312024-12-07T01:02:37.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31322024-12-07T01:02:37.121ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31332024-12-07T01:02:37.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31342024-12-07T01:02:37.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31352024-12-07T01:02:37.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31362024-12-07T01:02:37.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31372024-12-07T01:02:37.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31382024-12-07T01:02:37.130ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
31392024-12-07T01:02:37.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31402024-12-07T01:02:37.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31412024-12-07T01:02:37.140ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
31422024-12-07T01:02:37.140ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.641646ms iteration = 87 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
31432024-12-07T01:02:37.152ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31442024-12-07T01:02:37.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31452024-12-07T01:02:37.180ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.411071ms iteration = 87 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31462024-12-07T01:02:37.186ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2084 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = 7067a132-d6c5-460f-ba87-a39561f2c0d4 response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
31472024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 30 reason = Timeout
31482024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 30 reason = Timeout
31492024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
31502024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 30 reason = Timeout
31512024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31522024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
31532024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 32 reason = Timeout
31542024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 30 reason = Timeout
31552024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31562024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
31572024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 88 reason = Timeout
31582024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 30 reason = Timeout
31592024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
31602024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
31612024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 30 reason = Timeout
31622024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31632024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 88 reason = Timeout
31642024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31652024-12-07T01:02:47.116ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31662024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 30 reason = Timeout
31672024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
31682024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 33 reason = Timeout
31692024-12-07T01:02:47.117ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
31702024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31712024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 32 reason = Timeout
31722024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31732024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 30 reason = Timeout
31742024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
31752024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 30 reason = Timeout
31762024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
31772024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 30 reason = Timeout
31782024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
31792024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 30 reason = Timeout
31802024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
31812024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 30 reason = Timeout
31822024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31832024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
31842024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
31852024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
31862024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31872024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31882024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31892024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31902024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
31912024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
31922024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
31932024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
31942024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31952024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
31962024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
31972024-12-07T01:02:47.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31982024-12-07T01:02:47.119ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31992024-12-07T01:02:47.120ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4630, tv_nsec: 345482255 } })
32002024-12-07T01:02:47.120ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1103
32012024-12-07T01:02:47.120ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.879356ms iteration = 30 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
32022024-12-07T01:02:47.121ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
32032024-12-07T01:02:47.121ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32042024-12-07T01:02:47.121ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32052024-12-07T01:02:47.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32062024-12-07T01:02:47.123ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
32072024-12-07T01:02:47.123ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32082024-12-07T01:02:47.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32092024-12-07T01:02:47.127ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4630, tv_nsec: 352493064 } })
32102024-12-07T01:02:47.127ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1103
32112024-12-07T01:02:47.127ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32122024-12-07T01:02:47.129ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
32132024-12-07T01:02:47.129ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
32142024-12-07T01:02:47.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
32152024-12-07T01:02:47.138ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32162024-12-07T01:02:47.138ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
32172024-12-07T01:02:47.138ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 21.367372ms iteration = 30 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
32182024-12-07T01:02:47.138ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 4630, tv_nsec: 363734085 } })
32192024-12-07T01:02:47.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32202024-12-07T01:02:47.138ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1103 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
32212024-12-07T01:02:47.159ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32222024-12-07T01:02:47.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32232024-12-07T01:02:47.160ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 43.346124ms iteration = 30 status = {"errors":[],"records_set_to_done":[]}
32242024-12-07T01:02:47.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32252024-12-07T01:02:47.164ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
32262024-12-07T01:02:47.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
32272024-12-07T01:02:47.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
32282024-12-07T01:02:47.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32292024-12-07T01:02:47.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32302024-12-07T01:02:47.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32312024-12-07T01:02:47.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32322024-12-07T01:02:47.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32332024-12-07T01:02:47.170ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32342024-12-07T01:02:47.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32352024-12-07T01:02:47.171ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 54.466364ms iteration = 30 status = {"errors":[],"garbage_collect_requested":[]}
32362024-12-07T01:02:47.173ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
32372024-12-07T01:02:47.173ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
32382024-12-07T01:02:47.173ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
32392024-12-07T01:02:47.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 33cdb3e1-1067-4513-ae76-65cb52da1a70, lookup_type: ById(33cdb3e1-1067-4513-ae76-65cb52da1a70) } result = Ok(())
32402024-12-07T01:02:47.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32412024-12-07T01:02:47.173ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32422024-12-07T01:02:47.173ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32432024-12-07T01:02:47.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32442024-12-07T01:02:47.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
32452024-12-07T01:02:47.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32462024-12-07T01:02:47.174ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
32472024-12-07T01:02:47.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
32482024-12-07T01:02:47.175ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
32492024-12-07T01:02:47.175ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32502024-12-07T01:02:47.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32512024-12-07T01:02:47.175ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32522024-12-07T01:02:47.176ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32532024-12-07T01:02:47.176ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
32542024-12-07T01:02:47.176ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
32552024-12-07T01:02:47.176ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
32562024-12-07T01:02:47.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
32572024-12-07T01:02:47.178ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
32582024-12-07T01:02:47.178ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32592024-12-07T01:02:47.178ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3, lookup_type: ById(e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3) } result = Ok(())
32602024-12-07T01:02:47.178ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32612024-12-07T01:02:47.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32622024-12-07T01:02:47.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32632024-12-07T01:02:47.179ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
32642024-12-07T01:02:47.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
32652024-12-07T01:02:47.180ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 63.449411ms iteration = 30 status = 0
32662024-12-07T01:02:47.186ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2405 local_addr = [fd00:1122:3344:101::a]:36607 method = GET remote_addr = [fd00:1122:3344:101::d]:36600 req_id = dbc78214-2fd8-485d-bb53-38dcb9080aef response_code = 200 uri = /046d7edc-b85d-4313-a8cb-1398fbeee633
32672024-12-07T01:02:47.187ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 original_time_created = 2024-12-07 00:48:03.718366 UTC target_id = 33cdb3e1-1067-4513-ae76-65cb52da1a70 time_created = 2024-12-07 00:48:03.718366 UTC
32682024-12-07T01:02:47.187ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 70.725895ms iteration = 88 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"33cdb3e1-1067-4513-ae76-65cb52da1a70","time_created":"2024-12-07T00:48:03.718366Z"}
32692024-12-07T01:02:47.189ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32702024-12-07T01:02:47.191ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
32712024-12-07T01:02:47.191ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32722024-12-07T01:02:47.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32732024-12-07T01:02:47.193ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
32742024-12-07T01:02:47.193ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32752024-12-07T01:02:47.193ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "19537e27-cc1e-4ce8-992e-6f262908b766", "content-length": "26", "date": "Sat, 07 Dec 2024 01:02:46 GMT"} })
32762024-12-07T01:02:47.193ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32772024-12-07T01:02:47.195ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32782024-12-07T01:02:47.198ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 81.328594ms iteration = 88 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
32792024-12-07T01:02:47.200ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32802024-12-07T01:02:47.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32812024-12-07T01:02:47.203ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
32822024-12-07T01:02:47.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
32832024-12-07T01:02:47.212ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1111 zone_address = fd00:1122:3344:101::2
32842024-12-07T01:02:47.212ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32852024-12-07T01:02:47.212ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6470345-d265-47ea-9485-5452cfc43ac9", "content-length": "26", "date": "Sat, 07 Dec 2024 01:02:47 GMT"} })
32862024-12-07T01:02:47.212ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1116 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32872024-12-07T01:02:47.213ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 96.520563ms iteration = 30 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
32882024-12-07T01:02:47.215ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32892024-12-07T01:02:47.220ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 103.517381ms iteration = 30 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
32902024-12-07T01:02:47.223ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
32912024-12-07T01:02:47.223ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
32922024-12-07T01:02:47.223ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 106.729905ms iteration = 30 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
32932024-12-07T01:02:47.234ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32942024-12-07T01:02:47.235ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1111 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a zone_address = fd00:1122:3344:101::2
32952024-12-07T01:02:47.235ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32962024-12-07T01:02:47.235ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1875618-3ba3-4316-9bae-c1bddbeac27f", "content-length": "26", "date": "Sat, 07 Dec 2024 01:02:47 GMT"} })
32972024-12-07T01:02:47.235ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1116 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32982024-12-07T01:02:47.236ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
32992024-12-07T01:02:47.236ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33002024-12-07T01:02:47.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
33012024-12-07T01:02:47.237ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1148 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
33022024-12-07T01:02:47.240ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 123.549986ms iteration = 30 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
33032024-12-07T01:02:47.247ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33042024-12-07T01:02:47.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33052024-12-07T01:02:47.272ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 155.779542ms iteration = 32 status = {"physical_disks_added":0}
33062024-12-07T01:02:47.304ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 187.361925ms iteration = 30 status = {}
33072024-12-07T01:02:47.320ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
33082024-12-07T01:02:47.320ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d378f1a-b537-4616-b1be-a1d84d63643a", "content-length": "2", "date": "Sat, 07 Dec 2024 01:02:46 GMT"} })
33092024-12-07T01:02:47.321ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
33102024-12-07T01:02:47.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
33112024-12-07T01:02:47.323ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 207.138065ms iteration = 30 status = null
33122024-12-07T01:02:47.335ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33132024-12-07T01:02:47.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33142024-12-07T01:02:47.341ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
33152024-12-07T01:02:47.341ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
33162024-12-07T01:02:47.341ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e80357d8-4bd0-4c49-85bb-259417535c27", "content-length": "372", "date": "Sat, 07 Dec 2024 01:02:47 GMT"} })
33172024-12-07T01:02:47.341ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33182024-12-07T01:02:47.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33192024-12-07T01:02:47.370ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33202024-12-07T01:02:47.372ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
33212024-12-07T01:02:47.372ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33222024-12-07T01:02:47.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33232024-12-07T01:02:47.374ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33242024-12-07T01:02:47.375ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
33252024-12-07T01:02:47.375ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33262024-12-07T01:02:47.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33272024-12-07T01:02:47.387ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
33282024-12-07T01:02:47.387ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
33292024-12-07T01:02:47.387ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "65e0853d-ab37-4b28-9546-b2ea06e1d3e4", "date": "Sat, 07 Dec 2024 01:02:47 GMT"} })
33302024-12-07T01:02:47.387ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33312024-12-07T01:02:47.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33322024-12-07T01:02:47.389ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33332024-12-07T01:02:47.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33342024-12-07T01:02:47.393ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33352024-12-07T01:02:47.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33362024-12-07T01:02:47.396ZINFOnexus (BackgroundTasks): VPC route manager sled e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
33372024-12-07T01:02:47.396ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
33382024-12-07T01:02:47.397ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "78c00eb3-85e9-456e-b287-667371e27ba7", "date": "Sat, 07 Dec 2024 01:02:47 GMT"} })
33392024-12-07T01:02:47.398ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 281.840428ms iteration = 33 status = {}
33402024-12-07T01:02:47.443ZDEBGnexus (BackgroundTasks): client request SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
33412024-12-07T01:02:47.443ZDEBGnexus (BackgroundTasks): client response SledAgent = e6c44bf0-8a98-4b5e-823b-bbdeea63c1d3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "73152ede-03c6-4386-a4fc-1d75d31fbdde", "content-length": "691", "date": "Sat, 07 Dec 2024 01:02:47 GMT"} })
33422024-12-07T01:02:47.443ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
33432024-12-07T01:02:47.443ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
33442024-12-07T01:02:47.443ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
33452024-12-07T01:02:47.445ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 328.298024ms iteration = 32 status = {}
33462024-12-07T01:02:47.450ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33472024-12-07T01:02:47.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33482024-12-07T01:02:47.453ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 68b8f65f-54d5-4192-ac26-facf91340245, rack_id: 667e4910-296b-4cf9-a580-7c638e7aa45a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(e642095a-6c9c-4557-b65a-98313f61cfeb) } rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0
33492024-12-07T01:02:47.453ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33502024-12-07T01:02:47.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33512024-12-07T01:02:47.463ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
33522024-12-07T01:02:47.463ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "8998b26f-4379-4be0-a8dc-f5269f907fcb", "content-length": "134", "date": "Sat, 07 Dec 2024 01:02:47 GMT"} })
33532024-12-07T01:02:47.463ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
33542024-12-07T01:02:47.463ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = e642095a-6c9c-4557-b65a-98313f61cfeb
33552024-12-07T01:02:47.463ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
33562024-12-07T01:02:47.463ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
33572024-12-07T01:02:47.463ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33582024-12-07T01:02:47.463ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33592024-12-07T01:02:47.463ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33602024-12-07T01:02:47.463ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
33612024-12-07T01:02:47.464ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33622024-12-07T01:02:47.464ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
33632024-12-07T01:02:47.464ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a
33642024-12-07T01:02:47.464ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33652024-12-07T01:02:47.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33662024-12-07T01:02:47.466ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
33672024-12-07T01:02:47.466ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "187d26bb-2ea3-4d45-b296-e6931c2bf2d9", "content-length": "2", "date": "Sat, 07 Dec 2024 01:02:47 GMT"} })
33682024-12-07T01:02:47.466ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
33692024-12-07T01:02:47.466ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "959ac767-c967-4152-8f43-1d2767f4116d", "content-length": "52", "date": "Sat, 07 Dec 2024 01:02:47 GMT"} })
33702024-12-07T01:02:47.466ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch_location = Switch0
33712024-12-07T01:02:47.466ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
33722024-12-07T01:02:47.554ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "a489d090-2782-4609-a24e-0191c3b65278", "date": "Sat, 07 Dec 2024 01:02:47 GMT"} })
33732024-12-07T01:02:47.554ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a switch = Switch0
33742024-12-07T01:02:47.554ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
33752024-12-07T01:02:47.555ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 667e4910-296b-4cf9-a580-7c638e7aa45a result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0498424c-60c2-4a19-9253-fcd468e81405", "content-length": "682", "date": "Sat, 07 Dec 2024 01:02:47 GMT"} })
33762024-12-07T01:02:47.555ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33772024-12-07T01:02:47.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33782024-12-07T01:02:47.559ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33792024-12-07T01:02:47.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33802024-12-07T01:02:47.563ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33812024-12-07T01:02:47.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33822024-12-07T01:02:47.564ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33832024-12-07T01:02:47.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33842024-12-07T01:02:47.572ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 455.993112ms iteration = 30 status = {}