Max level shown:
12024-03-28T08:15:04.870ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 5 reason = Timeout
22024-03-28T08:15:04.870ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
32024-03-28T08:15:04.870ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
42024-03-28T08:15:04.870ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
52024-03-28T08:15:04.870ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
62024-03-28T08:15:04.870ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
72024-03-28T08:15:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
82024-03-28T08:15:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
92024-03-28T08:15:04.870ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
102024-03-28T08:15:04.870ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 5 reason = Timeout
112024-03-28T08:15:04.870ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
122024-03-28T08:15:04.870ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
132024-03-28T08:15:04.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 11 reason = Timeout
142024-03-28T08:15:04.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
152024-03-28T08:15:04.870ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 5 reason = Timeout
162024-03-28T08:15:04.870ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 4.68\u{b5}s iteration = 5
172024-03-28T08:15:04.870ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
182024-03-28T08:15:04.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
192024-03-28T08:15:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
202024-03-28T08:15:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
212024-03-28T08:15:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
222024-03-28T08:15:04.872ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2634, tv_nsec: 428783801 } })
232024-03-28T08:15:04.872ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242024-03-28T08:15:04.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
252024-03-28T08:15:04.873ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
262024-03-28T08:15:04.873ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.512964ms iteration = 5
272024-03-28T08:15:04.873ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
282024-03-28T08:15:04.873ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2634, tv_nsec: 429801479 } })
292024-03-28T08:15:04.873ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
302024-03-28T08:15:04.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
312024-03-28T08:15:04.875ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
322024-03-28T08:15:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
332024-03-28T08:15:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
342024-03-28T08:15:04.876ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
352024-03-28T08:15:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
362024-03-28T08:15:04.876ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
372024-03-28T08:15:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
382024-03-28T08:15:04.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
392024-03-28T08:15:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
402024-03-28T08:15:04.877ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
412024-03-28T08:15:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
422024-03-28T08:15:04.878ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2634, tv_nsec: 434782656 } })
432024-03-28T08:15:04.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
442024-03-28T08:15:04.878ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
452024-03-28T08:15:04.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
462024-03-28T08:15:04.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
472024-03-28T08:15:04.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
482024-03-28T08:15:04.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
492024-03-28T08:15:04.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
502024-03-28T08:15:04.886ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
512024-03-28T08:15:04.886ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.981898ms iteration = 11
522024-03-28T08:15:04.887ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
532024-03-28T08:15:04.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
542024-03-28T08:15:04.888ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
552024-03-28T08:15:04.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
562024-03-28T08:15:04.889ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
572024-03-28T08:15:04.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
582024-03-28T08:15:04.892ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
592024-03-28T08:15:04.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
602024-03-28T08:15:04.893ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 22.719254ms iteration = 5
612024-03-28T08:15:04.954ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
622024-03-28T08:15:04.954ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
632024-03-28T08:15:04.954ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "99c27b4b-a39b-466d-be7a-07fbbd440232", "content-length": "26", "date": "Thu, 28 Mar 2024 08:15:04 GMT"} })
642024-03-28T08:15:04.954ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
652024-03-28T08:15:04.956ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
662024-03-28T08:15:05.037ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 166.935437ms iteration = 5
672024-03-28T08:15:05.049ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
682024-03-28T08:15:05.049ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
692024-03-28T08:15:05.049ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2c51a5db-4050-4896-b518-09ef4c492d4a", "content-length": "26", "date": "Thu, 28 Mar 2024 08:15:04 GMT"} })
702024-03-28T08:15:05.049ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
712024-03-28T08:15:05.051ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
722024-03-28T08:15:05.053ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
732024-03-28T08:15:05.053ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
742024-03-28T08:15:05.054ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5033c8eb-8402-4f14-9e2e-a05e40c92381", "content-length": "26", "date": "Thu, 28 Mar 2024 08:15:04 GMT"} })
752024-03-28T08:15:05.054ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
762024-03-28T08:15:05.056ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
772024-03-28T08:15:05.102ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
782024-03-28T08:15:05.103ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "43efa05d-8318-4e8a-ad8f-d6070d083e80", "content-length": "1", "date": "Thu, 28 Mar 2024 08:15:04 GMT"} })
792024-03-28T08:15:05.104ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
802024-03-28T08:15:05.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
812024-03-28T08:15:05.106ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 235.994284ms iteration = 5
822024-03-28T08:15:05.165ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
832024-03-28T08:15:05.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
842024-03-28T08:15:05.166ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
852024-03-28T08:15:05.166ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
862024-03-28T08:15:05.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
872024-03-28T08:15:05.173ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
882024-03-28T08:15:05.173ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2575eeae-5748-49a0-b024-f2c7dfb138ed", "content-length": "121", "date": "Thu, 28 Mar 2024 08:15:04 GMT"} })
892024-03-28T08:15:05.173ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
902024-03-28T08:15:05.173ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
912024-03-28T08:15:05.174ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
922024-03-28T08:15:05.174ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
932024-03-28T08:15:05.174ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
942024-03-28T08:15:05.174ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
952024-03-28T08:15:05.174ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
962024-03-28T08:15:05.174ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
972024-03-28T08:15:05.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
982024-03-28T08:15:05.176ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
992024-03-28T08:15:05.176ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ae99ce34-1320-4909-b82c-6cafe140c0d4", "content-length": "2", "date": "Thu, 28 Mar 2024 08:15:04 GMT"} })
1002024-03-28T08:15:05.176ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
1012024-03-28T08:15:05.176ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "89a2a92f-2614-4907-9474-8467de537069", "content-length": "52", "date": "Thu, 28 Mar 2024 08:15:04 GMT"} })
1022024-03-28T08:15:05.176ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
1032024-03-28T08:15:05.176ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
1042024-03-28T08:15:05.220ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "85e33724-c640-4c92-b529-0c6505784bea", "date": "Thu, 28 Mar 2024 08:15:05 GMT"} })
1052024-03-28T08:15:05.220ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
1062024-03-28T08:15:05.220ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
1072024-03-28T08:15:05.221ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d2c06b0f-4e38-42b9-8862-41b76f684aa7", "content-length": "452", "date": "Thu, 28 Mar 2024 08:15:05 GMT"} })
1082024-03-28T08:15:05.221ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1092024-03-28T08:15:05.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1102024-03-28T08:15:05.224ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1112024-03-28T08:15:05.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1122024-03-28T08:15:05.226ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1132024-03-28T08:15:05.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1142024-03-28T08:15:05.227ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1152024-03-28T08:15:05.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1162024-03-28T08:15:05.235ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 364.138887ms iteration = 5
1172024-03-28T08:15:05.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 164 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = c637903d-4a78-4f67-a0f8-9c9f24188d06 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
1182024-03-28T08:15:14.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 12 reason = Timeout
1192024-03-28T08:15:14.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1202024-03-28T08:15:14.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1212024-03-28T08:15:14.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1222024-03-28T08:15:14.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1232024-03-28T08:15:14.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1242024-03-28T08:15:14.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1252024-03-28T08:15:14.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1262024-03-28T08:15:14.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1272024-03-28T08:15:14.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1282024-03-28T08:15:14.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
1292024-03-28T08:15:14.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1302024-03-28T08:15:14.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1312024-03-28T08:15:14.887ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
1322024-03-28T08:15:14.887ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.043889ms iteration = 12
1332024-03-28T08:15:15.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 157 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = bf3ffe67-0bce-47ea-b03a-66bf05b589c9 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
1342024-03-28T08:15:24.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
1352024-03-28T08:15:24.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1362024-03-28T08:15:24.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1372024-03-28T08:15:24.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1382024-03-28T08:15:24.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1392024-03-28T08:15:24.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1402024-03-28T08:15:24.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1412024-03-28T08:15:24.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1422024-03-28T08:15:24.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1432024-03-28T08:15:24.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1442024-03-28T08:15:24.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
1452024-03-28T08:15:24.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1462024-03-28T08:15:24.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1472024-03-28T08:15:24.887ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
1482024-03-28T08:15:24.887ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.523917ms iteration = 13
1492024-03-28T08:15:25.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 148 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = b6b1359e-3a6d-465e-81f8-d9842c4c91c0 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
1502024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 6 reason = Timeout
1512024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1522024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
1532024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
1542024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
1552024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1562024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
1572024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
1582024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
1592024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
1602024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
1612024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
1622024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1632024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1642024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
1652024-03-28T08:15:34.870ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
1662024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
1672024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
1682024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1692024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1702024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1712024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
1722024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
1732024-03-28T08:15:34.870ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
1742024-03-28T08:15:34.870ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
1752024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.39\u{b5}s iteration = 6
1762024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1772024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
1782024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
1792024-03-28T08:15:34.870ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 807102f2-0ae5-4b05-920b-677957cec7b1
1802024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.04\u{b5}s iteration = 5
1812024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
1822024-03-28T08:15:34.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1832024-03-28T08:15:34.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1842024-03-28T08:15:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1852024-03-28T08:15:34.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1862024-03-28T08:15:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1872024-03-28T08:15:34.871ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("58a99ec9-4663-432a-a76d-aa94f13dfa3e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 2664, tv_nsec: 427300642 } })
1882024-03-28T08:15:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1892024-03-28T08:15:34.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1902024-03-28T08:15:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1912024-03-28T08:15:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1922024-03-28T08:15:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1932024-03-28T08:15:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1942024-03-28T08:15:34.871ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
1952024-03-28T08:15:34.871ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.73706ms iteration = 4
1962024-03-28T08:15:34.871ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5df01d54-1bbe-4940-a2d2-252f1cdd47b1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8802bd0e-3008-47c5-a929-4b5b39083872.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 2664, tv_nsec: 428155597 } })
1972024-03-28T08:15:34.872ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1982024-03-28T08:15:34.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1992024-03-28T08:15:34.872ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
2002024-03-28T08:15:34.872ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.717418ms iteration = 6
2012024-03-28T08:15:34.873ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2022024-03-28T08:15:34.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2032024-03-28T08:15:34.873ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
2042024-03-28T08:15:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.545701ms iteration = 4
2052024-03-28T08:15:34.874ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2062024-03-28T08:15:34.875ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2664, tv_nsec: 431290661 } })
2072024-03-28T08:15:34.875ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2664, tv_nsec: 431298621 } })
2082024-03-28T08:15:34.875ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2092024-03-28T08:15:34.875ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2102024-03-28T08:15:34.875ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
2112024-03-28T08:15:34.875ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
2122024-03-28T08:15:34.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2132024-03-28T08:15:34.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2142024-03-28T08:15:34.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2152024-03-28T08:15:34.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2162024-03-28T08:15:34.877ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2172024-03-28T08:15:34.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2182024-03-28T08:15:34.877ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2192024-03-28T08:15:34.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2202024-03-28T08:15:34.877ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2212024-03-28T08:15:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2222024-03-28T08:15:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2232024-03-28T08:15:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2242024-03-28T08:15:34.878ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2252024-03-28T08:15:34.879ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
2262024-03-28T08:15:34.879ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2272024-03-28T08:15:34.879ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2282024-03-28T08:15:34.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2292024-03-28T08:15:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2302024-03-28T08:15:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2312024-03-28T08:15:34.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2322024-03-28T08:15:34.879ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2332024-03-28T08:15:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2342024-03-28T08:15:34.880ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2664, tv_nsec: 436686575 } })
2352024-03-28T08:15:34.880ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
2362024-03-28T08:15:34.880ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2372024-03-28T08:15:34.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2382024-03-28T08:15:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2392024-03-28T08:15:34.881ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
2402024-03-28T08:15:34.881ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
2412024-03-28T08:15:34.881ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
2422024-03-28T08:15:34.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2432024-03-28T08:15:34.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2442024-03-28T08:15:34.883ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2452024-03-28T08:15:34.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2462024-03-28T08:15:34.883ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2472024-03-28T08:15:34.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2482024-03-28T08:15:34.885ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2492024-03-28T08:15:34.885ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2502024-03-28T08:15:34.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2512024-03-28T08:15:34.885ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2522024-03-28T08:15:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2532024-03-28T08:15:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2542024-03-28T08:15:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2552024-03-28T08:15:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
2562024-03-28T08:15:34.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2572024-03-28T08:15:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2582024-03-28T08:15:34.886ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2592024-03-28T08:15:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2602024-03-28T08:15:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2612024-03-28T08:15:34.887ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2622024-03-28T08:15:34.888ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
2632024-03-28T08:15:34.888ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
2642024-03-28T08:15:34.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2652024-03-28T08:15:34.888ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2662024-03-28T08:15:34.888ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2672024-03-28T08:15:34.888ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
2682024-03-28T08:15:34.888ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 0 version = 2
2692024-03-28T08:15:34.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2702024-03-28T08:15:34.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2712024-03-28T08:15:34.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2722024-03-28T08:15:34.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2732024-03-28T08:15:34.890ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2742024-03-28T08:15:34.890ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2752024-03-28T08:15:34.891ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
2762024-03-28T08:15:34.891ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.270062ms iteration = 14
2772024-03-28T08:15:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2782024-03-28T08:15:34.893ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2792024-03-28T08:15:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2802024-03-28T08:15:34.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2812024-03-28T08:15:34.894ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2822024-03-28T08:15:34.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2832024-03-28T08:15:34.895ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 1 version = 2
2842024-03-28T08:15:34.895ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
2852024-03-28T08:15:34.895ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-03-28 08:13:43.437455 UTC dns_group = external generation = 2
2862024-03-28T08:15:34.895ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 25.056065ms iteration = 4
2872024-03-28T08:15:34.896ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
2882024-03-28T08:15:34.896ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
2892024-03-28T08:15:34.896ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-03-28 08:13:42.962945 UTC dns_group = internal generation = 1
2902024-03-28T08:15:34.896ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 25.88659ms iteration = 4
2912024-03-28T08:15:34.900ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2922024-03-28T08:15:34.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2932024-03-28T08:15:34.901ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2942024-03-28T08:15:34.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2952024-03-28T08:15:34.902ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-28T08:13:34.933339Z, time_modified: 2024-03-28T08:13:34.933339Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
2962024-03-28T08:15:34.902ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 32.222229ms iteration = 4
2972024-03-28T08:15:34.902ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2982024-03-28T08:15:34.905ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
2992024-03-28T08:15:34.905ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3002024-03-28T08:15:34.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3012024-03-28T08:15:34.907ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 36.180008ms iteration = 6
3022024-03-28T08:15:35.017ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
3032024-03-28T08:15:35.028ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
3042024-03-28T08:15:35.028ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3052024-03-28T08:15:35.028ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c96a5997-9a39-47ea-b5f1-f414bea8d0ca", "content-length": "26", "date": "Thu, 28 Mar 2024 08:15:35 GMT"} })
3062024-03-28T08:15:35.028ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3072024-03-28T08:15:35.030ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
3082024-03-28T08:15:35.036ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
3092024-03-28T08:15:35.036ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3102024-03-28T08:15:35.036ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2616e4fc-1128-48a0-b71e-f07f1da1303c", "content-length": "26", "date": "Thu, 28 Mar 2024 08:15:35 GMT"} })
3112024-03-28T08:15:35.036ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3122024-03-28T08:15:35.039ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3132024-03-28T08:15:35.151ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
3142024-03-28T08:15:35.166ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
3152024-03-28T08:15:35.166ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3162024-03-28T08:15:35.166ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "69ed7c33-448c-46bc-bcef-810b92281d0b", "content-length": "26", "date": "Thu, 28 Mar 2024 08:15:35 GMT"} })
3172024-03-28T08:15:35.166ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3182024-03-28T08:15:35.168ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3192024-03-28T08:15:35.299ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3202024-03-28T08:15:35.300ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 430.440819ms iteration = 6
3212024-03-28T08:15:35.303ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
3222024-03-28T08:15:35.304ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b2dae13f-411f-4916-8e89-e791d5315a30", "date": "Thu, 28 Mar 2024 08:15:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3232024-03-28T08:15:35.304ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3242024-03-28T08:15:35.306ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f385f24e-7abd-4beb-bf0a-fcebd021da3a", "date": "Thu, 28 Mar 2024 08:15:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3252024-03-28T08:15:35.306ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3262024-03-28T08:15:35.306ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3272024-03-28T08:15:35.306ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2cadbc05-3afb-46e4-a267-cda47eaebf63", "content-length": "1", "date": "Thu, 28 Mar 2024 08:15:34 GMT"} })
3282024-03-28T08:15:35.308ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3292024-03-28T08:15:35.308ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 438.859904ms iteration = 6
3302024-03-28T08:15:35.310ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3312024-03-28T08:15:35.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3322024-03-28T08:15:35.312ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 442.13402ms iteration = 6
3332024-03-28T08:15:35.367ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
3342024-03-28T08:15:35.367ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e049b1d4-807a-460d-866b-a3c30840d61e", "date": "Thu, 28 Mar 2024 08:15:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
3352024-03-28T08:15:35.367ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
3362024-03-28T08:15:35.369ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b7d9972c-3ab3-4397-9229-decc01f21476", "date": "Thu, 28 Mar 2024 08:15:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
3372024-03-28T08:15:35.369ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
3382024-03-28T08:15:35.369ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3392024-03-28T08:15:35.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3402024-03-28T08:15:35.370ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "25338e51-45c9-4c35-81eb-a4110f3675a2", "date": "Thu, 28 Mar 2024 08:15:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
3412024-03-28T08:15:35.370ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
3422024-03-28T08:15:35.371ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
3432024-03-28T08:15:35.371ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 501.299548ms iteration = 6
3442024-03-28T08:15:35.371ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
3452024-03-28T08:15:35.371ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3462024-03-28T08:15:35.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3472024-03-28T08:15:35.378ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
3482024-03-28T08:15:35.378ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "805faa67-d3e4-4f05-ab37-1b7abd92188e", "content-length": "121", "date": "Thu, 28 Mar 2024 08:15:34 GMT"} })
3492024-03-28T08:15:35.379ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
3502024-03-28T08:15:35.379ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
3512024-03-28T08:15:35.379ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
3522024-03-28T08:15:35.379ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
3532024-03-28T08:15:35.379ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
3542024-03-28T08:15:35.379ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
3552024-03-28T08:15:35.379ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
3562024-03-28T08:15:35.379ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3572024-03-28T08:15:35.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3582024-03-28T08:15:35.381ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
3592024-03-28T08:15:35.381ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "576a5911-cf76-4a87-937b-d296c9ab83d6", "content-length": "2", "date": "Thu, 28 Mar 2024 08:15:34 GMT"} })
3602024-03-28T08:15:35.381ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
3612024-03-28T08:15:35.381ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7fe6530a-7a67-46de-9f2f-064203917ab6", "content-length": "52", "date": "Thu, 28 Mar 2024 08:15:34 GMT"} })
3622024-03-28T08:15:35.381ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
3632024-03-28T08:15:35.381ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
3642024-03-28T08:15:35.430ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "254e03ef-bf58-41ac-b9d2-e32ca7289122", "date": "Thu, 28 Mar 2024 08:15:35 GMT"} })
3652024-03-28T08:15:35.430ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
3662024-03-28T08:15:35.430ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
3672024-03-28T08:15:35.430ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e8a24b5f-a8f4-4dcf-a337-d524efb11503", "content-length": "452", "date": "Thu, 28 Mar 2024 08:15:35 GMT"} })
3682024-03-28T08:15:35.430ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3692024-03-28T08:15:35.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3702024-03-28T08:15:35.434ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3712024-03-28T08:15:35.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3722024-03-28T08:15:35.435ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3732024-03-28T08:15:35.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3742024-03-28T08:15:35.436ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3752024-03-28T08:15:35.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3762024-03-28T08:15:35.443ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 572.31684ms iteration = 6
3772024-03-28T08:15:35.458ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 909 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = c7de605f-933d-487a-bb81-331898d22aad response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
3782024-03-28T08:15:44.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
3792024-03-28T08:15:44.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3802024-03-28T08:15:44.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3812024-03-28T08:15:44.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3822024-03-28T08:15:44.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3832024-03-28T08:15:44.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3842024-03-28T08:15:44.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3852024-03-28T08:15:44.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3862024-03-28T08:15:44.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3872024-03-28T08:15:44.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3882024-03-28T08:15:44.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
3892024-03-28T08:15:44.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3902024-03-28T08:15:44.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3912024-03-28T08:15:44.884ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
3922024-03-28T08:15:44.884ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.20379ms iteration = 15
3932024-03-28T08:15:45.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 204 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 1813f795-76d0-4c0a-b9c6-9c22fa9cbf91 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
3942024-03-28T08:15:54.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
3952024-03-28T08:15:54.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3962024-03-28T08:15:54.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3972024-03-28T08:15:54.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3982024-03-28T08:15:54.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3992024-03-28T08:15:54.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4002024-03-28T08:15:54.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4012024-03-28T08:15:54.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4022024-03-28T08:15:54.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4032024-03-28T08:15:54.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4042024-03-28T08:15:54.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
4052024-03-28T08:15:54.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4062024-03-28T08:15:54.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4072024-03-28T08:15:54.884ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
4082024-03-28T08:15:54.884ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.817889ms iteration = 16
4092024-03-28T08:15:55.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 872 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 4c9dd2a5-fd83-4cdc-9479-4ed0270992c4 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
4102024-03-28T08:16:04.870ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 7 reason = Timeout
4112024-03-28T08:16:04.870ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4122024-03-28T08:16:04.870ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
4132024-03-28T08:16:04.870ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
4142024-03-28T08:16:04.870ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
4152024-03-28T08:16:04.870ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4162024-03-28T08:16:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4172024-03-28T08:16:04.871ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
4182024-03-28T08:16:04.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4192024-03-28T08:16:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4202024-03-28T08:16:04.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
4212024-03-28T08:16:04.871ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
4222024-03-28T08:16:04.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4232024-03-28T08:16:04.871ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
4242024-03-28T08:16:04.871ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
4252024-03-28T08:16:04.871ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
4262024-03-28T08:16:04.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4272024-03-28T08:16:04.871ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 11.751\u{b5}s iteration = 7
4282024-03-28T08:16:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4292024-03-28T08:16:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4302024-03-28T08:16:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4312024-03-28T08:16:04.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4322024-03-28T08:16:04.873ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4332024-03-28T08:16:04.873ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4342024-03-28T08:16:04.873ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2694, tv_nsec: 429358775 } })
4352024-03-28T08:16:04.873ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
4362024-03-28T08:16:04.873ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
4372024-03-28T08:16:04.873ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.163866ms iteration = 7
4382024-03-28T08:16:04.874ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2694, tv_nsec: 430425543 } })
4392024-03-28T08:16:04.874ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
4402024-03-28T08:16:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
4412024-03-28T08:16:04.876ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4422024-03-28T08:16:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4432024-03-28T08:16:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4442024-03-28T08:16:04.876ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4452024-03-28T08:16:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4462024-03-28T08:16:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4472024-03-28T08:16:04.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4482024-03-28T08:16:04.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4492024-03-28T08:16:04.879ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2694, tv_nsec: 435434900 } })
4502024-03-28T08:16:04.879ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
4512024-03-28T08:16:04.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4522024-03-28T08:16:04.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4532024-03-28T08:16:04.880ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4542024-03-28T08:16:04.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4552024-03-28T08:16:04.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4562024-03-28T08:16:04.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
4572024-03-28T08:16:04.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4582024-03-28T08:16:04.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4592024-03-28T08:16:04.887ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
4602024-03-28T08:16:04.887ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.257012ms iteration = 17
4612024-03-28T08:16:04.895ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4622024-03-28T08:16:04.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4632024-03-28T08:16:04.897ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4642024-03-28T08:16:04.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4652024-03-28T08:16:04.898ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4662024-03-28T08:16:04.901ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
4672024-03-28T08:16:04.901ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4682024-03-28T08:16:04.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4692024-03-28T08:16:04.902ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 31.520427ms iteration = 7
4702024-03-28T08:16:04.964ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
4712024-03-28T08:16:04.964ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4722024-03-28T08:16:04.965ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bf618360-75a6-48dd-bbb4-c15a8b8c74b2", "content-length": "26", "date": "Thu, 28 Mar 2024 08:16:04 GMT"} })
4732024-03-28T08:16:04.965ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4742024-03-28T08:16:04.967ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
4752024-03-28T08:16:05.061ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
4762024-03-28T08:16:05.061ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4772024-03-28T08:16:05.062ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "502c44e5-0263-435c-8262-27dd078815e7", "content-length": "26", "date": "Thu, 28 Mar 2024 08:16:04 GMT"} })
4782024-03-28T08:16:05.062ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4792024-03-28T08:16:05.064ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4802024-03-28T08:16:05.078ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
4812024-03-28T08:16:05.078ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4822024-03-28T08:16:05.079ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d0460589-d37f-487b-81bb-dd14a80feac6", "content-length": "26", "date": "Thu, 28 Mar 2024 08:16:04 GMT"} })
4832024-03-28T08:16:05.079ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4842024-03-28T08:16:05.080ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4852024-03-28T08:16:05.142ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4862024-03-28T08:16:05.143ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cba93c0d-4bc3-4665-b9d4-654395db4fff", "content-length": "1", "date": "Thu, 28 Mar 2024 08:16:04 GMT"} })
4872024-03-28T08:16:05.145ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4882024-03-28T08:16:05.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4892024-03-28T08:16:05.146ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 276.235923ms iteration = 7
4902024-03-28T08:16:05.149ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 278.37958ms iteration = 7
4912024-03-28T08:16:05.175ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4922024-03-28T08:16:05.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4932024-03-28T08:16:05.178ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
4942024-03-28T08:16:05.178ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4952024-03-28T08:16:05.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4962024-03-28T08:16:05.185ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4972024-03-28T08:16:05.185ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e79af9af-587e-4439-90a4-a33ff3d50eb3", "content-length": "121", "date": "Thu, 28 Mar 2024 08:16:04 GMT"} })
4982024-03-28T08:16:05.185ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
4992024-03-28T08:16:05.185ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
5002024-03-28T08:16:05.185ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
5012024-03-28T08:16:05.185ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
5022024-03-28T08:16:05.185ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
5032024-03-28T08:16:05.185ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
5042024-03-28T08:16:05.185ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
5052024-03-28T08:16:05.185ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5062024-03-28T08:16:05.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5072024-03-28T08:16:05.187ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
5082024-03-28T08:16:05.187ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7a59f2e3-2fb7-4f0a-b7dd-3959e681b175", "content-length": "2", "date": "Thu, 28 Mar 2024 08:16:04 GMT"} })
5092024-03-28T08:16:05.187ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
5102024-03-28T08:16:05.188ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a3d24a75-18a9-4f5d-b618-4c8887980dc6", "content-length": "52", "date": "Thu, 28 Mar 2024 08:16:04 GMT"} })
5112024-03-28T08:16:05.188ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
5122024-03-28T08:16:05.188ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
5132024-03-28T08:16:05.235ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "02e2aa55-02d0-4c9f-b9aa-da45f4f7a2ad", "date": "Thu, 28 Mar 2024 08:16:05 GMT"} })
5142024-03-28T08:16:05.235ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
5152024-03-28T08:16:05.235ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
5162024-03-28T08:16:05.236ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d87aba58-77b3-4306-a86a-9dbe04e0238d", "content-length": "452", "date": "Thu, 28 Mar 2024 08:16:05 GMT"} })
5172024-03-28T08:16:05.236ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5182024-03-28T08:16:05.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5192024-03-28T08:16:05.239ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5202024-03-28T08:16:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5212024-03-28T08:16:05.242ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5222024-03-28T08:16:05.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5232024-03-28T08:16:05.243ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5242024-03-28T08:16:05.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5252024-03-28T08:16:05.249ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 378.07398ms iteration = 7
5262024-03-28T08:16:05.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 127 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 65c94918-06ac-4aff-88f9-604d71219efb response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
5272024-03-28T08:16:14.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
5282024-03-28T08:16:14.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5292024-03-28T08:16:14.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5302024-03-28T08:16:14.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5312024-03-28T08:16:14.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5322024-03-28T08:16:14.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5332024-03-28T08:16:14.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5342024-03-28T08:16:14.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5352024-03-28T08:16:14.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5362024-03-28T08:16:14.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5372024-03-28T08:16:14.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
5382024-03-28T08:16:14.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5392024-03-28T08:16:14.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5402024-03-28T08:16:14.887ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
5412024-03-28T08:16:14.887ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.972088ms iteration = 18
5422024-03-28T08:16:15.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 211 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = dee25257-2988-4c30-b416-98049941c32e response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
5432024-03-28T08:16:24.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
5442024-03-28T08:16:24.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5452024-03-28T08:16:24.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5462024-03-28T08:16:24.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5472024-03-28T08:16:24.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5482024-03-28T08:16:24.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5492024-03-28T08:16:24.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5502024-03-28T08:16:24.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5512024-03-28T08:16:24.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5522024-03-28T08:16:24.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5532024-03-28T08:16:24.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
5542024-03-28T08:16:24.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5552024-03-28T08:16:24.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5562024-03-28T08:16:24.885ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
5572024-03-28T08:16:24.885ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.938782ms iteration = 19
5582024-03-28T08:16:25.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 241 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 9fa968ea-3ebe-4a78-a3a8-f4afc3c08a0f response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
5592024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
5602024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5612024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
5622024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
5632024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5642024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
5652024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
5662024-03-28T08:16:34.869ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
5672024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 8 reason = Timeout
5682024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
5692024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5702024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
5712024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
5722024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
5732024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
5742024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5752024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
5762024-03-28T08:16:34.869ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5772024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5782024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5792024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5802024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5812024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5822024-03-28T08:16:34.870ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("58a99ec9-4663-432a-a76d-aa94f13dfa3e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 2724, tv_nsec: 426808877 } })
5832024-03-28T08:16:34.870ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5df01d54-1bbe-4940-a2d2-252f1cdd47b1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8802bd0e-3008-47c5-a929-4b5b39083872.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 2724, tv_nsec: 426854218 } })
5842024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
5852024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
5862024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
5872024-03-28T08:16:34.870ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
5882024-03-28T08:16:34.870ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
5892024-03-28T08:16:34.870ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 807102f2-0ae5-4b05-920b-677957cec7b1
5902024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.72\u{b5}s iteration = 8
5912024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
5922024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.38\u{b5}s iteration = 6
5932024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5942024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
5952024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5962024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5972024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5982024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5992024-03-28T08:16:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6002024-03-28T08:16:34.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6012024-03-28T08:16:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6022024-03-28T08:16:34.871ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6032024-03-28T08:16:34.871ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
6042024-03-28T08:16:34.871ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.919943ms iteration = 5
6052024-03-28T08:16:34.872ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6062024-03-28T08:16:34.872ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6072024-03-28T08:16:34.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6082024-03-28T08:16:34.872ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6092024-03-28T08:16:34.872ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6102024-03-28T08:16:34.873ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
6112024-03-28T08:16:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.319837ms iteration = 8
6122024-03-28T08:16:34.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6132024-03-28T08:16:34.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6142024-03-28T08:16:34.873ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
6152024-03-28T08:16:34.873ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2724, tv_nsec: 430315128 } })
6162024-03-28T08:16:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.123292ms iteration = 5
6172024-03-28T08:16:34.875ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
6182024-03-28T08:16:34.875ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6192024-03-28T08:16:34.875ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6202024-03-28T08:16:34.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6212024-03-28T08:16:34.876ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2724, tv_nsec: 432454895 } })
6222024-03-28T08:16:34.876ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
6232024-03-28T08:16:34.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6242024-03-28T08:16:34.876ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6252024-03-28T08:16:34.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6262024-03-28T08:16:34.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6272024-03-28T08:16:34.877ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6282024-03-28T08:16:34.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6292024-03-28T08:16:34.877ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6302024-03-28T08:16:34.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6312024-03-28T08:16:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6322024-03-28T08:16:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6332024-03-28T08:16:34.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6342024-03-28T08:16:34.878ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2724, tv_nsec: 435113851 } })
6352024-03-28T08:16:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6362024-03-28T08:16:34.878ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6372024-03-28T08:16:34.878ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6382024-03-28T08:16:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6392024-03-28T08:16:34.879ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
6402024-03-28T08:16:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6412024-03-28T08:16:34.879ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
6422024-03-28T08:16:34.879ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
6432024-03-28T08:16:34.879ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
6442024-03-28T08:16:34.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6452024-03-28T08:16:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6462024-03-28T08:16:34.880ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
6472024-03-28T08:16:34.880ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
6482024-03-28T08:16:34.880ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
6492024-03-28T08:16:34.880ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6502024-03-28T08:16:34.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6512024-03-28T08:16:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6522024-03-28T08:16:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6532024-03-28T08:16:34.880ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6542024-03-28T08:16:34.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6552024-03-28T08:16:34.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6562024-03-28T08:16:34.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
6572024-03-28T08:16:34.882ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6582024-03-28T08:16:34.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6592024-03-28T08:16:34.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6602024-03-28T08:16:34.884ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6612024-03-28T08:16:34.884ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6622024-03-28T08:16:34.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6632024-03-28T08:16:34.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6642024-03-28T08:16:34.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6652024-03-28T08:16:34.885ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6662024-03-28T08:16:34.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
6672024-03-28T08:16:34.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6682024-03-28T08:16:34.885ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6692024-03-28T08:16:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6702024-03-28T08:16:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6712024-03-28T08:16:34.886ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
6722024-03-28T08:16:34.886ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
6732024-03-28T08:16:34.886ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6742024-03-28T08:16:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6752024-03-28T08:16:34.887ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
6762024-03-28T08:16:34.887ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 0 version = 2
6772024-03-28T08:16:34.887ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6782024-03-28T08:16:34.887ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6792024-03-28T08:16:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6802024-03-28T08:16:34.887ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6812024-03-28T08:16:34.887ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6822024-03-28T08:16:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6832024-03-28T08:16:34.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6842024-03-28T08:16:34.889ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
6852024-03-28T08:16:34.889ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.076661ms iteration = 20
6862024-03-28T08:16:34.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6872024-03-28T08:16:34.890ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6882024-03-28T08:16:34.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6892024-03-28T08:16:34.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6902024-03-28T08:16:34.891ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6912024-03-28T08:16:34.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6922024-03-28T08:16:34.892ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
6932024-03-28T08:16:34.892ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
6942024-03-28T08:16:34.892ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-03-28 08:13:42.962945 UTC dns_group = internal generation = 1
6952024-03-28T08:16:34.892ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 23.145961ms iteration = 5
6962024-03-28T08:16:34.893ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 1 version = 2
6972024-03-28T08:16:34.893ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
6982024-03-28T08:16:34.893ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-03-28 08:13:43.437455 UTC dns_group = external generation = 2
6992024-03-28T08:16:34.893ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 23.674161ms iteration = 5
7002024-03-28T08:16:34.893ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7012024-03-28T08:16:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7022024-03-28T08:16:34.895ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7032024-03-28T08:16:34.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7042024-03-28T08:16:34.896ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-28T08:13:34.933339Z, time_modified: 2024-03-28T08:13:34.933339Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }) }
7052024-03-28T08:16:34.896ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 26.740204ms iteration = 5
7062024-03-28T08:16:34.896ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7072024-03-28T08:16:34.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
7082024-03-28T08:16:34.898ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7092024-03-28T08:16:34.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7102024-03-28T08:16:34.900ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 29.455601ms iteration = 8
7112024-03-28T08:16:35.013ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
7122024-03-28T08:16:35.143ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
7132024-03-28T08:16:35.143ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7142024-03-28T08:16:35.144ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "07176824-2f69-4a17-914d-a9a42ae619cc", "content-length": "26", "date": "Thu, 28 Mar 2024 08:16:35 GMT"} })
7152024-03-28T08:16:35.144ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7162024-03-28T08:16:35.146ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
7172024-03-28T08:16:35.146ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7182024-03-28T08:16:35.146ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
7192024-03-28T08:16:35.146ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "62f550d2-4283-4830-9024-94e13a200746", "content-length": "26", "date": "Thu, 28 Mar 2024 08:16:35 GMT"} })
7202024-03-28T08:16:35.146ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7212024-03-28T08:16:35.146ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
7222024-03-28T08:16:35.146ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7232024-03-28T08:16:35.147ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "278a7255-42c3-4f4e-aa37-29b0c6bf84da", "content-length": "26", "date": "Thu, 28 Mar 2024 08:16:35 GMT"} })
7242024-03-28T08:16:35.147ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7252024-03-28T08:16:35.147ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7262024-03-28T08:16:35.150ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
7272024-03-28T08:16:35.150ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7282024-03-28T08:16:35.165ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
7292024-03-28T08:16:35.165ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9105b589-a240-49ac-b3b0-d0f1822225b6", "date": "Thu, 28 Mar 2024 08:16:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7302024-03-28T08:16:35.165ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7312024-03-28T08:16:35.167ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "140239b9-ecaa-415f-8e06-1d7cf5032c4f", "date": "Thu, 28 Mar 2024 08:16:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7322024-03-28T08:16:35.167ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7332024-03-28T08:16:35.168ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7342024-03-28T08:16:35.168ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 298.763824ms iteration = 7
7352024-03-28T08:16:35.247ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 377.555461ms iteration = 8
7362024-03-28T08:16:35.252ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
7372024-03-28T08:16:35.252ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7382024-03-28T08:16:35.253ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b0480353-140f-4052-9737-3c6b70baf234", "content-length": "1", "date": "Thu, 28 Mar 2024 08:16:34 GMT"} })
7392024-03-28T08:16:35.253ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7402024-03-28T08:16:35.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7412024-03-28T08:16:35.255ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 385.752513ms iteration = 8
7422024-03-28T08:16:35.315ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
7432024-03-28T08:16:35.315ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "08896e5c-880a-45ba-9a50-5fa515edd6d2", "date": "Thu, 28 Mar 2024 08:16:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
7442024-03-28T08:16:35.315ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
7452024-03-28T08:16:35.317ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ce7d296a-5a41-4eca-ba17-3d019f929e42", "date": "Thu, 28 Mar 2024 08:16:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
7462024-03-28T08:16:35.317ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
7472024-03-28T08:16:35.318ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6dabc39b-2308-42c5-96a1-56701af61e8c", "date": "Thu, 28 Mar 2024 08:16:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
7482024-03-28T08:16:35.318ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
7492024-03-28T08:16:35.320ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
7502024-03-28T08:16:35.320ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 450.545567ms iteration = 7
7512024-03-28T08:16:35.345ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7522024-03-28T08:16:35.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7532024-03-28T08:16:35.347ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
7542024-03-28T08:16:35.347ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7552024-03-28T08:16:35.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7562024-03-28T08:16:35.353ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
7572024-03-28T08:16:35.353ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5b6a3dd0-6e17-403f-9921-40c4923bdcf5", "content-length": "121", "date": "Thu, 28 Mar 2024 08:16:34 GMT"} })
7582024-03-28T08:16:35.353ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
7592024-03-28T08:16:35.353ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
7602024-03-28T08:16:35.353ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
7612024-03-28T08:16:35.353ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
7622024-03-28T08:16:35.353ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
7632024-03-28T08:16:35.353ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
7642024-03-28T08:16:35.353ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
7652024-03-28T08:16:35.353ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7662024-03-28T08:16:35.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7672024-03-28T08:16:35.355ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
7682024-03-28T08:16:35.356ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f7479b34-68b7-4afa-a7da-062d0bb5dd65", "content-length": "2", "date": "Thu, 28 Mar 2024 08:16:34 GMT"} })
7692024-03-28T08:16:35.356ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
7702024-03-28T08:16:35.356ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cd71533c-379d-4809-a4e6-10be6c09084a", "content-length": "52", "date": "Thu, 28 Mar 2024 08:16:34 GMT"} })
7712024-03-28T08:16:35.356ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
7722024-03-28T08:16:35.356ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
7732024-03-28T08:16:35.405ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8c2b2755-5ca1-48c1-82cb-ecadec1d2670", "date": "Thu, 28 Mar 2024 08:16:35 GMT"} })
7742024-03-28T08:16:35.405ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
7752024-03-28T08:16:35.405ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
7762024-03-28T08:16:35.405ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "124dac01-42b5-49c0-ad5b-b05fc9bf66ad", "content-length": "452", "date": "Thu, 28 Mar 2024 08:16:35 GMT"} })
7772024-03-28T08:16:35.405ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7782024-03-28T08:16:35.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7792024-03-28T08:16:35.408ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7802024-03-28T08:16:35.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7812024-03-28T08:16:35.411ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7822024-03-28T08:16:35.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7832024-03-28T08:16:35.412ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7842024-03-28T08:16:35.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7852024-03-28T08:16:35.419ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 548.945794ms iteration = 8
7862024-03-28T08:16:35.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 138 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 77de3c1e-c210-4ff1-89a5-da654017e2c8 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
7872024-03-28T08:16:44.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
7882024-03-28T08:16:44.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7892024-03-28T08:16:44.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7902024-03-28T08:16:44.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7912024-03-28T08:16:44.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7922024-03-28T08:16:44.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7932024-03-28T08:16:44.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7942024-03-28T08:16:44.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7952024-03-28T08:16:44.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7962024-03-28T08:16:44.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7972024-03-28T08:16:44.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
7982024-03-28T08:16:44.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7992024-03-28T08:16:44.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8002024-03-28T08:16:44.883ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
8012024-03-28T08:16:44.883ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.25829ms iteration = 21
8022024-03-28T08:16:45.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 144 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = e8fc363b-4ee7-46f9-add7-2807bc93f656 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
8032024-03-28T08:16:54.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
8042024-03-28T08:16:54.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8052024-03-28T08:16:54.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8062024-03-28T08:16:54.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8072024-03-28T08:16:54.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8082024-03-28T08:16:54.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8092024-03-28T08:16:54.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8102024-03-28T08:16:54.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8112024-03-28T08:16:54.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8122024-03-28T08:16:54.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8132024-03-28T08:16:54.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
8142024-03-28T08:16:54.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8152024-03-28T08:16:54.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8162024-03-28T08:16:54.887ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
8172024-03-28T08:16:54.887ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.051156ms iteration = 22
8182024-03-28T08:16:55.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 208 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = d7f7e0f5-3a1d-43d6-b914-cf8e323f87b0 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
8192024-03-28T08:17:04.869ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
8202024-03-28T08:17:04.869ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8212024-03-28T08:17:04.869ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
8222024-03-28T08:17:04.869ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
8232024-03-28T08:17:04.869ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
8242024-03-28T08:17:04.869ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
8252024-03-28T08:17:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8262024-03-28T08:17:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
8272024-03-28T08:17:04.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
8282024-03-28T08:17:04.870ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
8292024-03-28T08:17:04.870ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
8302024-03-28T08:17:04.870ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8312024-03-28T08:17:04.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8322024-03-28T08:17:04.870ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
8332024-03-28T08:17:04.870ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
8342024-03-28T08:17:04.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8352024-03-28T08:17:04.870ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
8362024-03-28T08:17:04.870ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.669\u{b5}s iteration = 9
8372024-03-28T08:17:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8382024-03-28T08:17:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8392024-03-28T08:17:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8402024-03-28T08:17:04.871ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2754, tv_nsec: 428304671 } })
8412024-03-28T08:17:04.872ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
8422024-03-28T08:17:04.872ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8432024-03-28T08:17:04.872ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8442024-03-28T08:17:04.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8452024-03-28T08:17:04.872ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
8462024-03-28T08:17:04.872ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.104874ms iteration = 9
8472024-03-28T08:17:04.873ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2754, tv_nsec: 429731896 } })
8482024-03-28T08:17:04.873ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
8492024-03-28T08:17:04.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
8502024-03-28T08:17:04.875ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8512024-03-28T08:17:04.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8522024-03-28T08:17:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8532024-03-28T08:17:04.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8542024-03-28T08:17:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8552024-03-28T08:17:04.876ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8562024-03-28T08:17:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8572024-03-28T08:17:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8582024-03-28T08:17:04.878ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2754, tv_nsec: 434451688 } })
8592024-03-28T08:17:04.878ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
8602024-03-28T08:17:04.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8612024-03-28T08:17:04.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8622024-03-28T08:17:04.878ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8632024-03-28T08:17:04.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8642024-03-28T08:17:04.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8652024-03-28T08:17:04.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
8662024-03-28T08:17:04.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8672024-03-28T08:17:04.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8682024-03-28T08:17:04.885ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
8692024-03-28T08:17:04.885ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.115972ms iteration = 23
8702024-03-28T08:17:04.892ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8712024-03-28T08:17:04.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8722024-03-28T08:17:04.894ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8732024-03-28T08:17:04.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8742024-03-28T08:17:04.895ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8752024-03-28T08:17:04.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
8762024-03-28T08:17:04.897ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8772024-03-28T08:17:04.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8782024-03-28T08:17:04.898ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 27.891814ms iteration = 9
8792024-03-28T08:17:04.958ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
8802024-03-28T08:17:04.958ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8812024-03-28T08:17:04.958ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e4d82ac5-85cc-47a6-b45b-41fba758e56d", "content-length": "26", "date": "Thu, 28 Mar 2024 08:17:04 GMT"} })
8822024-03-28T08:17:04.958ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8832024-03-28T08:17:04.960ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8842024-03-28T08:17:05.058ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
8852024-03-28T08:17:05.059ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c575d108-ee0d-457b-8b77-c950af45298c", "content-length": "1", "date": "Thu, 28 Mar 2024 08:17:04 GMT"} })
8862024-03-28T08:17:05.060ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8872024-03-28T08:17:05.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8882024-03-28T08:17:05.062ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
8892024-03-28T08:17:05.062ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8902024-03-28T08:17:05.062ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 192.915287ms iteration = 9
8912024-03-28T08:17:05.063ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7f7defe5-4743-47d6-a7f1-56eb7977ab60", "content-length": "26", "date": "Thu, 28 Mar 2024 08:17:04 GMT"} })
8922024-03-28T08:17:05.063ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8932024-03-28T08:17:05.063ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
8942024-03-28T08:17:05.063ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8952024-03-28T08:17:05.063ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fc3099fb-f1e8-4515-ae1e-649f5e910244", "content-length": "26", "date": "Thu, 28 Mar 2024 08:17:04 GMT"} })
8962024-03-28T08:17:05.063ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8972024-03-28T08:17:05.064ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8982024-03-28T08:17:05.065ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
8992024-03-28T08:17:05.103ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 234.159183ms iteration = 9
9002024-03-28T08:17:05.162ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9012024-03-28T08:17:05.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9022024-03-28T08:17:05.165ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
9032024-03-28T08:17:05.165ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9042024-03-28T08:17:05.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9052024-03-28T08:17:05.171ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
9062024-03-28T08:17:05.171ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "17dec7ca-91aa-469c-8319-923f84255ff5", "content-length": "121", "date": "Thu, 28 Mar 2024 08:17:04 GMT"} })
9072024-03-28T08:17:05.171ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
9082024-03-28T08:17:05.171ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
9092024-03-28T08:17:05.171ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
9102024-03-28T08:17:05.171ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
9112024-03-28T08:17:05.171ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
9122024-03-28T08:17:05.171ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
9132024-03-28T08:17:05.171ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
9142024-03-28T08:17:05.171ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9152024-03-28T08:17:05.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9162024-03-28T08:17:05.175ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
9172024-03-28T08:17:05.176ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "831e9ce1-6a57-446a-ae5e-d7b7b80f5162", "content-length": "2", "date": "Thu, 28 Mar 2024 08:17:04 GMT"} })
9182024-03-28T08:17:05.176ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
9192024-03-28T08:17:05.176ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c12e60c8-c0be-4284-a18e-396b3b0266f7", "content-length": "52", "date": "Thu, 28 Mar 2024 08:17:04 GMT"} })
9202024-03-28T08:17:05.176ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
9212024-03-28T08:17:05.176ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
9222024-03-28T08:17:05.222ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9049c146-d226-491d-8dcb-8115c65c7e54", "date": "Thu, 28 Mar 2024 08:17:05 GMT"} })
9232024-03-28T08:17:05.222ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
9242024-03-28T08:17:05.222ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
9252024-03-28T08:17:05.223ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fe4740fe-354b-41dc-9b2f-c57f9f508790", "content-length": "452", "date": "Thu, 28 Mar 2024 08:17:05 GMT"} })
9262024-03-28T08:17:05.223ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9272024-03-28T08:17:05.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9282024-03-28T08:17:05.226ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9292024-03-28T08:17:05.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9302024-03-28T08:17:05.229ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9312024-03-28T08:17:05.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9322024-03-28T08:17:05.230ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9332024-03-28T08:17:05.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9342024-03-28T08:17:05.236ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 365.677725ms iteration = 9
9352024-03-28T08:17:05.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 154 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 2a9991b6-8ae8-4a01-ac34-04583460f8b1 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
9362024-03-28T08:17:14.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
9372024-03-28T08:17:14.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9382024-03-28T08:17:14.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9392024-03-28T08:17:14.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9402024-03-28T08:17:14.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9412024-03-28T08:17:14.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9422024-03-28T08:17:14.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9432024-03-28T08:17:14.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9442024-03-28T08:17:14.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9452024-03-28T08:17:14.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9462024-03-28T08:17:14.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
9472024-03-28T08:17:14.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9482024-03-28T08:17:14.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9492024-03-28T08:17:14.884ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
9502024-03-28T08:17:14.884ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.854761ms iteration = 24
9512024-03-28T08:17:15.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 138 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 64c12c68-968f-41a0-a556-11d5c5d6bce8 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
9522024-03-28T08:17:24.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
9532024-03-28T08:17:24.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9542024-03-28T08:17:24.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9552024-03-28T08:17:24.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9562024-03-28T08:17:24.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9572024-03-28T08:17:24.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9582024-03-28T08:17:24.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9592024-03-28T08:17:24.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9602024-03-28T08:17:24.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9612024-03-28T08:17:24.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9622024-03-28T08:17:24.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
9632024-03-28T08:17:24.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9642024-03-28T08:17:24.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9652024-03-28T08:17:24.885ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
9662024-03-28T08:17:24.885ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.012473ms iteration = 25
9672024-03-28T08:17:25.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 130 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 25a57919-f56f-4183-b81b-4078263ed8f5 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
9682024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
9692024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
9702024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9712024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9722024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
9732024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
9742024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
9752024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
9762024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9772024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
9782024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
9792024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9802024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
9812024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
9822024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
9832024-03-28T08:17:34.870ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
9842024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
9852024-03-28T08:17:34.870ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9862024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
9872024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
9882024-03-28T08:17:34.871ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
9892024-03-28T08:17:34.871ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
9902024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.64\u{b5}s iteration = 10
9912024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
9922024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
9932024-03-28T08:17:34.871ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 807102f2-0ae5-4b05-920b-677957cec7b1
9942024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9952024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.391\u{b5}s iteration = 7
9962024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9972024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
9982024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9992024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10002024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
10012024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10022024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10032024-03-28T08:17:34.871ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5df01d54-1bbe-4940-a2d2-252f1cdd47b1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8802bd0e-3008-47c5-a929-4b5b39083872.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 2784, tv_nsec: 427886071 } })
10042024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10052024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10062024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10072024-03-28T08:17:34.871ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("58a99ec9-4663-432a-a76d-aa94f13dfa3e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 2784, tv_nsec: 428047664 } })
10082024-03-28T08:17:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10092024-03-28T08:17:34.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10102024-03-28T08:17:34.873ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10112024-03-28T08:17:34.873ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10122024-03-28T08:17:34.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10132024-03-28T08:17:34.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10142024-03-28T08:17:34.873ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
10152024-03-28T08:17:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.980131ms iteration = 10
10162024-03-28T08:17:34.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10172024-03-28T08:17:34.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10182024-03-28T08:17:34.874ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
10192024-03-28T08:17:34.874ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.583312ms iteration = 6
10202024-03-28T08:17:34.874ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10212024-03-28T08:17:34.875ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
10222024-03-28T08:17:34.875ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.364415ms iteration = 6
10232024-03-28T08:17:34.875ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2784, tv_nsec: 431449333 } })
10242024-03-28T08:17:34.875ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2784, tv_nsec: 431484313 } })
10252024-03-28T08:17:34.875ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10262024-03-28T08:17:34.875ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10272024-03-28T08:17:34.875ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
10282024-03-28T08:17:34.875ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
10292024-03-28T08:17:34.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
10302024-03-28T08:17:34.877ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10312024-03-28T08:17:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10322024-03-28T08:17:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
10332024-03-28T08:17:34.878ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10342024-03-28T08:17:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10352024-03-28T08:17:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10362024-03-28T08:17:34.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10372024-03-28T08:17:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10382024-03-28T08:17:34.879ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
10392024-03-28T08:17:34.879ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10402024-03-28T08:17:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10412024-03-28T08:17:34.879ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2784, tv_nsec: 436363388 } })
10422024-03-28T08:17:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10432024-03-28T08:17:34.880ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10442024-03-28T08:17:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10452024-03-28T08:17:34.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10462024-03-28T08:17:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10472024-03-28T08:17:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10482024-03-28T08:17:34.880ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
10492024-03-28T08:17:34.881ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
10502024-03-28T08:17:34.881ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
10512024-03-28T08:17:34.881ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
10522024-03-28T08:17:34.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10532024-03-28T08:17:34.881ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10542024-03-28T08:17:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10552024-03-28T08:17:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10562024-03-28T08:17:34.881ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
10572024-03-28T08:17:34.881ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
10582024-03-28T08:17:34.881ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
10592024-03-28T08:17:34.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10602024-03-28T08:17:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10612024-03-28T08:17:34.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10622024-03-28T08:17:34.882ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10632024-03-28T08:17:34.882ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10642024-03-28T08:17:34.882ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10652024-03-28T08:17:34.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
10662024-03-28T08:17:34.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10672024-03-28T08:17:34.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10682024-03-28T08:17:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10692024-03-28T08:17:34.886ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10702024-03-28T08:17:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
10712024-03-28T08:17:34.886ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10722024-03-28T08:17:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10732024-03-28T08:17:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10742024-03-28T08:17:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10752024-03-28T08:17:34.887ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10762024-03-28T08:17:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10772024-03-28T08:17:34.888ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10782024-03-28T08:17:34.888ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
10792024-03-28T08:17:34.888ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 0 version = 2
10802024-03-28T08:17:34.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10812024-03-28T08:17:34.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10822024-03-28T08:17:34.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10832024-03-28T08:17:34.888ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
10842024-03-28T08:17:34.888ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
10852024-03-28T08:17:34.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10862024-03-28T08:17:34.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10872024-03-28T08:17:34.889ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
10882024-03-28T08:17:34.889ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.227086ms iteration = 26
10892024-03-28T08:17:34.889ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10902024-03-28T08:17:34.889ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10912024-03-28T08:17:34.890ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10922024-03-28T08:17:34.891ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
10932024-03-28T08:17:34.891ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10942024-03-28T08:17:34.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10952024-03-28T08:17:34.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10962024-03-28T08:17:34.892ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10972024-03-28T08:17:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10982024-03-28T08:17:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10992024-03-28T08:17:34.893ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11002024-03-28T08:17:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11012024-03-28T08:17:34.894ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 1 version = 2
11022024-03-28T08:17:34.894ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
11032024-03-28T08:17:34.894ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-03-28 08:13:43.437455 UTC dns_group = external generation = 2
11042024-03-28T08:17:34.894ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 23.378296ms iteration = 6
11052024-03-28T08:17:34.895ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
11062024-03-28T08:17:34.895ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
11072024-03-28T08:17:34.895ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-03-28 08:13:42.962945 UTC dns_group = internal generation = 1
11082024-03-28T08:17:34.895ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 24.432204ms iteration = 6
11092024-03-28T08:17:34.895ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11102024-03-28T08:17:34.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11112024-03-28T08:17:34.897ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11122024-03-28T08:17:34.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11132024-03-28T08:17:34.898ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11142024-03-28T08:17:34.900ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
11152024-03-28T08:17:34.900ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11162024-03-28T08:17:34.900ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-28T08:13:34.933339Z, time_modified: 2024-03-28T08:13:34.933339Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }) }
11172024-03-28T08:17:34.900ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 30.352327ms iteration = 6
11182024-03-28T08:17:34.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11192024-03-28T08:17:34.902ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 31.623929ms iteration = 10
11202024-03-28T08:17:35.018ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
11212024-03-28T08:17:35.019ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11222024-03-28T08:17:35.019ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b0d3ccae-ccb9-42b7-ade0-03d28cb2e4d1", "content-length": "26", "date": "Thu, 28 Mar 2024 08:17:35 GMT"} })
11232024-03-28T08:17:35.019ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11242024-03-28T08:17:35.020ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11252024-03-28T08:17:35.129ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
11262024-03-28T08:17:35.152ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
11272024-03-28T08:17:35.152ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11282024-03-28T08:17:35.152ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2210a05b-4ab6-4f9d-849a-1b29eb40651c", "content-length": "26", "date": "Thu, 28 Mar 2024 08:17:35 GMT"} })
11292024-03-28T08:17:35.152ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11302024-03-28T08:17:35.154ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11312024-03-28T08:17:35.155ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
11322024-03-28T08:17:35.162ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
11332024-03-28T08:17:35.162ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "17ed5b8f-1868-44d0-bc61-c950d448f2cc", "content-length": "1", "date": "Thu, 28 Mar 2024 08:17:34 GMT"} })
11342024-03-28T08:17:35.164ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
11352024-03-28T08:17:35.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
11362024-03-28T08:17:35.166ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 295.639859ms iteration = 10
11372024-03-28T08:17:35.182ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
11382024-03-28T08:17:35.182ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11392024-03-28T08:17:35.183ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0ba785d2-7ccf-4836-8897-a0780d19095f", "content-length": "26", "date": "Thu, 28 Mar 2024 08:17:35 GMT"} })
11402024-03-28T08:17:35.183ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11412024-03-28T08:17:35.184ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
11422024-03-28T08:17:35.277ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 407.320137ms iteration = 10
11432024-03-28T08:17:35.283ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
11442024-03-28T08:17:35.363ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
11452024-03-28T08:17:35.364ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "081bd721-6632-42ce-8ee6-81ce32d35a2e", "date": "Thu, 28 Mar 2024 08:17:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11462024-03-28T08:17:35.364ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11472024-03-28T08:17:35.366ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "dac778b6-8248-4602-bd3f-4dff0ce1fc64", "date": "Thu, 28 Mar 2024 08:17:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11482024-03-28T08:17:35.366ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11492024-03-28T08:17:35.367ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11502024-03-28T08:17:35.367ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 496.531755ms iteration = 8
11512024-03-28T08:17:35.376ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
11522024-03-28T08:17:35.376ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b0532cc0-a7a4-4497-9895-73bd5036940a", "date": "Thu, 28 Mar 2024 08:17:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
11532024-03-28T08:17:35.376ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
11542024-03-28T08:17:35.378ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5a2c45cc-0ea2-426a-bb2f-5821e5bac9a4", "date": "Thu, 28 Mar 2024 08:17:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
11552024-03-28T08:17:35.378ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
11562024-03-28T08:17:35.380ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1660a3c5-bef6-4b7b-939e-e09520edde1c", "date": "Thu, 28 Mar 2024 08:17:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
11572024-03-28T08:17:35.380ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
11582024-03-28T08:17:35.381ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
11592024-03-28T08:17:35.381ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 510.724532ms iteration = 8
11602024-03-28T08:17:35.434ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11612024-03-28T08:17:35.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11622024-03-28T08:17:35.436ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
11632024-03-28T08:17:35.436ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11642024-03-28T08:17:35.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11652024-03-28T08:17:35.442ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
11662024-03-28T08:17:35.442ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "31ec4768-aeab-4869-ae39-74fc08044e26", "content-length": "121", "date": "Thu, 28 Mar 2024 08:17:34 GMT"} })
11672024-03-28T08:17:35.442ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
11682024-03-28T08:17:35.442ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
11692024-03-28T08:17:35.442ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
11702024-03-28T08:17:35.443ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
11712024-03-28T08:17:35.443ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
11722024-03-28T08:17:35.443ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
11732024-03-28T08:17:35.443ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
11742024-03-28T08:17:35.443ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11752024-03-28T08:17:35.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11762024-03-28T08:17:35.444ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
11772024-03-28T08:17:35.444ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "14e78a60-70d3-492a-b4fc-dc53385be95d", "content-length": "2", "date": "Thu, 28 Mar 2024 08:17:34 GMT"} })
11782024-03-28T08:17:35.444ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
11792024-03-28T08:17:35.444ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "686135cc-fa34-491d-b7a0-aa7336bcbc27", "content-length": "52", "date": "Thu, 28 Mar 2024 08:17:34 GMT"} })
11802024-03-28T08:17:35.444ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
11812024-03-28T08:17:35.444ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
11822024-03-28T08:17:35.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 135 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = fdd666d1-4f60-4e9a-9f88-d7cebf163501 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
11832024-03-28T08:17:35.492ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6139745a-63af-46d3-80db-eb09aead8a3d", "date": "Thu, 28 Mar 2024 08:17:35 GMT"} })
11842024-03-28T08:17:35.492ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
11852024-03-28T08:17:35.492ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
11862024-03-28T08:17:35.492ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "42028909-0d62-4cbf-a796-fbf8c1f811f2", "content-length": "452", "date": "Thu, 28 Mar 2024 08:17:35 GMT"} })
11872024-03-28T08:17:35.492ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11882024-03-28T08:17:35.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11892024-03-28T08:17:35.495ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11902024-03-28T08:17:35.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11912024-03-28T08:17:35.497ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11922024-03-28T08:17:35.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11932024-03-28T08:17:35.499ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11942024-03-28T08:17:35.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11952024-03-28T08:17:35.506ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 635.300462ms iteration = 10
11962024-03-28T08:17:44.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
11972024-03-28T08:17:44.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11982024-03-28T08:17:44.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11992024-03-28T08:17:44.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12002024-03-28T08:17:44.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12012024-03-28T08:17:44.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12022024-03-28T08:17:44.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12032024-03-28T08:17:44.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12042024-03-28T08:17:44.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12052024-03-28T08:17:44.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12062024-03-28T08:17:44.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
12072024-03-28T08:17:44.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12082024-03-28T08:17:44.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12092024-03-28T08:17:44.884ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
12102024-03-28T08:17:44.884ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 12.982975ms iteration = 27
12112024-03-28T08:17:45.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 224 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 6108c27c-41b2-462c-ad0a-24436a1a8609 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
12122024-03-28T08:17:54.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
12132024-03-28T08:17:54.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12142024-03-28T08:17:54.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12152024-03-28T08:17:54.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12162024-03-28T08:17:54.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12172024-03-28T08:17:54.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12182024-03-28T08:17:54.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12192024-03-28T08:17:54.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12202024-03-28T08:17:54.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12212024-03-28T08:17:54.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12222024-03-28T08:17:54.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
12232024-03-28T08:17:54.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12242024-03-28T08:17:54.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12252024-03-28T08:17:54.885ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
12262024-03-28T08:17:54.885ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.788616ms iteration = 28
12272024-03-28T08:17:55.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 148 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 3afa0ee4-ab79-4d2c-a01a-df8cbfc9d85f response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
12282024-03-28T08:18:04.870ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
12292024-03-28T08:18:04.870ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
12302024-03-28T08:18:04.870ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
12312024-03-28T08:18:04.870ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
12322024-03-28T08:18:04.870ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
12332024-03-28T08:18:04.870ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12342024-03-28T08:18:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
12352024-03-28T08:18:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12362024-03-28T08:18:04.871ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
12372024-03-28T08:18:04.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
12382024-03-28T08:18:04.871ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
12392024-03-28T08:18:04.871ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
12402024-03-28T08:18:04.871ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
12412024-03-28T08:18:04.871ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 4.76\u{b5}s iteration = 11
12422024-03-28T08:18:04.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12432024-03-28T08:18:04.871ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
12442024-03-28T08:18:04.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12452024-03-28T08:18:04.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12462024-03-28T08:18:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12472024-03-28T08:18:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12482024-03-28T08:18:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12492024-03-28T08:18:04.872ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2814, tv_nsec: 428804196 } })
12502024-03-28T08:18:04.873ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12512024-03-28T08:18:04.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12522024-03-28T08:18:04.873ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12532024-03-28T08:18:04.873ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2814, tv_nsec: 429709192 } })
12542024-03-28T08:18:04.873ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
12552024-03-28T08:18:04.873ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.163015ms iteration = 11
12562024-03-28T08:18:04.873ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
12572024-03-28T08:18:04.874ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
12582024-03-28T08:18:04.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
12592024-03-28T08:18:04.875ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12602024-03-28T08:18:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12612024-03-28T08:18:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
12622024-03-28T08:18:04.876ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12632024-03-28T08:18:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12642024-03-28T08:18:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12652024-03-28T08:18:04.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12662024-03-28T08:18:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12672024-03-28T08:18:04.878ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2814, tv_nsec: 434813940 } })
12682024-03-28T08:18:04.878ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12692024-03-28T08:18:04.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12702024-03-28T08:18:04.879ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
12712024-03-28T08:18:04.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12722024-03-28T08:18:04.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12732024-03-28T08:18:04.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12742024-03-28T08:18:04.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
12752024-03-28T08:18:04.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12762024-03-28T08:18:04.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12772024-03-28T08:18:04.887ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
12782024-03-28T08:18:04.887ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.689552ms iteration = 29
12792024-03-28T08:18:04.889ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12802024-03-28T08:18:04.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12812024-03-28T08:18:04.891ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12822024-03-28T08:18:04.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12832024-03-28T08:18:04.892ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12842024-03-28T08:18:04.895ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
12852024-03-28T08:18:04.895ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12862024-03-28T08:18:04.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12872024-03-28T08:18:04.896ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 25.434361ms iteration = 11
12882024-03-28T08:18:04.965ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
12892024-03-28T08:18:04.965ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12902024-03-28T08:18:04.965ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b1455be2-2660-4898-9735-ffca319c3ccc", "content-length": "26", "date": "Thu, 28 Mar 2024 08:18:04 GMT"} })
12912024-03-28T08:18:04.965ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12922024-03-28T08:18:04.966ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
12932024-03-28T08:18:05.058ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
12942024-03-28T08:18:05.058ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12952024-03-28T08:18:05.058ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fcd310a0-5f44-405f-a303-08792d6a9ea3", "content-length": "26", "date": "Thu, 28 Mar 2024 08:18:04 GMT"} })
12962024-03-28T08:18:05.058ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12972024-03-28T08:18:05.059ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12982024-03-28T08:18:05.067ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
12992024-03-28T08:18:05.067ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13002024-03-28T08:18:05.067ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9cf88e63-b45c-4cde-b202-aad58d9024a4", "content-length": "26", "date": "Thu, 28 Mar 2024 08:18:04 GMT"} })
13012024-03-28T08:18:05.067ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13022024-03-28T08:18:05.069ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13032024-03-28T08:18:05.110ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 239.990114ms iteration = 11
13042024-03-28T08:18:05.114ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
13052024-03-28T08:18:05.114ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2764419f-414c-4b5d-bcc4-ba0a7810695f", "content-length": "1", "date": "Thu, 28 Mar 2024 08:18:04 GMT"} })
13062024-03-28T08:18:05.116ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13072024-03-28T08:18:05.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13082024-03-28T08:18:05.117ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 247.390642ms iteration = 11
13092024-03-28T08:18:05.137ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13102024-03-28T08:18:05.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13112024-03-28T08:18:05.139ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
13122024-03-28T08:18:05.139ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13132024-03-28T08:18:05.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13142024-03-28T08:18:05.147ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
13152024-03-28T08:18:05.147ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "464cbe93-1b46-45da-83f0-ea0144a64454", "content-length": "121", "date": "Thu, 28 Mar 2024 08:18:04 GMT"} })
13162024-03-28T08:18:05.147ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
13172024-03-28T08:18:05.147ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
13182024-03-28T08:18:05.147ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
13192024-03-28T08:18:05.147ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
13202024-03-28T08:18:05.147ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
13212024-03-28T08:18:05.147ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
13222024-03-28T08:18:05.147ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
13232024-03-28T08:18:05.147ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13242024-03-28T08:18:05.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13252024-03-28T08:18:05.149ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
13262024-03-28T08:18:05.150ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fc503fa8-de39-4068-a2f2-e3a4d0713c58", "content-length": "2", "date": "Thu, 28 Mar 2024 08:18:04 GMT"} })
13272024-03-28T08:18:05.150ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
13282024-03-28T08:18:05.150ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "08142a61-ba71-4329-af40-9aa04e420539", "content-length": "52", "date": "Thu, 28 Mar 2024 08:18:04 GMT"} })
13292024-03-28T08:18:05.150ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
13302024-03-28T08:18:05.150ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
13312024-03-28T08:18:05.196ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "93016d78-5d53-47cc-ab37-93aa31eb8388", "date": "Thu, 28 Mar 2024 08:18:05 GMT"} })
13322024-03-28T08:18:05.196ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
13332024-03-28T08:18:05.196ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
13342024-03-28T08:18:05.197ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "16a1966a-b6ea-4cb5-a491-7dd884e8ba7c", "content-length": "452", "date": "Thu, 28 Mar 2024 08:18:05 GMT"} })
13352024-03-28T08:18:05.197ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13362024-03-28T08:18:05.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13372024-03-28T08:18:05.200ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13382024-03-28T08:18:05.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13392024-03-28T08:18:05.202ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13402024-03-28T08:18:05.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13412024-03-28T08:18:05.203ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13422024-03-28T08:18:05.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13432024-03-28T08:18:05.208ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 337.478805ms iteration = 11
13442024-03-28T08:18:05.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 132 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 39a57677-bf1c-427a-bcc0-683226f36249 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
13452024-03-28T08:18:14.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
13462024-03-28T08:18:14.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13472024-03-28T08:18:14.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13482024-03-28T08:18:14.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13492024-03-28T08:18:14.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13502024-03-28T08:18:14.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13512024-03-28T08:18:14.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13522024-03-28T08:18:14.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13532024-03-28T08:18:14.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13542024-03-28T08:18:14.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13552024-03-28T08:18:14.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
13562024-03-28T08:18:14.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13572024-03-28T08:18:14.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13582024-03-28T08:18:14.884ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
13592024-03-28T08:18:14.884ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.24922ms iteration = 30
13602024-03-28T08:18:15.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 211 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = a62b714e-cdef-47d9-aa5a-428ebb1115a2 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
13612024-03-28T08:18:24.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
13622024-03-28T08:18:24.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13632024-03-28T08:18:24.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13642024-03-28T08:18:24.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13652024-03-28T08:18:24.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13662024-03-28T08:18:24.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13672024-03-28T08:18:24.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13682024-03-28T08:18:24.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13692024-03-28T08:18:24.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13702024-03-28T08:18:24.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13712024-03-28T08:18:24.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
13722024-03-28T08:18:24.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13732024-03-28T08:18:24.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13742024-03-28T08:18:24.884ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
13752024-03-28T08:18:24.884ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.445263ms iteration = 31
13762024-03-28T08:18:25.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 908 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = e1c0be0e-f933-4569-9d15-5c14c69827e1 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
13772024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
13782024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13792024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
13802024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13812024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
13822024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
13832024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
13842024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
13852024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13862024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
13872024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
13882024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13892024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
13902024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
13912024-03-28T08:18:34.870ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
13922024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
13932024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
13942024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
13952024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13962024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13972024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
13982024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13992024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14002024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
14012024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
14022024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
14032024-03-28T08:18:34.870ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
14042024-03-28T08:18:34.870ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
14052024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.7\u{b5}s iteration = 12
14062024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14072024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
14082024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14092024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14102024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
14112024-03-28T08:18:34.870ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 807102f2-0ae5-4b05-920b-677957cec7b1
14122024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 7.26\u{b5}s iteration = 8
14132024-03-28T08:18:34.870ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("58a99ec9-4663-432a-a76d-aa94f13dfa3e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 2844, tv_nsec: 427137785 } })
14142024-03-28T08:18:34.870ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5df01d54-1bbe-4940-a2d2-252f1cdd47b1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8802bd0e-3008-47c5-a929-4b5b39083872.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 2844, tv_nsec: 427194156 } })
14152024-03-28T08:18:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14162024-03-28T08:18:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14172024-03-28T08:18:34.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14182024-03-28T08:18:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14192024-03-28T08:18:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14202024-03-28T08:18:34.871ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
14212024-03-28T08:18:34.871ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.863152ms iteration = 7
14222024-03-28T08:18:34.872ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14232024-03-28T08:18:34.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14242024-03-28T08:18:34.872ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14252024-03-28T08:18:34.872ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14262024-03-28T08:18:34.872ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14272024-03-28T08:18:34.873ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14282024-03-28T08:18:34.873ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
14292024-03-28T08:18:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.402579ms iteration = 12
14302024-03-28T08:18:34.873ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
14312024-03-28T08:18:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.558332ms iteration = 7
14322024-03-28T08:18:34.873ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2844, tv_nsec: 430036475 } })
14332024-03-28T08:18:34.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14342024-03-28T08:18:34.874ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2844, tv_nsec: 430393831 } })
14352024-03-28T08:18:34.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14362024-03-28T08:18:34.874ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
14372024-03-28T08:18:34.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
14382024-03-28T08:18:34.876ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14392024-03-28T08:18:34.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14402024-03-28T08:18:34.877ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
14412024-03-28T08:18:34.877ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
14422024-03-28T08:18:34.877ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14432024-03-28T08:18:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
14442024-03-28T08:18:34.878ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14452024-03-28T08:18:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14462024-03-28T08:18:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14472024-03-28T08:18:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14482024-03-28T08:18:34.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14492024-03-28T08:18:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
14502024-03-28T08:18:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14512024-03-28T08:18:34.879ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2844, tv_nsec: 435608752 } })
14522024-03-28T08:18:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
14532024-03-28T08:18:34.879ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
14542024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14552024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14562024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
14572024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
14582024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
14592024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14602024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14612024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
14622024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
14632024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
14642024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14652024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14662024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14672024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14682024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14692024-03-28T08:18:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14702024-03-28T08:18:34.881ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14712024-03-28T08:18:34.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14722024-03-28T08:18:34.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14732024-03-28T08:18:34.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14742024-03-28T08:18:34.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
14752024-03-28T08:18:34.885ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14762024-03-28T08:18:34.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
14772024-03-28T08:18:34.885ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14782024-03-28T08:18:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14792024-03-28T08:18:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
14802024-03-28T08:18:34.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14812024-03-28T08:18:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14822024-03-28T08:18:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14832024-03-28T08:18:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
14842024-03-28T08:18:34.886ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14852024-03-28T08:18:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14862024-03-28T08:18:34.887ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
14872024-03-28T08:18:34.887ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 0 version = 2
14882024-03-28T08:18:34.887ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14892024-03-28T08:18:34.887ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14902024-03-28T08:18:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14912024-03-28T08:18:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14922024-03-28T08:18:34.887ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
14932024-03-28T08:18:34.887ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
14942024-03-28T08:18:34.887ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14952024-03-28T08:18:34.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14962024-03-28T08:18:34.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14972024-03-28T08:18:34.889ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14982024-03-28T08:18:34.889ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14992024-03-28T08:18:34.890ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
15002024-03-28T08:18:34.890ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.656881ms iteration = 32
15012024-03-28T08:18:34.891ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
15022024-03-28T08:18:34.891ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15032024-03-28T08:18:34.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15042024-03-28T08:18:34.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
15052024-03-28T08:18:34.892ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15062024-03-28T08:18:34.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15072024-03-28T08:18:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15082024-03-28T08:18:34.893ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15092024-03-28T08:18:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15102024-03-28T08:18:34.893ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15112024-03-28T08:18:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15122024-03-28T08:18:34.893ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 1 version = 2
15132024-03-28T08:18:34.893ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
15142024-03-28T08:18:34.893ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-03-28 08:13:43.437455 UTC dns_group = external generation = 2
15152024-03-28T08:18:34.893ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 23.690631ms iteration = 7
15162024-03-28T08:18:34.894ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
15172024-03-28T08:18:34.894ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
15182024-03-28T08:18:34.894ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-03-28 08:13:42.962945 UTC dns_group = internal generation = 1
15192024-03-28T08:18:34.894ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 24.76335ms iteration = 7
15202024-03-28T08:18:34.895ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15212024-03-28T08:18:34.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15222024-03-28T08:18:34.896ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15232024-03-28T08:18:34.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
15242024-03-28T08:18:34.898ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15252024-03-28T08:18:34.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15262024-03-28T08:18:34.900ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-28T08:13:34.933339Z, time_modified: 2024-03-28T08:13:34.933339Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }) }
15272024-03-28T08:18:34.900ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 30.097692ms iteration = 7
15282024-03-28T08:18:34.900ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 29.575534ms iteration = 12
15292024-03-28T08:18:34.991ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
15302024-03-28T08:18:35.000ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
15312024-03-28T08:18:35.008ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
15322024-03-28T08:18:35.008ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15332024-03-28T08:18:35.008ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
15342024-03-28T08:18:35.008ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15352024-03-28T08:18:35.008ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7e5dffe9-5bf7-4f3b-8e17-083eb6ca102b", "content-length": "26", "date": "Thu, 28 Mar 2024 08:18:35 GMT"} })
15362024-03-28T08:18:35.008ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15372024-03-28T08:18:35.008ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6461cb59-a547-4549-9813-a60bde4ae986", "content-length": "26", "date": "Thu, 28 Mar 2024 08:18:35 GMT"} })
15382024-03-28T08:18:35.008ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15392024-03-28T08:18:35.008ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
15402024-03-28T08:18:35.008ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15412024-03-28T08:18:35.009ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3553b963-8dd5-4ce7-9e24-1e05923e67e6", "content-length": "26", "date": "Thu, 28 Mar 2024 08:18:35 GMT"} })
15422024-03-28T08:18:35.009ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15432024-03-28T08:18:35.015ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
15442024-03-28T08:18:35.016ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
15452024-03-28T08:18:35.016ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
15462024-03-28T08:18:35.164ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
15472024-03-28T08:18:35.177ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
15482024-03-28T08:18:35.178ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "421a87e4-4e90-4da4-b3df-b0ba4fb5a15b", "content-length": "1", "date": "Thu, 28 Mar 2024 08:18:34 GMT"} })
15492024-03-28T08:18:35.178ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15502024-03-28T08:18:35.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
15512024-03-28T08:18:35.180ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 310.207932ms iteration = 12
15522024-03-28T08:18:35.233ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
15532024-03-28T08:18:35.233ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d0711aa6-b73b-4e56-8b6e-7fdfcce4ed13", "date": "Thu, 28 Mar 2024 08:18:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
15542024-03-28T08:18:35.233ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
15552024-03-28T08:18:35.235ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "061f53e5-b392-4a3a-94f3-e6d77c36e67a", "date": "Thu, 28 Mar 2024 08:18:34 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
15562024-03-28T08:18:35.235ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
15572024-03-28T08:18:35.237ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
15582024-03-28T08:18:35.237ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 366.976837ms iteration = 9
15592024-03-28T08:18:35.252ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 382.351254ms iteration = 12
15602024-03-28T08:18:35.269ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
15612024-03-28T08:18:35.270ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d0c3ce0e-a9ae-4f2c-a0cc-7da0a042976c", "date": "Thu, 28 Mar 2024 08:18:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
15622024-03-28T08:18:35.270ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
15632024-03-28T08:18:35.271ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "85c25a52-e8cd-48be-a5d0-8a538fbe0d50", "date": "Thu, 28 Mar 2024 08:18:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
15642024-03-28T08:18:35.271ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
15652024-03-28T08:18:35.272ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e0df6ce7-82cd-4e5a-b0d0-cfbd72e71948", "date": "Thu, 28 Mar 2024 08:18:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
15662024-03-28T08:18:35.272ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
15672024-03-28T08:18:35.274ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
15682024-03-28T08:18:35.274ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 404.271434ms iteration = 9
15692024-03-28T08:18:35.297ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15702024-03-28T08:18:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15712024-03-28T08:18:35.299ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
15722024-03-28T08:18:35.299ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15732024-03-28T08:18:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15742024-03-28T08:18:35.307ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
15752024-03-28T08:18:35.307ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bb2662d5-08e5-4407-8ddf-00074501fb95", "content-length": "121", "date": "Thu, 28 Mar 2024 08:18:34 GMT"} })
15762024-03-28T08:18:35.307ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
15772024-03-28T08:18:35.307ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
15782024-03-28T08:18:35.308ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
15792024-03-28T08:18:35.308ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
15802024-03-28T08:18:35.308ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
15812024-03-28T08:18:35.308ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
15822024-03-28T08:18:35.308ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
15832024-03-28T08:18:35.308ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15842024-03-28T08:18:35.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15852024-03-28T08:18:35.310ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
15862024-03-28T08:18:35.310ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b5595741-05c4-4a75-82d3-25c6099b32fa", "content-length": "2", "date": "Thu, 28 Mar 2024 08:18:34 GMT"} })
15872024-03-28T08:18:35.310ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
15882024-03-28T08:18:35.310ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8ead36c2-0b70-4acc-9d40-357f35692974", "content-length": "52", "date": "Thu, 28 Mar 2024 08:18:34 GMT"} })
15892024-03-28T08:18:35.310ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
15902024-03-28T08:18:35.310ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
15912024-03-28T08:18:35.359ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6b7e4193-94cf-4910-8b27-fa7563fd7271", "date": "Thu, 28 Mar 2024 08:18:35 GMT"} })
15922024-03-28T08:18:35.359ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
15932024-03-28T08:18:35.359ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
15942024-03-28T08:18:35.359ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4577bcb4-3381-466e-8568-6de2ce1a251e", "content-length": "452", "date": "Thu, 28 Mar 2024 08:18:35 GMT"} })
15952024-03-28T08:18:35.359ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15962024-03-28T08:18:35.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15972024-03-28T08:18:35.362ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15982024-03-28T08:18:35.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15992024-03-28T08:18:35.364ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16002024-03-28T08:18:35.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16012024-03-28T08:18:35.365ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16022024-03-28T08:18:35.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16032024-03-28T08:18:35.372ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 501.342738ms iteration = 12
16042024-03-28T08:18:35.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 137 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = f44859e5-e7d0-43d5-841b-9b575a55d32a response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
16052024-03-28T08:18:44.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
16062024-03-28T08:18:44.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16072024-03-28T08:18:44.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16082024-03-28T08:18:44.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16092024-03-28T08:18:44.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16102024-03-28T08:18:44.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16112024-03-28T08:18:44.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16122024-03-28T08:18:44.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16132024-03-28T08:18:44.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16142024-03-28T08:18:44.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16152024-03-28T08:18:44.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
16162024-03-28T08:18:44.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16172024-03-28T08:18:44.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16182024-03-28T08:18:44.883ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
16192024-03-28T08:18:44.883ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.103057ms iteration = 33
16202024-03-28T08:18:45.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 137 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 7f0404dc-b3f5-4d08-a90e-ca685dd594d6 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
16212024-03-28T08:18:54.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
16222024-03-28T08:18:54.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16232024-03-28T08:18:54.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16242024-03-28T08:18:54.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16252024-03-28T08:18:54.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16262024-03-28T08:18:54.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16272024-03-28T08:18:54.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16282024-03-28T08:18:54.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16292024-03-28T08:18:54.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16302024-03-28T08:18:54.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16312024-03-28T08:18:54.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
16322024-03-28T08:18:54.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16332024-03-28T08:18:54.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16342024-03-28T08:18:54.885ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
16352024-03-28T08:18:54.885ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.947589ms iteration = 34
16362024-03-28T08:18:55.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 790 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 253dee72-7f61-4576-9245-a36a4eaf5406 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
16372024-03-28T08:19:04.869ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
16382024-03-28T08:19:04.869ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
16392024-03-28T08:19:04.869ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
16402024-03-28T08:19:04.869ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
16412024-03-28T08:19:04.869ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
16422024-03-28T08:19:04.869ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
16432024-03-28T08:19:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
16442024-03-28T08:19:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
16452024-03-28T08:19:04.871ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
16462024-03-28T08:19:04.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
16472024-03-28T08:19:04.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16482024-03-28T08:19:04.871ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
16492024-03-28T08:19:04.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16502024-03-28T08:19:04.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16512024-03-28T08:19:04.871ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
16522024-03-28T08:19:04.871ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
16532024-03-28T08:19:04.871ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
16542024-03-28T08:19:04.871ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 10.65\u{b5}s iteration = 13
16552024-03-28T08:19:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16562024-03-28T08:19:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16572024-03-28T08:19:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16582024-03-28T08:19:04.871ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
16592024-03-28T08:19:04.871ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.016115ms iteration = 13
16602024-03-28T08:19:04.871ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2874, tv_nsec: 428053607 } })
16612024-03-28T08:19:04.872ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
16622024-03-28T08:19:04.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16632024-03-28T08:19:04.872ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16642024-03-28T08:19:04.872ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2874, tv_nsec: 429311559 } })
16652024-03-28T08:19:04.872ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16662024-03-28T08:19:04.873ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
16672024-03-28T08:19:04.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
16682024-03-28T08:19:04.875ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16692024-03-28T08:19:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16702024-03-28T08:19:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
16712024-03-28T08:19:04.876ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16722024-03-28T08:19:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16732024-03-28T08:19:04.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16742024-03-28T08:19:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16752024-03-28T08:19:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16762024-03-28T08:19:04.878ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16772024-03-28T08:19:04.878ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2874, tv_nsec: 435322563 } })
16782024-03-28T08:19:04.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16792024-03-28T08:19:04.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16802024-03-28T08:19:04.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16812024-03-28T08:19:04.879ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
16822024-03-28T08:19:04.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16832024-03-28T08:19:04.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
16842024-03-28T08:19:04.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16852024-03-28T08:19:04.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16862024-03-28T08:19:04.886ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
16872024-03-28T08:19:04.886ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.455589ms iteration = 35
16882024-03-28T08:19:04.894ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16892024-03-28T08:19:04.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16902024-03-28T08:19:04.895ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16912024-03-28T08:19:04.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16922024-03-28T08:19:04.896ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16932024-03-28T08:19:04.899ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
16942024-03-28T08:19:04.899ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16952024-03-28T08:19:04.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16962024-03-28T08:19:04.900ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 29.617254ms iteration = 13
16972024-03-28T08:19:04.944ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
16982024-03-28T08:19:04.944ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16992024-03-28T08:19:04.945ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "64dcfbbf-26e0-47e5-8093-af7e8e455213", "content-length": "26", "date": "Thu, 28 Mar 2024 08:19:04 GMT"} })
17002024-03-28T08:19:04.945ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17012024-03-28T08:19:04.945ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
17022024-03-28T08:19:04.946ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
17032024-03-28T08:19:04.946ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17042024-03-28T08:19:04.946ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f1e36d0a-2f4a-4e5b-9ec8-457ff968f5bf", "content-length": "26", "date": "Thu, 28 Mar 2024 08:19:04 GMT"} })
17052024-03-28T08:19:04.946ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17062024-03-28T08:19:04.947ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
17072024-03-28T08:19:04.954ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
17082024-03-28T08:19:04.954ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17092024-03-28T08:19:04.954ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a6c7b12f-8a67-42a1-9584-29289a244a1a", "content-length": "26", "date": "Thu, 28 Mar 2024 08:19:04 GMT"} })
17102024-03-28T08:19:04.954ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17112024-03-28T08:19:04.955ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
17122024-03-28T08:19:05.015ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
17132024-03-28T08:19:05.015ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c8519919-064a-4538-a7c6-8d6ef7cdbfb6", "content-length": "1", "date": "Thu, 28 Mar 2024 08:19:04 GMT"} })
17142024-03-28T08:19:05.017ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
17152024-03-28T08:19:05.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
17162024-03-28T08:19:05.018ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 148.976035ms iteration = 13
17172024-03-28T08:19:05.020ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 150.626703ms iteration = 13
17182024-03-28T08:19:05.083ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17192024-03-28T08:19:05.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17202024-03-28T08:19:05.085ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
17212024-03-28T08:19:05.085ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17222024-03-28T08:19:05.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17232024-03-28T08:19:05.092ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
17242024-03-28T08:19:05.093ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "362bb186-b376-45f0-b03e-379bf5bd9eb1", "content-length": "121", "date": "Thu, 28 Mar 2024 08:19:04 GMT"} })
17252024-03-28T08:19:05.093ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
17262024-03-28T08:19:05.093ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
17272024-03-28T08:19:05.093ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
17282024-03-28T08:19:05.093ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
17292024-03-28T08:19:05.093ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
17302024-03-28T08:19:05.093ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
17312024-03-28T08:19:05.093ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
17322024-03-28T08:19:05.093ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17332024-03-28T08:19:05.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17342024-03-28T08:19:05.095ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
17352024-03-28T08:19:05.095ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9d3fb01b-475c-4bf9-8b42-112a0c1cc57b", "content-length": "2", "date": "Thu, 28 Mar 2024 08:19:04 GMT"} })
17362024-03-28T08:19:05.095ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
17372024-03-28T08:19:05.095ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1594d130-3e2a-4619-9d81-69e1f0da47e8", "content-length": "52", "date": "Thu, 28 Mar 2024 08:19:04 GMT"} })
17382024-03-28T08:19:05.095ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
17392024-03-28T08:19:05.095ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
17402024-03-28T08:19:05.139ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fe863ec0-9bd3-4c8a-8735-7093b3fb1821", "date": "Thu, 28 Mar 2024 08:19:05 GMT"} })
17412024-03-28T08:19:05.139ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
17422024-03-28T08:19:05.139ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
17432024-03-28T08:19:05.140ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5fef1b99-1d6c-4199-a511-294b948eb5e3", "content-length": "452", "date": "Thu, 28 Mar 2024 08:19:05 GMT"} })
17442024-03-28T08:19:05.140ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17452024-03-28T08:19:05.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17462024-03-28T08:19:05.142ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17472024-03-28T08:19:05.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17482024-03-28T08:19:05.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17492024-03-28T08:19:05.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17502024-03-28T08:19:05.146ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17512024-03-28T08:19:05.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17522024-03-28T08:19:05.152ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 281.794749ms iteration = 13
17532024-03-28T08:19:05.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 154 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 4b8b21e1-ccfb-4658-b61a-01dc4c48aaea response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
17542024-03-28T08:19:14.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
17552024-03-28T08:19:14.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17562024-03-28T08:19:14.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17572024-03-28T08:19:14.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17582024-03-28T08:19:14.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17592024-03-28T08:19:14.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17602024-03-28T08:19:14.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17612024-03-28T08:19:14.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17622024-03-28T08:19:14.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17632024-03-28T08:19:14.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17642024-03-28T08:19:14.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
17652024-03-28T08:19:14.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17662024-03-28T08:19:14.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17672024-03-28T08:19:14.884ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
17682024-03-28T08:19:14.884ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.969053ms iteration = 36
17692024-03-28T08:19:15.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 127 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 372baf62-950e-44d5-872d-1a3a6edb6ae4 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
17702024-03-28T08:19:24.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
17712024-03-28T08:19:24.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17722024-03-28T08:19:24.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17732024-03-28T08:19:24.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17742024-03-28T08:19:24.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17752024-03-28T08:19:24.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17762024-03-28T08:19:24.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17772024-03-28T08:19:24.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17782024-03-28T08:19:24.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17792024-03-28T08:19:24.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17802024-03-28T08:19:24.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
17812024-03-28T08:19:24.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17822024-03-28T08:19:24.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17832024-03-28T08:19:24.886ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
17842024-03-28T08:19:24.886ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.596211ms iteration = 37
17852024-03-28T08:19:25.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 124 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 2c396e24-9137-47b7-b49f-deb9d52c5952 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
17862024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
17872024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
17882024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
17892024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
17902024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
17912024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17922024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
17932024-03-28T08:19:34.870ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
17942024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
17952024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
17962024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
17972024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
17982024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
17992024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
18002024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18012024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
18022024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
18032024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18042024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
18052024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18062024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
18072024-03-28T08:19:34.870ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
18082024-03-28T08:19:34.870ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
18092024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.07\u{b5}s iteration = 14
18102024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
18112024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
18122024-03-28T08:19:34.870ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 807102f2-0ae5-4b05-920b-677957cec7b1
18132024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.71\u{b5}s iteration = 9
18142024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
18152024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18162024-03-28T08:19:34.870ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18172024-03-28T08:19:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
18182024-03-28T08:19:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
18192024-03-28T08:19:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18202024-03-28T08:19:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18212024-03-28T08:19:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18222024-03-28T08:19:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18232024-03-28T08:19:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18242024-03-28T08:19:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18252024-03-28T08:19:34.871ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5df01d54-1bbe-4940-a2d2-252f1cdd47b1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8802bd0e-3008-47c5-a929-4b5b39083872.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 2904, tv_nsec: 427697026 } })
18262024-03-28T08:19:34.871ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("58a99ec9-4663-432a-a76d-aa94f13dfa3e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 2904, tv_nsec: 427799767 } })
18272024-03-28T08:19:34.872ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18282024-03-28T08:19:34.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18292024-03-28T08:19:34.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18302024-03-28T08:19:34.873ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
18312024-03-28T08:19:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.488553ms iteration = 8
18322024-03-28T08:19:34.873ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18332024-03-28T08:19:34.873ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
18342024-03-28T08:19:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.211976ms iteration = 14
18352024-03-28T08:19:34.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18362024-03-28T08:19:34.873ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18372024-03-28T08:19:34.874ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2904, tv_nsec: 430353132 } })
18382024-03-28T08:19:34.874ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
18392024-03-28T08:19:34.874ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.343098ms iteration = 8
18402024-03-28T08:19:34.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18412024-03-28T08:19:34.874ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18422024-03-28T08:19:34.874ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18432024-03-28T08:19:34.874ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
18442024-03-28T08:19:34.875ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18452024-03-28T08:19:34.875ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2904, tv_nsec: 431840407 } })
18462024-03-28T08:19:34.875ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
18472024-03-28T08:19:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
18482024-03-28T08:19:34.878ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18492024-03-28T08:19:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18502024-03-28T08:19:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
18512024-03-28T08:19:34.878ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18522024-03-28T08:19:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18532024-03-28T08:19:34.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18542024-03-28T08:19:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18552024-03-28T08:19:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18562024-03-28T08:19:34.879ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
18572024-03-28T08:19:34.879ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18582024-03-28T08:19:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18592024-03-28T08:19:34.879ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18602024-03-28T08:19:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
18612024-03-28T08:19:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
18622024-03-28T08:19:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18632024-03-28T08:19:34.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18642024-03-28T08:19:34.881ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
18652024-03-28T08:19:34.881ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
18662024-03-28T08:19:34.881ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
18672024-03-28T08:19:34.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18682024-03-28T08:19:34.881ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
18692024-03-28T08:19:34.881ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
18702024-03-28T08:19:34.881ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
18712024-03-28T08:19:34.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18722024-03-28T08:19:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18732024-03-28T08:19:34.881ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18742024-03-28T08:19:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18752024-03-28T08:19:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18762024-03-28T08:19:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18772024-03-28T08:19:34.882ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2904, tv_nsec: 438370241 } })
18782024-03-28T08:19:34.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18792024-03-28T08:19:34.882ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
18802024-03-28T08:19:34.882ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18812024-03-28T08:19:34.882ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18822024-03-28T08:19:34.882ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18832024-03-28T08:19:34.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
18842024-03-28T08:19:34.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18852024-03-28T08:19:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18862024-03-28T08:19:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
18872024-03-28T08:19:34.886ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18882024-03-28T08:19:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
18892024-03-28T08:19:34.887ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18902024-03-28T08:19:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18912024-03-28T08:19:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18922024-03-28T08:19:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
18932024-03-28T08:19:34.887ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18942024-03-28T08:19:34.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18952024-03-28T08:19:34.888ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18962024-03-28T08:19:34.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18972024-03-28T08:19:34.888ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
18982024-03-28T08:19:34.888ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 0 version = 2
18992024-03-28T08:19:34.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19002024-03-28T08:19:34.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19012024-03-28T08:19:34.889ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
19022024-03-28T08:19:34.889ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
19032024-03-28T08:19:34.889ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19042024-03-28T08:19:34.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19052024-03-28T08:19:34.889ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19062024-03-28T08:19:34.889ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19072024-03-28T08:19:34.890ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
19082024-03-28T08:19:34.890ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.432427ms iteration = 38
19092024-03-28T08:19:34.890ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19102024-03-28T08:19:34.891ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19112024-03-28T08:19:34.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19122024-03-28T08:19:34.892ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
19132024-03-28T08:19:34.892ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19142024-03-28T08:19:34.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19152024-03-28T08:19:34.893ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19162024-03-28T08:19:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19172024-03-28T08:19:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19182024-03-28T08:19:34.893ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19192024-03-28T08:19:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19202024-03-28T08:19:34.894ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19212024-03-28T08:19:34.894ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 1 version = 2
19222024-03-28T08:19:34.894ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
19232024-03-28T08:19:34.894ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-03-28 08:13:43.437455 UTC dns_group = external generation = 2
19242024-03-28T08:19:34.894ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 23.938155ms iteration = 8
19252024-03-28T08:19:34.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19262024-03-28T08:19:34.894ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19272024-03-28T08:19:34.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19282024-03-28T08:19:34.896ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
19292024-03-28T08:19:34.896ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
19302024-03-28T08:19:34.896ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-03-28 08:13:42.962945 UTC dns_group = internal generation = 1
19312024-03-28T08:19:34.896ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 25.875999ms iteration = 8
19322024-03-28T08:19:34.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
19332024-03-28T08:19:34.896ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19342024-03-28T08:19:34.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19352024-03-28T08:19:34.897ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 27.150191ms iteration = 14
19362024-03-28T08:19:34.901ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-28T08:13:34.933339Z, time_modified: 2024-03-28T08:13:34.933339Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }) }
19372024-03-28T08:19:34.901ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 31.337004ms iteration = 8
19382024-03-28T08:19:35.018ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
19392024-03-28T08:19:35.148ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
19402024-03-28T08:19:35.150ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
19412024-03-28T08:19:35.151ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19422024-03-28T08:19:35.151ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d7e7c55d-6062-4e63-b7ab-f25dadda633a", "content-length": "26", "date": "Thu, 28 Mar 2024 08:19:35 GMT"} })
19432024-03-28T08:19:35.151ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19442024-03-28T08:19:35.153ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
19452024-03-28T08:19:35.155ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
19462024-03-28T08:19:35.155ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19472024-03-28T08:19:35.155ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0a79e818-fd6d-405a-810c-9d351cf8ee03", "content-length": "26", "date": "Thu, 28 Mar 2024 08:19:35 GMT"} })
19482024-03-28T08:19:35.155ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19492024-03-28T08:19:35.157ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
19502024-03-28T08:19:35.170ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
19512024-03-28T08:19:35.172ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
19522024-03-28T08:19:35.172ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19532024-03-28T08:19:35.172ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cbbf69ea-43c4-47a0-afc7-ff8d057540aa", "content-length": "26", "date": "Thu, 28 Mar 2024 08:19:35 GMT"} })
19542024-03-28T08:19:35.172ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19552024-03-28T08:19:35.174ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
19562024-03-28T08:19:35.291ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 421.237598ms iteration = 14
19572024-03-28T08:19:35.292ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
19582024-03-28T08:19:35.292ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "773bedea-b400-406a-923e-00803a4181a8", "content-length": "1", "date": "Thu, 28 Mar 2024 08:19:34 GMT"} })
19592024-03-28T08:19:35.294ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19602024-03-28T08:19:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19612024-03-28T08:19:35.295ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
19622024-03-28T08:19:35.296ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 425.487022ms iteration = 14
19632024-03-28T08:19:35.297ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d717ae73-1a18-4062-9489-8765ed790aef", "date": "Thu, 28 Mar 2024 08:19:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
19642024-03-28T08:19:35.297ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
19652024-03-28T08:19:35.299ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4c02003a-7711-4a46-a1f7-dc10c4596ed8", "date": "Thu, 28 Mar 2024 08:19:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
19662024-03-28T08:19:35.299ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
19672024-03-28T08:19:35.301ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
19682024-03-28T08:19:35.301ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 430.712793ms iteration = 10
19692024-03-28T08:19:35.302ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
19702024-03-28T08:19:35.303ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "16b916ac-135f-493a-9798-c3c092174785", "date": "Thu, 28 Mar 2024 08:19:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
19712024-03-28T08:19:35.303ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
19722024-03-28T08:19:35.304ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d5bba61c-e4f8-442f-a379-50be390b49c7", "date": "Thu, 28 Mar 2024 08:19:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
19732024-03-28T08:19:35.304ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
19742024-03-28T08:19:35.306ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e1dae028-54bd-4284-8787-36079c50f5e8", "date": "Thu, 28 Mar 2024 08:19:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
19752024-03-28T08:19:35.306ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
19762024-03-28T08:19:35.308ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
19772024-03-28T08:19:35.308ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 437.924268ms iteration = 10
19782024-03-28T08:19:35.359ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19792024-03-28T08:19:35.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19802024-03-28T08:19:35.361ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
19812024-03-28T08:19:35.361ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19822024-03-28T08:19:35.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19832024-03-28T08:19:35.366ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
19842024-03-28T08:19:35.366ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "252a2e1f-90e8-4e35-bc15-75ba1f2db02e", "content-length": "121", "date": "Thu, 28 Mar 2024 08:19:34 GMT"} })
19852024-03-28T08:19:35.366ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
19862024-03-28T08:19:35.366ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
19872024-03-28T08:19:35.367ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
19882024-03-28T08:19:35.367ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
19892024-03-28T08:19:35.367ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
19902024-03-28T08:19:35.367ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
19912024-03-28T08:19:35.367ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
19922024-03-28T08:19:35.367ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19932024-03-28T08:19:35.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19942024-03-28T08:19:35.369ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
19952024-03-28T08:19:35.369ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d78f968a-bf69-4cc8-ac36-e33037f28ec8", "content-length": "2", "date": "Thu, 28 Mar 2024 08:19:34 GMT"} })
19962024-03-28T08:19:35.369ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
19972024-03-28T08:19:35.369ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2f17ec89-dfac-4cbc-92fb-23fdd6b1f870", "content-length": "52", "date": "Thu, 28 Mar 2024 08:19:34 GMT"} })
19982024-03-28T08:19:35.369ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
19992024-03-28T08:19:35.369ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
20002024-03-28T08:19:35.415ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "74302ac1-da1a-44e4-a458-cff24d9a2073", "date": "Thu, 28 Mar 2024 08:19:35 GMT"} })
20012024-03-28T08:19:35.415ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
20022024-03-28T08:19:35.415ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
20032024-03-28T08:19:35.416ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a02f4d95-bde6-4cfc-918d-43744680d7bf", "content-length": "452", "date": "Thu, 28 Mar 2024 08:19:35 GMT"} })
20042024-03-28T08:19:35.416ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20052024-03-28T08:19:35.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20062024-03-28T08:19:35.419ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20072024-03-28T08:19:35.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20082024-03-28T08:19:35.421ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20092024-03-28T08:19:35.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20102024-03-28T08:19:35.422ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20112024-03-28T08:19:35.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20122024-03-28T08:19:35.428ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 557.727046ms iteration = 14
20132024-03-28T08:19:35.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 127 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 70d51a92-e229-4587-a035-252ec05a7432 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
20142024-03-28T08:19:44.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
20152024-03-28T08:19:44.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20162024-03-28T08:19:44.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20172024-03-28T08:19:44.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20182024-03-28T08:19:44.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20192024-03-28T08:19:44.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20202024-03-28T08:19:44.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20212024-03-28T08:19:44.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20222024-03-28T08:19:44.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20232024-03-28T08:19:44.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20242024-03-28T08:19:44.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
20252024-03-28T08:19:44.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20262024-03-28T08:19:44.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20272024-03-28T08:19:44.885ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
20282024-03-28T08:19:44.885ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.553793ms iteration = 39
20292024-03-28T08:19:45.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 219 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 782fed28-5be4-4b07-8acf-ea781459eeb8 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
20302024-03-28T08:19:54.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
20312024-03-28T08:19:54.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20322024-03-28T08:19:54.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20332024-03-28T08:19:54.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20342024-03-28T08:19:54.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20352024-03-28T08:19:54.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20362024-03-28T08:19:54.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20372024-03-28T08:19:54.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20382024-03-28T08:19:54.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20392024-03-28T08:19:54.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20402024-03-28T08:19:54.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
20412024-03-28T08:19:54.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20422024-03-28T08:19:54.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20432024-03-28T08:19:54.885ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
20442024-03-28T08:19:54.885ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.596853ms iteration = 40
20452024-03-28T08:19:55.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 194 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 69e43e2f-7821-4a12-9ffc-b0042179c0ac response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
20462024-03-28T08:20:04.870ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
20472024-03-28T08:20:04.870ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
20482024-03-28T08:20:04.870ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
20492024-03-28T08:20:04.870ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
20502024-03-28T08:20:04.870ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
20512024-03-28T08:20:04.870ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
20522024-03-28T08:20:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
20532024-03-28T08:20:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
20542024-03-28T08:20:04.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
20552024-03-28T08:20:04.871ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
20562024-03-28T08:20:04.871ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
20572024-03-28T08:20:04.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20582024-03-28T08:20:04.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20592024-03-28T08:20:04.871ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
20602024-03-28T08:20:04.871ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
20612024-03-28T08:20:04.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20622024-03-28T08:20:04.871ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
20632024-03-28T08:20:04.871ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.261\u{b5}s iteration = 15
20642024-03-28T08:20:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20652024-03-28T08:20:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20662024-03-28T08:20:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20672024-03-28T08:20:04.872ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2934, tv_nsec: 428854493 } })
20682024-03-28T08:20:04.872ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2934, tv_nsec: 429039976 } })
20692024-03-28T08:20:04.872ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
20702024-03-28T08:20:04.872ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.498643ms iteration = 15
20712024-03-28T08:20:04.873ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20722024-03-28T08:20:04.873ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
20732024-03-28T08:20:04.873ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20742024-03-28T08:20:04.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20752024-03-28T08:20:04.873ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
20762024-03-28T08:20:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
20772024-03-28T08:20:04.876ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20782024-03-28T08:20:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20792024-03-28T08:20:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20802024-03-28T08:20:04.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20812024-03-28T08:20:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
20822024-03-28T08:20:04.877ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20832024-03-28T08:20:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20842024-03-28T08:20:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20852024-03-28T08:20:04.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20862024-03-28T08:20:04.878ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20872024-03-28T08:20:04.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20882024-03-28T08:20:04.878ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2934, tv_nsec: 435218833 } })
20892024-03-28T08:20:04.879ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
20902024-03-28T08:20:04.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20912024-03-28T08:20:04.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20922024-03-28T08:20:04.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
20932024-03-28T08:20:04.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20942024-03-28T08:20:04.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20952024-03-28T08:20:04.886ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
20962024-03-28T08:20:04.886ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.451408ms iteration = 41
20972024-03-28T08:20:04.890ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20982024-03-28T08:20:04.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20992024-03-28T08:20:04.892ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21002024-03-28T08:20:04.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21012024-03-28T08:20:04.893ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21022024-03-28T08:20:04.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
21032024-03-28T08:20:04.896ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21042024-03-28T08:20:04.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21052024-03-28T08:20:04.898ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.619362ms iteration = 15
21062024-03-28T08:20:04.947ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
21072024-03-28T08:20:04.947ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21082024-03-28T08:20:04.948ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "195c969f-7e19-44eb-9957-7e60d54d8a01", "content-length": "26", "date": "Thu, 28 Mar 2024 08:20:04 GMT"} })
21092024-03-28T08:20:04.948ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21102024-03-28T08:20:04.948ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
21112024-03-28T08:20:04.950ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
21122024-03-28T08:20:04.950ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21132024-03-28T08:20:04.951ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "436ac054-d6c7-4ce8-af11-90dff0cbe5e3", "content-length": "26", "date": "Thu, 28 Mar 2024 08:20:04 GMT"} })
21142024-03-28T08:20:04.951ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21152024-03-28T08:20:04.951ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
21162024-03-28T08:20:04.951ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21172024-03-28T08:20:04.951ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2bdb827e-d973-4c15-8274-b03cdce2ec0a", "content-length": "26", "date": "Thu, 28 Mar 2024 08:20:04 GMT"} })
21182024-03-28T08:20:04.951ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21192024-03-28T08:20:04.954ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
21202024-03-28T08:20:04.954ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
21212024-03-28T08:20:05.034ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 164.413202ms iteration = 15
21222024-03-28T08:20:05.042ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
21232024-03-28T08:20:05.043ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "eecedcbd-e59c-41f9-b1fd-81a46b2477d4", "content-length": "1", "date": "Thu, 28 Mar 2024 08:20:04 GMT"} })
21242024-03-28T08:20:05.044ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
21252024-03-28T08:20:05.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
21262024-03-28T08:20:05.046ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 175.740429ms iteration = 15
21272024-03-28T08:20:05.100ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21282024-03-28T08:20:05.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21292024-03-28T08:20:05.103ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
21302024-03-28T08:20:05.103ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21312024-03-28T08:20:05.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21322024-03-28T08:20:05.109ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
21332024-03-28T08:20:05.110ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a71b2fd9-fa4b-4347-abb7-be50a3ff1118", "content-length": "121", "date": "Thu, 28 Mar 2024 08:20:04 GMT"} })
21342024-03-28T08:20:05.110ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
21352024-03-28T08:20:05.110ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
21362024-03-28T08:20:05.110ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
21372024-03-28T08:20:05.110ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
21382024-03-28T08:20:05.110ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
21392024-03-28T08:20:05.110ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
21402024-03-28T08:20:05.110ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
21412024-03-28T08:20:05.110ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21422024-03-28T08:20:05.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21432024-03-28T08:20:05.112ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
21442024-03-28T08:20:05.112ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "37aadcca-fb55-4f19-9feb-23674f828b29", "content-length": "2", "date": "Thu, 28 Mar 2024 08:20:04 GMT"} })
21452024-03-28T08:20:05.112ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
21462024-03-28T08:20:05.112ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "33117a92-b36c-4ea9-b62c-b0984a9355a7", "content-length": "52", "date": "Thu, 28 Mar 2024 08:20:04 GMT"} })
21472024-03-28T08:20:05.112ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
21482024-03-28T08:20:05.113ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
21492024-03-28T08:20:05.160ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8c584695-fd67-4472-b1d4-963c5f69db2e", "date": "Thu, 28 Mar 2024 08:20:05 GMT"} })
21502024-03-28T08:20:05.160ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
21512024-03-28T08:20:05.160ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
21522024-03-28T08:20:05.160ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f0baa0ed-77cb-453a-8424-0160b544e048", "content-length": "452", "date": "Thu, 28 Mar 2024 08:20:05 GMT"} })
21532024-03-28T08:20:05.160ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21542024-03-28T08:20:05.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21552024-03-28T08:20:05.163ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21562024-03-28T08:20:05.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21572024-03-28T08:20:05.165ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21582024-03-28T08:20:05.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21592024-03-28T08:20:05.167ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21602024-03-28T08:20:05.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21612024-03-28T08:20:05.176ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 304.790058ms iteration = 15
21622024-03-28T08:20:05.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 127 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 74f3aa29-2756-4440-b71e-3fd1ff0d56b6 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
21632024-03-28T08:20:14.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
21642024-03-28T08:20:14.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21652024-03-28T08:20:14.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21662024-03-28T08:20:14.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21672024-03-28T08:20:14.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21682024-03-28T08:20:14.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21692024-03-28T08:20:14.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21702024-03-28T08:20:14.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21712024-03-28T08:20:14.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21722024-03-28T08:20:14.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21732024-03-28T08:20:14.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
21742024-03-28T08:20:14.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21752024-03-28T08:20:14.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21762024-03-28T08:20:14.886ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
21772024-03-28T08:20:14.886ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.621941ms iteration = 42
21782024-03-28T08:20:15.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 128 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 502433df-8ae4-45b5-9742-f987176f87bb response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
21792024-03-28T08:20:24.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
21802024-03-28T08:20:24.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21812024-03-28T08:20:24.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21822024-03-28T08:20:24.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21832024-03-28T08:20:24.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21842024-03-28T08:20:24.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21852024-03-28T08:20:24.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21862024-03-28T08:20:24.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21872024-03-28T08:20:24.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21882024-03-28T08:20:24.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21892024-03-28T08:20:24.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
21902024-03-28T08:20:24.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21912024-03-28T08:20:24.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21922024-03-28T08:20:24.885ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
21932024-03-28T08:20:24.885ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.210577ms iteration = 43
21942024-03-28T08:20:25.458ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 705 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = afc386bc-ad7c-470a-8762-5aa50d78945a response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
21952024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
21962024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
21972024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
21982024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21992024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
22002024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
22012024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
22022024-03-28T08:20:34.870ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
22032024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
22042024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
22052024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
22062024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
22072024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
22082024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
22092024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22102024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22112024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
22122024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
22132024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22142024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
22152024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
22162024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22172024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
22182024-03-28T08:20:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22192024-03-28T08:20:34.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22202024-03-28T08:20:34.871ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
22212024-03-28T08:20:34.871ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
22222024-03-28T08:20:34.871ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
22232024-03-28T08:20:34.871ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
22242024-03-28T08:20:34.871ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
22252024-03-28T08:20:34.871ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.53\u{b5}s iteration = 16
22262024-03-28T08:20:34.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22272024-03-28T08:20:34.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
22282024-03-28T08:20:34.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22292024-03-28T08:20:34.871ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 807102f2-0ae5-4b05-920b-677957cec7b1
22302024-03-28T08:20:34.871ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.17\u{b5}s iteration = 10
22312024-03-28T08:20:34.871ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5df01d54-1bbe-4940-a2d2-252f1cdd47b1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8802bd0e-3008-47c5-a929-4b5b39083872.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 2964, tv_nsec: 427298481 } })
22322024-03-28T08:20:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22332024-03-28T08:20:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22342024-03-28T08:20:34.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22352024-03-28T08:20:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22362024-03-28T08:20:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22372024-03-28T08:20:34.872ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("58a99ec9-4663-432a-a76d-aa94f13dfa3e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 2964, tv_nsec: 428135695 } })
22382024-03-28T08:20:34.872ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
22392024-03-28T08:20:34.872ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.542415ms iteration = 9
22402024-03-28T08:20:34.872ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22412024-03-28T08:20:34.872ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22422024-03-28T08:20:34.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22432024-03-28T08:20:34.873ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
22442024-03-28T08:20:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.841639ms iteration = 16
22452024-03-28T08:20:34.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22462024-03-28T08:20:34.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22472024-03-28T08:20:34.873ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
22482024-03-28T08:20:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.222136ms iteration = 9
22492024-03-28T08:20:34.873ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22502024-03-28T08:20:34.874ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2964, tv_nsec: 430446686 } })
22512024-03-28T08:20:34.874ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22522024-03-28T08:20:34.874ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22532024-03-28T08:20:34.875ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
22542024-03-28T08:20:34.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
22552024-03-28T08:20:34.877ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22562024-03-28T08:20:34.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22572024-03-28T08:20:34.877ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2964, tv_nsec: 434102749 } })
22582024-03-28T08:20:34.878ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
22592024-03-28T08:20:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22602024-03-28T08:20:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22612024-03-28T08:20:34.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22622024-03-28T08:20:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22632024-03-28T08:20:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22642024-03-28T08:20:34.879ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
22652024-03-28T08:20:34.879ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22662024-03-28T08:20:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
22672024-03-28T08:20:34.879ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22682024-03-28T08:20:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22692024-03-28T08:20:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22702024-03-28T08:20:34.879ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
22712024-03-28T08:20:34.879ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
22722024-03-28T08:20:34.879ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
22732024-03-28T08:20:34.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22742024-03-28T08:20:34.879ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2964, tv_nsec: 436021212 } })
22752024-03-28T08:20:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22762024-03-28T08:20:34.880ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
22772024-03-28T08:20:34.880ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
22782024-03-28T08:20:34.880ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
22792024-03-28T08:20:34.880ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22802024-03-28T08:20:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22812024-03-28T08:20:34.881ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
22822024-03-28T08:20:34.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22832024-03-28T08:20:34.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22842024-03-28T08:20:34.881ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22852024-03-28T08:20:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22862024-03-28T08:20:34.881ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22872024-03-28T08:20:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22882024-03-28T08:20:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22892024-03-28T08:20:34.882ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22902024-03-28T08:20:34.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22912024-03-28T08:20:34.882ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22922024-03-28T08:20:34.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22932024-03-28T08:20:34.885ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22942024-03-28T08:20:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22952024-03-28T08:20:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22962024-03-28T08:20:34.886ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22972024-03-28T08:20:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
22982024-03-28T08:20:34.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22992024-03-28T08:20:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23002024-03-28T08:20:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23012024-03-28T08:20:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
23022024-03-28T08:20:34.887ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23032024-03-28T08:20:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23042024-03-28T08:20:34.887ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
23052024-03-28T08:20:34.887ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 0 version = 2
23062024-03-28T08:20:34.887ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23072024-03-28T08:20:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23082024-03-28T08:20:34.888ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
23092024-03-28T08:20:34.888ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
23102024-03-28T08:20:34.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23112024-03-28T08:20:34.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23122024-03-28T08:20:34.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23132024-03-28T08:20:34.888ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23142024-03-28T08:20:34.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23152024-03-28T08:20:34.889ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23162024-03-28T08:20:34.889ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23172024-03-28T08:20:34.891ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
23182024-03-28T08:20:34.891ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.14314ms iteration = 44
23192024-03-28T08:20:34.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
23202024-03-28T08:20:34.892ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23212024-03-28T08:20:34.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23222024-03-28T08:20:34.892ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
23232024-03-28T08:20:34.892ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23242024-03-28T08:20:34.892ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23252024-03-28T08:20:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23262024-03-28T08:20:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23272024-03-28T08:20:34.893ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 1 version = 2
23282024-03-28T08:20:34.893ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
23292024-03-28T08:20:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
23302024-03-28T08:20:34.893ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-03-28 08:13:43.437455 UTC dns_group = external generation = 2
23312024-03-28T08:20:34.893ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23322024-03-28T08:20:34.893ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 23.270543ms iteration = 9
23332024-03-28T08:20:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23342024-03-28T08:20:34.894ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23352024-03-28T08:20:34.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23362024-03-28T08:20:34.895ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
23372024-03-28T08:20:34.895ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
23382024-03-28T08:20:34.895ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-03-28 08:13:42.962945 UTC dns_group = internal generation = 1
23392024-03-28T08:20:34.895ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 24.909382ms iteration = 9
23402024-03-28T08:20:34.896ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23412024-03-28T08:20:34.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
23422024-03-28T08:20:34.898ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23432024-03-28T08:20:34.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23442024-03-28T08:20:34.900ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 29.268878ms iteration = 16
23452024-03-28T08:20:34.906ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-28T08:13:34.933339Z, time_modified: 2024-03-28T08:13:34.933339Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
23462024-03-28T08:20:34.907ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 36.790228ms iteration = 9
23472024-03-28T08:20:35.005ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
23482024-03-28T08:20:35.016ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
23492024-03-28T08:20:35.016ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
23502024-03-28T08:20:35.016ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
23512024-03-28T08:20:35.016ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ba353d1d-99da-4406-936c-b5b570a20453", "content-length": "26", "date": "Thu, 28 Mar 2024 08:20:35 GMT"} })
23522024-03-28T08:20:35.016ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
23532024-03-28T08:20:35.018ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
23542024-03-28T08:20:35.019ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
23552024-03-28T08:20:35.020ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
23562024-03-28T08:20:35.020ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ad70d5cb-1db6-4f27-bbae-ae43a92af5fe", "content-length": "26", "date": "Thu, 28 Mar 2024 08:20:35 GMT"} })
23572024-03-28T08:20:35.020ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
23582024-03-28T08:20:35.021ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
23592024-03-28T08:20:35.023ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
23602024-03-28T08:20:35.023ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
23612024-03-28T08:20:35.024ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c27272ea-e1c7-441f-851d-41a458c8ec1b", "content-length": "26", "date": "Thu, 28 Mar 2024 08:20:35 GMT"} })
23622024-03-28T08:20:35.024ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
23632024-03-28T08:20:35.026ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
23642024-03-28T08:20:35.143ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
23652024-03-28T08:20:35.143ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3d81e2b9-00c2-473b-b548-86f19dbdcebf", "date": "Thu, 28 Mar 2024 08:20:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
23662024-03-28T08:20:35.143ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
23672024-03-28T08:20:35.145ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4193c63c-8855-43dc-870a-94f8c41dd725", "date": "Thu, 28 Mar 2024 08:20:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
23682024-03-28T08:20:35.145ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
23692024-03-28T08:20:35.146ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
23702024-03-28T08:20:35.146ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 276.313874ms iteration = 11
23712024-03-28T08:20:35.157ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 287.406117ms iteration = 16
23722024-03-28T08:20:35.247ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
23732024-03-28T08:20:35.247ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d9f31f7b-134c-47fc-bdf0-1c3a52c8ad6a", "content-length": "2", "date": "Thu, 28 Mar 2024 08:20:35 GMT"} })
23742024-03-28T08:20:35.249ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
23752024-03-28T08:20:35.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
23762024-03-28T08:20:35.251ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
23772024-03-28T08:20:35.251ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 381.186043ms iteration = 16
23782024-03-28T08:20:35.309ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23792024-03-28T08:20:35.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23802024-03-28T08:20:35.310ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
23812024-03-28T08:20:35.311ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
23822024-03-28T08:20:35.311ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23832024-03-28T08:20:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23842024-03-28T08:20:35.311ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "10fa6d46-f3b6-4743-9730-68f843a772a5", "date": "Thu, 28 Mar 2024 08:20:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
23852024-03-28T08:20:35.311ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
23862024-03-28T08:20:35.313ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "91c8aacd-1957-4104-961d-09e105e0a079", "date": "Thu, 28 Mar 2024 08:20:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
23872024-03-28T08:20:35.313ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
23882024-03-28T08:20:35.315ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5fafd852-abb3-4802-9f2f-b1e3d4151313", "date": "Thu, 28 Mar 2024 08:20:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
23892024-03-28T08:20:35.315ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
23902024-03-28T08:20:35.317ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
23912024-03-28T08:20:35.317ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 447.24456ms iteration = 11
23922024-03-28T08:20:35.317ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
23932024-03-28T08:20:35.317ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "babc871f-8122-4968-8f46-6804612ac66f", "content-length": "121", "date": "Thu, 28 Mar 2024 08:20:35 GMT"} })
23942024-03-28T08:20:35.317ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
23952024-03-28T08:20:35.317ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
23962024-03-28T08:20:35.318ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
23972024-03-28T08:20:35.318ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
23982024-03-28T08:20:35.318ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
23992024-03-28T08:20:35.318ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
24002024-03-28T08:20:35.318ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
24012024-03-28T08:20:35.318ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24022024-03-28T08:20:35.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24032024-03-28T08:20:35.319ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
24042024-03-28T08:20:35.320ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f777e678-a2ca-4c8f-9afb-09c9db661ff0", "content-length": "2", "date": "Thu, 28 Mar 2024 08:20:35 GMT"} })
24052024-03-28T08:20:35.320ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
24062024-03-28T08:20:35.320ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b344a530-c727-4b12-8532-5acc6d4c1ea2", "content-length": "52", "date": "Thu, 28 Mar 2024 08:20:35 GMT"} })
24072024-03-28T08:20:35.320ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
24082024-03-28T08:20:35.320ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
24092024-03-28T08:20:35.366ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e559e9e8-09b0-4717-a326-28640a7c14ea", "date": "Thu, 28 Mar 2024 08:20:35 GMT"} })
24102024-03-28T08:20:35.366ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
24112024-03-28T08:20:35.366ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
24122024-03-28T08:20:35.367ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "24449f17-3d3f-4b62-bd8c-5b21a445b5f7", "content-length": "452", "date": "Thu, 28 Mar 2024 08:20:35 GMT"} })
24132024-03-28T08:20:35.367ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24142024-03-28T08:20:35.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24152024-03-28T08:20:35.370ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24162024-03-28T08:20:35.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24172024-03-28T08:20:35.372ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24182024-03-28T08:20:35.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24192024-03-28T08:20:35.373ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24202024-03-28T08:20:35.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24212024-03-28T08:20:35.380ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 509.52103ms iteration = 16
24222024-03-28T08:20:35.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 140 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 21d5307e-d5f6-484e-ae1d-99a530e69c5e response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
24232024-03-28T08:20:44.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
24242024-03-28T08:20:44.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24252024-03-28T08:20:44.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24262024-03-28T08:20:44.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24272024-03-28T08:20:44.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24282024-03-28T08:20:44.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24292024-03-28T08:20:44.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24302024-03-28T08:20:44.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24312024-03-28T08:20:44.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24322024-03-28T08:20:44.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24332024-03-28T08:20:44.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
24342024-03-28T08:20:44.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24352024-03-28T08:20:44.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24362024-03-28T08:20:44.885ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
24372024-03-28T08:20:44.885ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.43631ms iteration = 45
24382024-03-28T08:20:45.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 238 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = b86aff4b-399c-4568-b418-493db1b917b2 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
24392024-03-28T08:20:54.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
24402024-03-28T08:20:54.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24412024-03-28T08:20:54.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24422024-03-28T08:20:54.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24432024-03-28T08:20:54.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24442024-03-28T08:20:54.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24452024-03-28T08:20:54.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24462024-03-28T08:20:54.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24472024-03-28T08:20:54.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24482024-03-28T08:20:54.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24492024-03-28T08:20:54.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
24502024-03-28T08:20:54.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24512024-03-28T08:20:54.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24522024-03-28T08:20:54.886ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
24532024-03-28T08:20:54.886ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.840098ms iteration = 46
24542024-03-28T08:20:55.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 206 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 9e49f42c-f0e3-4ee1-9914-eba296cbc75b response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
24552024-03-28T08:21:04.870ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
24562024-03-28T08:21:04.870ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
24572024-03-28T08:21:04.870ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
24582024-03-28T08:21:04.870ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
24592024-03-28T08:21:04.870ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
24602024-03-28T08:21:04.870ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
24612024-03-28T08:21:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
24622024-03-28T08:21:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
24632024-03-28T08:21:04.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
24642024-03-28T08:21:04.871ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
24652024-03-28T08:21:04.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24662024-03-28T08:21:04.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24672024-03-28T08:21:04.871ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
24682024-03-28T08:21:04.871ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
24692024-03-28T08:21:04.871ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
24702024-03-28T08:21:04.871ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.8\u{b5}s iteration = 17
24712024-03-28T08:21:04.871ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
24722024-03-28T08:21:04.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24732024-03-28T08:21:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24742024-03-28T08:21:04.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24752024-03-28T08:21:04.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24762024-03-28T08:21:04.873ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2994, tv_nsec: 429142040 } })
24772024-03-28T08:21:04.873ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
24782024-03-28T08:21:04.873ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.507273ms iteration = 17
24792024-03-28T08:21:04.873ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24802024-03-28T08:21:04.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24812024-03-28T08:21:04.873ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24822024-03-28T08:21:04.873ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
24832024-03-28T08:21:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
24842024-03-28T08:21:04.876ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24852024-03-28T08:21:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24862024-03-28T08:21:04.876ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2994, tv_nsec: 432738102 } })
24872024-03-28T08:21:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
24882024-03-28T08:21:04.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24892024-03-28T08:21:04.876ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24902024-03-28T08:21:04.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24912024-03-28T08:21:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24922024-03-28T08:21:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24932024-03-28T08:21:04.877ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
24942024-03-28T08:21:04.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24952024-03-28T08:21:04.878ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2994, tv_nsec: 434711466 } })
24962024-03-28T08:21:04.878ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24972024-03-28T08:21:04.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24982024-03-28T08:21:04.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24992024-03-28T08:21:04.879ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
25002024-03-28T08:21:04.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25012024-03-28T08:21:04.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
25022024-03-28T08:21:04.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25032024-03-28T08:21:04.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25042024-03-28T08:21:04.886ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
25052024-03-28T08:21:04.886ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.671465ms iteration = 47
25062024-03-28T08:21:04.889ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25072024-03-28T08:21:04.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25082024-03-28T08:21:04.890ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25092024-03-28T08:21:04.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25102024-03-28T08:21:04.891ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25112024-03-28T08:21:04.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
25122024-03-28T08:21:04.894ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25132024-03-28T08:21:04.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25142024-03-28T08:21:04.895ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 23.791103ms iteration = 17
25152024-03-28T08:21:04.954ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
25162024-03-28T08:21:04.954ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25172024-03-28T08:21:04.954ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "19f452d9-406f-4368-8690-0a31e101a3bc", "content-length": "26", "date": "Thu, 28 Mar 2024 08:21:04 GMT"} })
25182024-03-28T08:21:04.954ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25192024-03-28T08:21:04.956ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
25202024-03-28T08:21:04.966ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
25212024-03-28T08:21:04.966ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25222024-03-28T08:21:04.966ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "37b38c24-7035-4e37-89ef-c28f8f262cd7", "content-length": "26", "date": "Thu, 28 Mar 2024 08:21:04 GMT"} })
25232024-03-28T08:21:04.966ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25242024-03-28T08:21:04.967ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
25252024-03-28T08:21:04.971ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
25262024-03-28T08:21:04.971ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25272024-03-28T08:21:04.972ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "eeb50a00-fadd-45a2-b91a-3abf78cc47bb", "content-length": "26", "date": "Thu, 28 Mar 2024 08:21:04 GMT"} })
25282024-03-28T08:21:04.972ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25292024-03-28T08:21:04.973ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
25302024-03-28T08:21:05.034ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
25312024-03-28T08:21:05.034ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e2c49502-c83a-4bb0-9e34-cc95219182f6", "content-length": "2", "date": "Thu, 28 Mar 2024 08:21:04 GMT"} })
25322024-03-28T08:21:05.036ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
25332024-03-28T08:21:05.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
25342024-03-28T08:21:05.037ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 167.279413ms iteration = 17
25352024-03-28T08:21:05.041ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 171.109138ms iteration = 17
25362024-03-28T08:21:05.101ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25372024-03-28T08:21:05.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25382024-03-28T08:21:05.103ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
25392024-03-28T08:21:05.103ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25402024-03-28T08:21:05.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25412024-03-28T08:21:05.111ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
25422024-03-28T08:21:05.111ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4a105e5c-21e3-4819-bee9-4b77dfa32cb9", "content-length": "121", "date": "Thu, 28 Mar 2024 08:21:05 GMT"} })
25432024-03-28T08:21:05.111ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
25442024-03-28T08:21:05.111ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
25452024-03-28T08:21:05.111ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
25462024-03-28T08:21:05.111ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
25472024-03-28T08:21:05.111ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
25482024-03-28T08:21:05.111ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
25492024-03-28T08:21:05.111ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
25502024-03-28T08:21:05.111ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25512024-03-28T08:21:05.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25522024-03-28T08:21:05.113ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
25532024-03-28T08:21:05.113ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f2ff1ad3-3819-4aee-913e-7a2842346472", "content-length": "2", "date": "Thu, 28 Mar 2024 08:21:05 GMT"} })
25542024-03-28T08:21:05.113ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
25552024-03-28T08:21:05.114ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d67de5d7-996a-4290-a8a9-ba485fc4431b", "content-length": "52", "date": "Thu, 28 Mar 2024 08:21:05 GMT"} })
25562024-03-28T08:21:05.114ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
25572024-03-28T08:21:05.114ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
25582024-03-28T08:21:05.161ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8c02e329-6324-4421-bee3-9fc1ec2dd78f", "date": "Thu, 28 Mar 2024 08:21:05 GMT"} })
25592024-03-28T08:21:05.161ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
25602024-03-28T08:21:05.161ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
25612024-03-28T08:21:05.161ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1369da06-509a-45a8-a693-d9109af47302", "content-length": "452", "date": "Thu, 28 Mar 2024 08:21:05 GMT"} })
25622024-03-28T08:21:05.161ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25632024-03-28T08:21:05.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25642024-03-28T08:21:05.164ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25652024-03-28T08:21:05.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25662024-03-28T08:21:05.166ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25672024-03-28T08:21:05.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25682024-03-28T08:21:05.167ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25692024-03-28T08:21:05.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25702024-03-28T08:21:05.174ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 302.498858ms iteration = 17
25712024-03-28T08:21:05.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 148 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = b5104fe7-c90e-4806-8477-8f7d242fcafd response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
25722024-03-28T08:21:14.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
25732024-03-28T08:21:14.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25742024-03-28T08:21:14.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25752024-03-28T08:21:14.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25762024-03-28T08:21:14.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25772024-03-28T08:21:14.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25782024-03-28T08:21:14.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25792024-03-28T08:21:14.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25802024-03-28T08:21:14.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25812024-03-28T08:21:14.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25822024-03-28T08:21:14.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
25832024-03-28T08:21:14.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25842024-03-28T08:21:14.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25852024-03-28T08:21:14.885ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
25862024-03-28T08:21:14.885ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.079304ms iteration = 48
25872024-03-28T08:21:15.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 206 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = a4b3f2a3-424f-4c21-958c-902dfaab52b5 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
25882024-03-28T08:21:24.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
25892024-03-28T08:21:24.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25902024-03-28T08:21:24.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25912024-03-28T08:21:24.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25922024-03-28T08:21:24.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25932024-03-28T08:21:24.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25942024-03-28T08:21:24.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25952024-03-28T08:21:24.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25962024-03-28T08:21:24.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25972024-03-28T08:21:24.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25982024-03-28T08:21:24.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
25992024-03-28T08:21:24.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26002024-03-28T08:21:24.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26012024-03-28T08:21:24.888ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
26022024-03-28T08:21:24.888ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.854042ms iteration = 49
26032024-03-28T08:21:25.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 204 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 77032af2-0acb-4c5e-a457-bb75d19e9cfa response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
26042024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
26052024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
26062024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
26072024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26082024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
26092024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
26102024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
26112024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
26122024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26132024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
26142024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26152024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
26162024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
26172024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
26182024-03-28T08:21:34.870ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
26192024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
26202024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
26212024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
26222024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
26232024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26242024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26252024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26262024-03-28T08:21:34.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
26272024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
26282024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
26292024-03-28T08:21:34.871ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
26302024-03-28T08:21:34.871ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
26312024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.21\u{b5}s iteration = 18
26322024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
26332024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
26342024-03-28T08:21:34.871ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 807102f2-0ae5-4b05-920b-677957cec7b1
26352024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26362024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.67\u{b5}s iteration = 11
26372024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
26382024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26392024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26402024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26412024-03-28T08:21:34.871ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("58a99ec9-4663-432a-a76d-aa94f13dfa3e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 3024, tv_nsec: 427815325 } })
26422024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26432024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26442024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26452024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26462024-03-28T08:21:34.871ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5df01d54-1bbe-4940-a2d2-252f1cdd47b1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8802bd0e-3008-47c5-a929-4b5b39083872.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 3024, tv_nsec: 427921027 } })
26472024-03-28T08:21:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26482024-03-28T08:21:34.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26492024-03-28T08:21:34.873ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26502024-03-28T08:21:34.873ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
26512024-03-28T08:21:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.190636ms iteration = 18
26522024-03-28T08:21:34.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26532024-03-28T08:21:34.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26542024-03-28T08:21:34.873ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
26552024-03-28T08:21:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.666974ms iteration = 10
26562024-03-28T08:21:34.873ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 3024, tv_nsec: 429993413 } })
26572024-03-28T08:21:34.874ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26582024-03-28T08:21:34.874ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 3024, tv_nsec: 430141165 } })
26592024-03-28T08:21:34.874ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26602024-03-28T08:21:34.874ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26612024-03-28T08:21:34.875ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
26622024-03-28T08:21:34.875ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 5.540736ms iteration = 10
26632024-03-28T08:21:34.875ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
26642024-03-28T08:21:34.875ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
26652024-03-28T08:21:34.877ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
26662024-03-28T08:21:34.877ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26672024-03-28T08:21:34.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26682024-03-28T08:21:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
26692024-03-28T08:21:34.878ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26702024-03-28T08:21:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26712024-03-28T08:21:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26722024-03-28T08:21:34.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26732024-03-28T08:21:34.879ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26742024-03-28T08:21:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26752024-03-28T08:21:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
26762024-03-28T08:21:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26772024-03-28T08:21:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
26782024-03-28T08:21:34.880ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 3024, tv_nsec: 436477465 } })
26792024-03-28T08:21:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
26802024-03-28T08:21:34.880ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26812024-03-28T08:21:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26822024-03-28T08:21:34.880ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
26832024-03-28T08:21:34.880ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
26842024-03-28T08:21:34.880ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
26852024-03-28T08:21:34.880ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26862024-03-28T08:21:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26872024-03-28T08:21:34.881ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26882024-03-28T08:21:34.881ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
26892024-03-28T08:21:34.881ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
26902024-03-28T08:21:34.881ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
26912024-03-28T08:21:34.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26922024-03-28T08:21:34.881ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
26932024-03-28T08:21:34.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26942024-03-28T08:21:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26952024-03-28T08:21:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26962024-03-28T08:21:34.882ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26972024-03-28T08:21:34.882ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26982024-03-28T08:21:34.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26992024-03-28T08:21:34.882ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27002024-03-28T08:21:34.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27012024-03-28T08:21:34.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
27022024-03-28T08:21:34.885ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
27032024-03-28T08:21:34.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
27042024-03-28T08:21:34.885ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
27052024-03-28T08:21:34.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
27062024-03-28T08:21:34.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
27072024-03-28T08:21:34.886ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
27082024-03-28T08:21:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
27092024-03-28T08:21:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
27102024-03-28T08:21:34.887ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
27112024-03-28T08:21:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
27122024-03-28T08:21:34.887ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
27132024-03-28T08:21:34.887ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 0 version = 2
27142024-03-28T08:21:34.887ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
27152024-03-28T08:21:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
27162024-03-28T08:21:34.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
27172024-03-28T08:21:34.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27182024-03-28T08:21:34.888ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27192024-03-28T08:21:34.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27202024-03-28T08:21:34.888ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
27212024-03-28T08:21:34.888ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
27222024-03-28T08:21:34.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
27232024-03-28T08:21:34.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27242024-03-28T08:21:34.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
27252024-03-28T08:21:34.889ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27262024-03-28T08:21:34.889ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
27272024-03-28T08:21:34.889ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
27282024-03-28T08:21:34.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
27292024-03-28T08:21:34.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
27302024-03-28T08:21:34.891ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
27312024-03-28T08:21:34.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
27322024-03-28T08:21:34.891ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
27332024-03-28T08:21:34.891ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.328702ms iteration = 50
27342024-03-28T08:21:34.892ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 1 version = 2
27352024-03-28T08:21:34.892ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
27362024-03-28T08:21:34.892ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-03-28 08:13:43.437455 UTC dns_group = external generation = 2
27372024-03-28T08:21:34.892ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 22.226686ms iteration = 10
27382024-03-28T08:21:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
27392024-03-28T08:21:34.893ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
27402024-03-28T08:21:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
27412024-03-28T08:21:34.895ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
27422024-03-28T08:21:34.895ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
27432024-03-28T08:21:34.895ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-03-28 08:13:42.962945 UTC dns_group = internal generation = 1
27442024-03-28T08:21:34.895ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 24.81807ms iteration = 10
27452024-03-28T08:21:34.895ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27462024-03-28T08:21:34.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27472024-03-28T08:21:34.897ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27482024-03-28T08:21:34.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27492024-03-28T08:21:34.898ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27502024-03-28T08:21:34.898ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-28T08:13:34.933339Z, time_modified: 2024-03-28T08:13:34.933339Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }) }
27512024-03-28T08:21:34.898ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 28.27706ms iteration = 10
27522024-03-28T08:21:34.901ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
27532024-03-28T08:21:34.901ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27542024-03-28T08:21:34.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27552024-03-28T08:21:34.902ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 31.12159ms iteration = 18
27562024-03-28T08:21:35.023ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
27572024-03-28T08:21:35.023ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27582024-03-28T08:21:35.023ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "084a7276-7bee-4418-93af-d28bfa9b26c2", "content-length": "26", "date": "Thu, 28 Mar 2024 08:21:35 GMT"} })
27592024-03-28T08:21:35.023ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27602024-03-28T08:21:35.025ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
27612024-03-28T08:21:35.133ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
27622024-03-28T08:21:35.153ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
27632024-03-28T08:21:35.163ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
27642024-03-28T08:21:35.163ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27652024-03-28T08:21:35.164ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3c7309ff-80a6-4ae1-a8b2-bf4b14fa93af", "content-length": "26", "date": "Thu, 28 Mar 2024 08:21:35 GMT"} })
27662024-03-28T08:21:35.164ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27672024-03-28T08:21:35.165ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
27682024-03-28T08:21:35.165ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27692024-03-28T08:21:35.166ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "29cefb24-1879-4b3e-9493-eec717514937", "content-length": "26", "date": "Thu, 28 Mar 2024 08:21:35 GMT"} })
27702024-03-28T08:21:35.166ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27712024-03-28T08:21:35.169ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
27722024-03-28T08:21:35.170ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
27732024-03-28T08:21:35.170ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 300.783599ms iteration = 18
27742024-03-28T08:21:35.265ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
27752024-03-28T08:21:35.265ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "85eb4cf5-454b-46cb-b259-def5e5b146ee", "date": "Thu, 28 Mar 2024 08:21:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
27762024-03-28T08:21:35.266ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
27772024-03-28T08:21:35.267ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5b99d799-a5ad-4209-874f-faa42b272546", "date": "Thu, 28 Mar 2024 08:21:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
27782024-03-28T08:21:35.267ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
27792024-03-28T08:21:35.269ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
27802024-03-28T08:21:35.269ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 398.84258ms iteration = 12
27812024-03-28T08:21:35.353ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
27822024-03-28T08:21:35.353ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6b6d3f84-4a3f-4fb7-b075-598d7b80b7e0", "content-length": "2", "date": "Thu, 28 Mar 2024 08:21:35 GMT"} })
27832024-03-28T08:21:35.356ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
27842024-03-28T08:21:35.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
27852024-03-28T08:21:35.357ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 487.463937ms iteration = 18
27862024-03-28T08:21:35.360ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
27872024-03-28T08:21:35.416ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27882024-03-28T08:21:35.417ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
27892024-03-28T08:21:35.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27902024-03-28T08:21:35.417ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f755483b-a110-4e91-8dbe-1d81e9a9cad7", "date": "Thu, 28 Mar 2024 08:21:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
27912024-03-28T08:21:35.417ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
27922024-03-28T08:21:35.419ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
27932024-03-28T08:21:35.419ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27942024-03-28T08:21:35.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27952024-03-28T08:21:35.419ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7ca28847-44bb-464e-8235-6aa40ef7d7c7", "date": "Thu, 28 Mar 2024 08:21:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
27962024-03-28T08:21:35.419ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
27972024-03-28T08:21:35.421ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "77b929ac-63c4-49ff-ae0f-eb476b0787cb", "date": "Thu, 28 Mar 2024 08:21:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
27982024-03-28T08:21:35.421ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
27992024-03-28T08:21:35.423ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
28002024-03-28T08:21:35.423ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 553.403991ms iteration = 12
28012024-03-28T08:21:35.425ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
28022024-03-28T08:21:35.425ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5fad8cbf-cdfa-4255-b9d1-45cc84f6d371", "content-length": "121", "date": "Thu, 28 Mar 2024 08:21:35 GMT"} })
28032024-03-28T08:21:35.425ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
28042024-03-28T08:21:35.425ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
28052024-03-28T08:21:35.426ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
28062024-03-28T08:21:35.426ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
28072024-03-28T08:21:35.426ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
28082024-03-28T08:21:35.426ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
28092024-03-28T08:21:35.426ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
28102024-03-28T08:21:35.426ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28112024-03-28T08:21:35.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28122024-03-28T08:21:35.428ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
28132024-03-28T08:21:35.428ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ba891e08-e7f0-4056-8e3e-9db59f9624aa", "content-length": "2", "date": "Thu, 28 Mar 2024 08:21:35 GMT"} })
28142024-03-28T08:21:35.428ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
28152024-03-28T08:21:35.428ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d6a5423f-d6c7-48f3-b326-9714679c146b", "content-length": "52", "date": "Thu, 28 Mar 2024 08:21:35 GMT"} })
28162024-03-28T08:21:35.428ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
28172024-03-28T08:21:35.428ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
28182024-03-28T08:21:35.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 113 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = ac331242-9b39-4a2c-bf3f-a62d8503a7c5 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
28192024-03-28T08:21:35.476ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "471fc61c-fa59-477a-b6f9-fa54ae40f91b", "date": "Thu, 28 Mar 2024 08:21:35 GMT"} })
28202024-03-28T08:21:35.476ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
28212024-03-28T08:21:35.476ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
28222024-03-28T08:21:35.478ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8a235314-930f-4c4a-9f27-1f2d89cb117b", "content-length": "452", "date": "Thu, 28 Mar 2024 08:21:35 GMT"} })
28232024-03-28T08:21:35.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28242024-03-28T08:21:35.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28252024-03-28T08:21:35.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28262024-03-28T08:21:35.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28272024-03-28T08:21:35.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28282024-03-28T08:21:35.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28292024-03-28T08:21:35.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28302024-03-28T08:21:35.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28312024-03-28T08:21:35.490ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 619.443517ms iteration = 18
28322024-03-28T08:21:44.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
28332024-03-28T08:21:44.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28342024-03-28T08:21:44.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28352024-03-28T08:21:44.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28362024-03-28T08:21:44.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28372024-03-28T08:21:44.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28382024-03-28T08:21:44.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28392024-03-28T08:21:44.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28402024-03-28T08:21:44.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28412024-03-28T08:21:44.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28422024-03-28T08:21:44.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
28432024-03-28T08:21:44.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28442024-03-28T08:21:44.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28452024-03-28T08:21:44.885ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
28462024-03-28T08:21:44.885ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.533192ms iteration = 51
28472024-03-28T08:21:45.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 583 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = d3e72c51-99d3-4438-b8f4-980de7ff095a response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
28482024-03-28T08:21:54.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
28492024-03-28T08:21:54.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28502024-03-28T08:21:54.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28512024-03-28T08:21:54.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28522024-03-28T08:21:54.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28532024-03-28T08:21:54.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28542024-03-28T08:21:54.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28552024-03-28T08:21:54.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28562024-03-28T08:21:54.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28572024-03-28T08:21:54.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28582024-03-28T08:21:54.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
28592024-03-28T08:21:54.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28602024-03-28T08:21:54.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28612024-03-28T08:21:54.885ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
28622024-03-28T08:21:54.885ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.564333ms iteration = 52
28632024-03-28T08:21:55.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 212 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = f199da0e-f94d-4e47-8b54-106b524f2020 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
28642024-03-28T08:22:04.870ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
28652024-03-28T08:22:04.870ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
28662024-03-28T08:22:04.870ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
28672024-03-28T08:22:04.870ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
28682024-03-28T08:22:04.870ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
28692024-03-28T08:22:04.870ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
28702024-03-28T08:22:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
28712024-03-28T08:22:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
28722024-03-28T08:22:04.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
28732024-03-28T08:22:04.871ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
28742024-03-28T08:22:04.871ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
28752024-03-28T08:22:04.871ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
28762024-03-28T08:22:04.871ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
28772024-03-28T08:22:04.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28782024-03-28T08:22:04.871ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.93\u{b5}s iteration = 19
28792024-03-28T08:22:04.871ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
28802024-03-28T08:22:04.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28812024-03-28T08:22:04.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28822024-03-28T08:22:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28832024-03-28T08:22:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28842024-03-28T08:22:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28852024-03-28T08:22:04.872ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 3054, tv_nsec: 428431841 } })
28862024-03-28T08:22:04.872ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
28872024-03-28T08:22:04.872ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.196578ms iteration = 19
28882024-03-28T08:22:04.872ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
28892024-03-28T08:22:04.873ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28902024-03-28T08:22:04.873ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 3054, tv_nsec: 429051821 } })
28912024-03-28T08:22:04.873ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
28922024-03-28T08:22:04.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28932024-03-28T08:22:04.873ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28942024-03-28T08:22:04.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
28952024-03-28T08:22:04.875ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28962024-03-28T08:22:04.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28972024-03-28T08:22:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28982024-03-28T08:22:04.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28992024-03-28T08:22:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29002024-03-28T08:22:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
29012024-03-28T08:22:04.877ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29022024-03-28T08:22:04.877ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 3054, tv_nsec: 433767853 } })
29032024-03-28T08:22:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29042024-03-28T08:22:04.878ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
29052024-03-28T08:22:04.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29062024-03-28T08:22:04.879ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29072024-03-28T08:22:04.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29082024-03-28T08:22:04.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29092024-03-28T08:22:04.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29102024-03-28T08:22:04.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
29112024-03-28T08:22:04.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29122024-03-28T08:22:04.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29132024-03-28T08:22:04.886ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
29142024-03-28T08:22:04.886ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.465318ms iteration = 53
29152024-03-28T08:22:04.894ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29162024-03-28T08:22:04.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29172024-03-28T08:22:04.895ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29182024-03-28T08:22:04.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29192024-03-28T08:22:04.896ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29202024-03-28T08:22:04.899ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
29212024-03-28T08:22:04.899ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29222024-03-28T08:22:04.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29232024-03-28T08:22:04.900ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 29.147006ms iteration = 19
29242024-03-28T08:22:04.977ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
29252024-03-28T08:22:04.977ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29262024-03-28T08:22:04.977ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b7d5e033-4ef0-407d-bfdc-9c983ebd7fd2", "content-length": "26", "date": "Thu, 28 Mar 2024 08:22:04 GMT"} })
29272024-03-28T08:22:04.977ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29282024-03-28T08:22:04.979ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29292024-03-28T08:22:05.070ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
29302024-03-28T08:22:05.070ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29312024-03-28T08:22:05.070ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c24fcdcf-10f1-413a-9821-057528fa4a44", "content-length": "26", "date": "Thu, 28 Mar 2024 08:22:04 GMT"} })
29322024-03-28T08:22:05.070ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29332024-03-28T08:22:05.073ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29342024-03-28T08:22:05.076ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
29352024-03-28T08:22:05.076ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29362024-03-28T08:22:05.076ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "90b41072-436b-4d4c-9215-858fe65b7a23", "content-length": "26", "date": "Thu, 28 Mar 2024 08:22:04 GMT"} })
29372024-03-28T08:22:05.076ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29382024-03-28T08:22:05.078ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
29392024-03-28T08:22:05.080ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 210.366809ms iteration = 19
29402024-03-28T08:22:05.131ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
29412024-03-28T08:22:05.131ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "02049428-409e-43a2-aed5-216721479780", "content-length": "2", "date": "Thu, 28 Mar 2024 08:22:04 GMT"} })
29422024-03-28T08:22:05.132ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
29432024-03-28T08:22:05.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
29442024-03-28T08:22:05.134ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 264.360826ms iteration = 19
29452024-03-28T08:22:05.192ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29462024-03-28T08:22:05.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29472024-03-28T08:22:05.196ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
29482024-03-28T08:22:05.196ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29492024-03-28T08:22:05.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29502024-03-28T08:22:05.203ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
29512024-03-28T08:22:05.204ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "669d7435-ad28-485b-9f87-b94581f9bd67", "content-length": "121", "date": "Thu, 28 Mar 2024 08:22:05 GMT"} })
29522024-03-28T08:22:05.204ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
29532024-03-28T08:22:05.204ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
29542024-03-28T08:22:05.204ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
29552024-03-28T08:22:05.204ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
29562024-03-28T08:22:05.204ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
29572024-03-28T08:22:05.204ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
29582024-03-28T08:22:05.204ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
29592024-03-28T08:22:05.204ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29602024-03-28T08:22:05.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29612024-03-28T08:22:05.206ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
29622024-03-28T08:22:05.206ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4ff4f80a-d2f5-4b77-8672-bfe27e8c8108", "content-length": "2", "date": "Thu, 28 Mar 2024 08:22:05 GMT"} })
29632024-03-28T08:22:05.206ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
29642024-03-28T08:22:05.206ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cbcee47e-1256-4c84-a5d3-88e164cff07f", "content-length": "52", "date": "Thu, 28 Mar 2024 08:22:05 GMT"} })
29652024-03-28T08:22:05.206ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
29662024-03-28T08:22:05.206ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
29672024-03-28T08:22:05.254ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "181fa11f-75d3-4edf-a0ac-a6304f793f70", "date": "Thu, 28 Mar 2024 08:22:05 GMT"} })
29682024-03-28T08:22:05.254ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
29692024-03-28T08:22:05.254ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
29702024-03-28T08:22:05.255ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f0c90d3e-ff03-492e-af3f-72664ba9714c", "content-length": "452", "date": "Thu, 28 Mar 2024 08:22:05 GMT"} })
29712024-03-28T08:22:05.255ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29722024-03-28T08:22:05.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29732024-03-28T08:22:05.257ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29742024-03-28T08:22:05.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29752024-03-28T08:22:05.260ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29762024-03-28T08:22:05.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29772024-03-28T08:22:05.261ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29782024-03-28T08:22:05.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29792024-03-28T08:22:05.266ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 394.978183ms iteration = 19
29802024-03-28T08:22:05.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 141 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = cf762b21-3ef8-47f3-b66a-d81873a5c41e response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
29812024-03-28T08:22:14.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
29822024-03-28T08:22:14.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29832024-03-28T08:22:14.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29842024-03-28T08:22:14.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29852024-03-28T08:22:14.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29862024-03-28T08:22:14.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29872024-03-28T08:22:14.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29882024-03-28T08:22:14.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29892024-03-28T08:22:14.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29902024-03-28T08:22:14.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29912024-03-28T08:22:14.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
29922024-03-28T08:22:14.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29932024-03-28T08:22:14.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29942024-03-28T08:22:14.886ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
29952024-03-28T08:22:14.886ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.482161ms iteration = 54
29962024-03-28T08:22:15.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 206 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 8f030f88-eb2c-47e0-a2e1-1c47a26d6b1f response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
29972024-03-28T08:22:24.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
29982024-03-28T08:22:24.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29992024-03-28T08:22:24.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30002024-03-28T08:22:24.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30012024-03-28T08:22:24.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30022024-03-28T08:22:24.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30032024-03-28T08:22:24.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30042024-03-28T08:22:24.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30052024-03-28T08:22:24.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30062024-03-28T08:22:24.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30072024-03-28T08:22:24.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
30082024-03-28T08:22:24.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30092024-03-28T08:22:24.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30102024-03-28T08:22:24.886ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
30112024-03-28T08:22:24.886ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.005291ms iteration = 55
30122024-03-28T08:22:25.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 212 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = d945433e-d642-41c6-af0c-1b958fb9aa47 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
30132024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
30142024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
30152024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
30162024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30172024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
30182024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
30192024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
30202024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
30212024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30222024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
30232024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30242024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
30252024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
30262024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
30272024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
30282024-03-28T08:22:34.870ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
30292024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
30302024-03-28T08:22:34.870ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
30312024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
30322024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30332024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30342024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30352024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
30362024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
30372024-03-28T08:22:34.871ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
30382024-03-28T08:22:34.871ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
30392024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
30402024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.78\u{b5}s iteration = 20
30412024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
30422024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30432024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
30442024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
30452024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30462024-03-28T08:22:34.871ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 807102f2-0ae5-4b05-920b-677957cec7b1
30472024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30482024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 7.47\u{b5}s iteration = 12
30492024-03-28T08:22:34.871ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5df01d54-1bbe-4940-a2d2-252f1cdd47b1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8802bd0e-3008-47c5-a929-4b5b39083872.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("cc978d71-bdf8-47d8-aea0-72f2d8e5cad8.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 3084, tv_nsec: 427379912 } })
30502024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30512024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30522024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30532024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30542024-03-28T08:22:34.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30552024-03-28T08:22:34.872ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("58a99ec9-4663-432a-a76d-aa94f13dfa3e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("887cc2b3-aa55-43ce-84d9-982ca1a97d5e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 3084, tv_nsec: 427949102 } })
30562024-03-28T08:22:34.872ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
30572024-03-28T08:22:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.182478ms iteration = 11
30582024-03-28T08:22:34.873ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
30592024-03-28T08:22:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.159398ms iteration = 11
30602024-03-28T08:22:34.873ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30612024-03-28T08:22:34.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30622024-03-28T08:22:34.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30632024-03-28T08:22:34.873ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
30642024-03-28T08:22:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.494763ms iteration = 20
30652024-03-28T08:22:34.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30662024-03-28T08:22:34.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30672024-03-28T08:22:34.873ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30682024-03-28T08:22:34.874ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30692024-03-28T08:22:34.874ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 3084, tv_nsec: 430715170 } })
30702024-03-28T08:22:34.874ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 3084, tv_nsec: 430863612 } })
30712024-03-28T08:22:34.875ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30722024-03-28T08:22:34.875ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
30732024-03-28T08:22:34.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
30742024-03-28T08:22:34.877ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30752024-03-28T08:22:34.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30762024-03-28T08:22:34.877ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
30772024-03-28T08:22:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
30782024-03-28T08:22:34.878ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
30792024-03-28T08:22:34.878ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30802024-03-28T08:22:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30812024-03-28T08:22:34.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30822024-03-28T08:22:34.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30832024-03-28T08:22:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30842024-03-28T08:22:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
30852024-03-28T08:22:34.879ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30862024-03-28T08:22:34.879ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
30872024-03-28T08:22:34.879ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
30882024-03-28T08:22:34.879ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
30892024-03-28T08:22:34.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30902024-03-28T08:22:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30912024-03-28T08:22:34.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30922024-03-28T08:22:34.879ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b628d990-af53-4085-954a-41da44d1109e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 3084, tv_nsec: 435699816 } })
30932024-03-28T08:22:34.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
30942024-03-28T08:22:34.880ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30952024-03-28T08:22:34.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30962024-03-28T08:22:34.880ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
30972024-03-28T08:22:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30982024-03-28T08:22:34.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30992024-03-28T08:22:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31002024-03-28T08:22:34.881ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31012024-03-28T08:22:34.881ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
31022024-03-28T08:22:34.881ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
31032024-03-28T08:22:34.881ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
31042024-03-28T08:22:34.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31052024-03-28T08:22:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31062024-03-28T08:22:34.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31072024-03-28T08:22:34.882ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31082024-03-28T08:22:34.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31092024-03-28T08:22:34.882ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31102024-03-28T08:22:34.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31112024-03-28T08:22:34.885ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31122024-03-28T08:22:34.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31132024-03-28T08:22:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
31142024-03-28T08:22:34.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31152024-03-28T08:22:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31162024-03-28T08:22:34.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31172024-03-28T08:22:34.886ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31182024-03-28T08:22:34.886ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
31192024-03-28T08:22:34.886ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 0 version = 2
31202024-03-28T08:22:34.886ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31212024-03-28T08:22:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31222024-03-28T08:22:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31232024-03-28T08:22:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
31242024-03-28T08:22:34.887ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31252024-03-28T08:22:34.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31262024-03-28T08:22:34.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31272024-03-28T08:22:34.888ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
31282024-03-28T08:22:34.888ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
31292024-03-28T08:22:34.888ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31302024-03-28T08:22:34.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31312024-03-28T08:22:34.888ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31322024-03-28T08:22:34.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31332024-03-28T08:22:34.889ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31342024-03-28T08:22:34.890ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31352024-03-28T08:22:34.890ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
31362024-03-28T08:22:34.890ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.323605ms iteration = 56
31372024-03-28T08:22:34.892ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
31382024-03-28T08:22:34.892ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31392024-03-28T08:22:34.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31402024-03-28T08:22:34.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31412024-03-28T08:22:34.892ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31422024-03-28T08:22:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31432024-03-28T08:22:34.893ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31442024-03-28T08:22:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31452024-03-28T08:22:34.893ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31462024-03-28T08:22:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31472024-03-28T08:22:34.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31482024-03-28T08:22:34.894ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 74470880-55af-4d52-ba54-2f16ae27dbb9 dns_zone_name = oxide.test found_so_far = 1 version = 2
31492024-03-28T08:22:34.894ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31502024-03-28T08:22:34.894ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
31512024-03-28T08:22:34.894ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-03-28 08:13:43.437455 UTC dns_group = external generation = 2
31522024-03-28T08:22:34.894ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 23.486437ms iteration = 11
31532024-03-28T08:22:34.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31542024-03-28T08:22:34.894ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b96a0124-5898-4f57-ab2e-0f17a67ca815 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
31552024-03-28T08:22:34.894ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
31562024-03-28T08:22:34.894ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-03-28 08:13:42.962945 UTC dns_group = internal generation = 1
31572024-03-28T08:22:34.894ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 24.089018ms iteration = 11
31582024-03-28T08:22:34.895ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31592024-03-28T08:22:34.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b628d990-af53-4085-954a-41da44d1109e, lookup_type: ById(b628d990-af53-4085-954a-41da44d1109e) } result = Ok(())
31602024-03-28T08:22:34.897ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31612024-03-28T08:22:34.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31622024-03-28T08:22:34.899ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.020126ms iteration = 20
31632024-03-28T08:22:34.905ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-03-28T08:13:34.933339Z, time_modified: 2024-03-28T08:13:34.933339Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, db_silo: Silo { identity: SiloIdentity { id: ddd43748-3439-46b0-b7f1-7d05c1f224ad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-03-28T08:13:43.396849Z, time_modified: 2024-03-28T08:13:43.396849Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "0cf11dcb59b746f0c86bdb7502ef782e2abd22ca8be8f750b4b39e9d86d54f10" }] }) }
31642024-03-28T08:22:34.905ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 34.538509ms iteration = 11
31652024-03-28T08:22:34.988ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
31662024-03-28T08:22:34.999ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
31672024-03-28T08:22:34.999ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
31682024-03-28T08:22:34.999ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "550e477a-c292-406b-a46a-1b98d6b079ed", "content-length": "26", "date": "Thu, 28 Mar 2024 08:22:34 GMT"} })
31692024-03-28T08:22:34.999ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
31702024-03-28T08:22:35.001ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
31712024-03-28T08:22:35.016ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
31722024-03-28T08:22:35.016ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
31732024-03-28T08:22:35.017ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "95781cc5-4f86-45ea-8d08-ba083195a447", "content-length": "26", "date": "Thu, 28 Mar 2024 08:22:34 GMT"} })
31742024-03-28T08:22:35.017ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
31752024-03-28T08:22:35.019ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
31762024-03-28T08:22:35.019ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
31772024-03-28T08:22:35.028ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 zone_address = fd00:1122:3344:101::2
31782024-03-28T08:22:35.028ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
31792024-03-28T08:22:35.028ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3b6f067a-dd69-4bfc-938a-75a6042a3a92", "content-length": "26", "date": "Thu, 28 Mar 2024 08:22:34 GMT"} })
31802024-03-28T08:22:35.028ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
31812024-03-28T08:22:35.029ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
31822024-03-28T08:22:35.261ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
31832024-03-28T08:22:35.270ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 399.801006ms iteration = 20
31842024-03-28T08:22:35.293ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
31852024-03-28T08:22:35.293ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7d5743df-6b3e-4ba0-b4ef-13330e08162b", "content-length": "2", "date": "Thu, 28 Mar 2024 08:22:35 GMT"} })
31862024-03-28T08:22:35.294ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
31872024-03-28T08:22:35.294ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "60c38ca1-d05a-4fd7-ac6e-a09f3923a2b1", "date": "Thu, 28 Mar 2024 08:22:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
31882024-03-28T08:22:35.294ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
31892024-03-28T08:22:35.295ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
31902024-03-28T08:22:35.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
31912024-03-28T08:22:35.296ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b91ab98f-c916-4a24-b8c6-a4b5504132b5", "date": "Thu, 28 Mar 2024 08:22:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
31922024-03-28T08:22:35.296ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
31932024-03-28T08:22:35.297ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 426.659412ms iteration = 20
31942024-03-28T08:22:35.298ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
31952024-03-28T08:22:35.298ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 427.559818ms iteration = 13
31962024-03-28T08:22:35.333ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
31972024-03-28T08:22:35.334ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "33fa67d0-d9d5-4b5a-9b37-1a31b0b36d48", "date": "Thu, 28 Mar 2024 08:22:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
31982024-03-28T08:22:35.334ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
31992024-03-28T08:22:35.335ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d0afc876-91dc-4775-8a74-4079b80ecb8e", "date": "Thu, 28 Mar 2024 08:22:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
32002024-03-28T08:22:35.335ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
32012024-03-28T08:22:35.337ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3551af0a-1983-4d3d-a4a7-f75f9d6c3fc8", "date": "Thu, 28 Mar 2024 08:22:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
32022024-03-28T08:22:35.337ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
32032024-03-28T08:22:35.338ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
32042024-03-28T08:22:35.338ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 467.380249ms iteration = 13
32052024-03-28T08:22:35.348ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32062024-03-28T08:22:35.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32072024-03-28T08:22:35.350ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 2b282d64-1c1d-4b22-a29a-9722fbde6e67, rack_id: 3ea8fde4-479e-43ee-8ad7-3aed785da4a9, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1) } rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
32082024-03-28T08:22:35.350ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32092024-03-28T08:22:35.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32102024-03-28T08:22:35.356ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
32112024-03-28T08:22:35.356ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4d3e6854-9133-4fa7-847f-bf483dd6cd52", "content-length": "121", "date": "Thu, 28 Mar 2024 08:22:35 GMT"} })
32122024-03-28T08:22:35.356ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
32132024-03-28T08:22:35.356ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = b7b6e89f-2f78-492c-8aec-7c23b6f4ecb1
32142024-03-28T08:22:35.356ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
32152024-03-28T08:22:35.356ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
32162024-03-28T08:22:35.356ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
32172024-03-28T08:22:35.356ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 routes = {}
32182024-03-28T08:22:35.356ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9
32192024-03-28T08:22:35.356ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32202024-03-28T08:22:35.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32212024-03-28T08:22:35.358ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
32222024-03-28T08:22:35.358ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "297e3cf9-19b1-4e66-b81a-213565ccf46c", "content-length": "2", "date": "Thu, 28 Mar 2024 08:22:35 GMT"} })
32232024-03-28T08:22:35.358ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
32242024-03-28T08:22:35.358ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f64012e2-fa9d-4f0b-bb65-26fc02a720af", "content-length": "52", "date": "Thu, 28 Mar 2024 08:22:35 GMT"} })
32252024-03-28T08:22:35.358ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch_location = Switch0
32262024-03-28T08:22:35.358ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
32272024-03-28T08:22:35.404ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e1efb5bd-0701-476d-bd3f-9628b4ae3906", "date": "Thu, 28 Mar 2024 08:22:35 GMT"} })
32282024-03-28T08:22:35.404ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 switch = Switch0
32292024-03-28T08:22:35.404ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
32302024-03-28T08:22:35.404ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3ea8fde4-479e-43ee-8ad7-3aed785da4a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "30c351ca-efe3-495e-b266-c4794d03e335", "content-length": "452", "date": "Thu, 28 Mar 2024 08:22:35 GMT"} })
32312024-03-28T08:22:35.404ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32322024-03-28T08:22:35.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32332024-03-28T08:22:35.407ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32342024-03-28T08:22:35.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32352024-03-28T08:22:35.410ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32362024-03-28T08:22:35.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32372024-03-28T08:22:35.411ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32382024-03-28T08:22:35.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32392024-03-28T08:22:35.417ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 546.353289ms iteration = 20
32402024-03-28T08:22:35.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 642 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = 03d370d8-26ad-460b-8c44-0009e4f3ebaa response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c
32412024-03-28T08:22:44.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
32422024-03-28T08:22:44.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32432024-03-28T08:22:44.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32442024-03-28T08:22:44.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32452024-03-28T08:22:44.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
32462024-03-28T08:22:44.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32472024-03-28T08:22:44.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32482024-03-28T08:22:44.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32492024-03-28T08:22:44.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32502024-03-28T08:22:44.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32512024-03-28T08:22:44.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 807102f2-0ae5-4b05-920b-677957cec7b1, lookup_type: ById(807102f2-0ae5-4b05-920b-677957cec7b1) } result = Ok(())
32522024-03-28T08:22:44.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32532024-03-28T08:22:44.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32542024-03-28T08:22:44.885ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 original_time_created = 2024-03-28 08:13:29.876968 UTC target_id = 807102f2-0ae5-4b05-920b-677957cec7b1 time_created = 2024-03-28 08:13:29.876968 UTC
32552024-03-28T08:22:44.885ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.105855ms iteration = 57
32562024-03-28T08:22:45.458ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 787 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64557 req_id = adb696c7-6ee9-47fb-a5ce-0ce01347eb27 response_code = 200 uri = /metrics/collect/9a1d7070-92a1-4f06-9f51-04369e671d7c