Max level shown:
12024-05-14T04:15:04.517ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 11 reason = Timeout
22024-05-14T04:15:04.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32024-05-14T04:15:04.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42024-05-14T04:15:04.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52024-05-14T04:15:04.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
62024-05-14T04:15:04.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72024-05-14T04:15:04.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82024-05-14T04:15:04.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92024-05-14T04:15:04.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
102024-05-14T04:15:04.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112024-05-14T04:15:04.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
122024-05-14T04:15:04.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132024-05-14T04:15:04.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
142024-05-14T04:15:04.533ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
152024-05-14T04:15:04.533ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.557116ms iteration = 11
162024-05-14T04:15:05.331ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 759 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 723fa37e-6e8f-4d03-b3a1-f776f2351e97 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
172024-05-14T04:15:14.517ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 12 reason = Timeout
182024-05-14T04:15:14.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
192024-05-14T04:15:14.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
202024-05-14T04:15:14.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212024-05-14T04:15:14.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
222024-05-14T04:15:14.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
232024-05-14T04:15:14.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
242024-05-14T04:15:14.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
252024-05-14T04:15:14.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
262024-05-14T04:15:14.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272024-05-14T04:15:14.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
282024-05-14T04:15:14.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
292024-05-14T04:15:14.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
302024-05-14T04:15:14.535ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
312024-05-14T04:15:14.535ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.883634ms iteration = 12
322024-05-14T04:15:15.331ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 141 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 14af592d-81b9-48e6-82b7-5adf07c572da response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
332024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
342024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 3 reason = Timeout
352024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
362024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 4 reason = Timeout
372024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
382024-05-14T04:15:24.517ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce
392024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
402024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.65\u{b5}s iteration = 4
412024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 5 reason = Timeout
422024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 5 reason = Timeout
432024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 3 reason = Timeout
442024-05-14T04:15:24.517ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
452024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 14\u{b5}s iteration = 3
462024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 3 reason = Timeout
472024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
482024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 3 reason = Timeout
492024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
502024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
512024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
522024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 5 reason = Timeout
532024-05-14T04:15:24.517ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
542024-05-14T04:15:24.517ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
552024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.531\u{b5}s iteration = 5
562024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 3 reason = Timeout
572024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
582024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 3 reason = Timeout
592024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
602024-05-14T04:15:24.517ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
612024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 5 reason = Timeout
622024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
632024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
642024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 6 reason = Timeout
652024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
662024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 5 reason = Timeout
672024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
682024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
692024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
702024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
712024-05-14T04:15:24.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
722024-05-14T04:15:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
732024-05-14T04:15:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
742024-05-14T04:15:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
752024-05-14T04:15:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
762024-05-14T04:15:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
772024-05-14T04:15:24.518ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("08a9c078-20b8-4325-90f1-e360e31ca4ab.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("3f6ad63d-f2d7-4b36-a169-d3099a239267.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("3f6ad63d-f2d7-4b36-a169-d3099a239267.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 6749, tv_nsec: 180686348 } })
782024-05-14T04:15:24.518ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
792024-05-14T04:15:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
802024-05-14T04:15:24.518ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0dd5d513-ee28-4da3-83f2-7a3d7e3b267d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6aa90bd6-c3db-4dfa-88c0-97308b03c15f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9a8a4960-0835-4c5e-b472-b2521417e003.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9a8a4960-0835-4c5e-b472-b2521417e003.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6749, tv_nsec: 180751459 } })
812024-05-14T04:15:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
822024-05-14T04:15:24.518ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
832024-05-14T04:15:24.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
842024-05-14T04:15:24.519ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
852024-05-14T04:15:24.519ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.177234ms iteration = 3
862024-05-14T04:15:24.519ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
872024-05-14T04:15:24.520ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
882024-05-14T04:15:24.520ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
892024-05-14T04:15:24.521ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
902024-05-14T04:15:24.521ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.49024ms iteration = 3
912024-05-14T04:15:24.521ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
922024-05-14T04:15:24.521ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
932024-05-14T04:15:24.521ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.705206ms iteration = 5
942024-05-14T04:15:24.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
952024-05-14T04:15:24.522ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6749, tv_nsec: 184876553 } })
962024-05-14T04:15:24.522ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6749, tv_nsec: 185034596 } })
972024-05-14T04:15:24.522ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
982024-05-14T04:15:24.523ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 5 reason = Timeout
992024-05-14T04:15:24.523ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1002024-05-14T04:15:24.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1012024-05-14T04:15:24.524ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
1022024-05-14T04:15:24.525ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1032024-05-14T04:15:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1042024-05-14T04:15:24.525ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1052024-05-14T04:15:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1062024-05-14T04:15:24.525ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1072024-05-14T04:15:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1082024-05-14T04:15:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1092024-05-14T04:15:24.525ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1102024-05-14T04:15:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1112024-05-14T04:15:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1122024-05-14T04:15:24.526ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1132024-05-14T04:15:24.526ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1142024-05-14T04:15:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1152024-05-14T04:15:24.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1162024-05-14T04:15:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1172024-05-14T04:15:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1182024-05-14T04:15:24.527ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1192024-05-14T04:15:24.527ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1202024-05-14T04:15:24.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1212024-05-14T04:15:24.527ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
1222024-05-14T04:15:24.527ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1232024-05-14T04:15:24.527ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1242024-05-14T04:15:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1252024-05-14T04:15:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1262024-05-14T04:15:24.528ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1272024-05-14T04:15:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1282024-05-14T04:15:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1292024-05-14T04:15:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1302024-05-14T04:15:24.528ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1312024-05-14T04:15:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1322024-05-14T04:15:24.529ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6749, tv_nsec: 191447407 } })
1332024-05-14T04:15:24.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1342024-05-14T04:15:24.529ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1352024-05-14T04:15:24.530ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1362024-05-14T04:15:24.530ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
1372024-05-14T04:15:24.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1382024-05-14T04:15:24.532ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
1392024-05-14T04:15:24.532ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 8.34714ms iteration = 5
1402024-05-14T04:15:24.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1412024-05-14T04:15:24.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1422024-05-14T04:15:24.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1432024-05-14T04:15:24.532ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
1442024-05-14T04:15:24.532ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
1452024-05-14T04:15:24.532ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
1462024-05-14T04:15:24.532ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1472024-05-14T04:15:24.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1482024-05-14T04:15:24.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1492024-05-14T04:15:24.534ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1502024-05-14T04:15:24.534ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
1512024-05-14T04:15:24.534ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
1522024-05-14T04:15:24.534ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
1532024-05-14T04:15:24.534ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1542024-05-14T04:15:24.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
1552024-05-14T04:15:24.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1562024-05-14T04:15:24.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1572024-05-14T04:15:24.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1582024-05-14T04:15:24.535ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1592024-05-14T04:15:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
1602024-05-14T04:15:24.535ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1612024-05-14T04:15:24.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1622024-05-14T04:15:24.538ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1632024-05-14T04:15:24.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1642024-05-14T04:15:24.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1652024-05-14T04:15:24.539ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1662024-05-14T04:15:24.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1672024-05-14T04:15:24.539ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1682024-05-14T04:15:24.540ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
1692024-05-14T04:15:24.540ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.62218ms iteration = 13
1702024-05-14T04:15:24.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1712024-05-14T04:15:24.540ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1722024-05-14T04:15:24.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1732024-05-14T04:15:24.541ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
1742024-05-14T04:15:24.541ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = f07c863b-7137-4303-9021-5a5c6e9ed458 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
1752024-05-14T04:15:24.541ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1762024-05-14T04:15:24.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1772024-05-14T04:15:24.541ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
1782024-05-14T04:15:24.541ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 4ac98832-1cdd-4449-9949-cf5fe247d837 dns_zone_name = oxide.test found_so_far = 0 version = 2
1792024-05-14T04:15:24.541ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1802024-05-14T04:15:24.541ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1812024-05-14T04:15:24.541ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1822024-05-14T04:15:24.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1832024-05-14T04:15:24.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1842024-05-14T04:15:24.542ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1852024-05-14T04:15:24.543ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1862024-05-14T04:15:24.544ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1872024-05-14T04:15:24.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1882024-05-14T04:15:24.546ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1892024-05-14T04:15:24.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1902024-05-14T04:15:24.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1912024-05-14T04:15:24.547ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1922024-05-14T04:15:24.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1932024-05-14T04:15:24.547ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1942024-05-14T04:15:24.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1952024-05-14T04:15:24.548ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1962024-05-14T04:15:24.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1972024-05-14T04:15:24.549ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = f07c863b-7137-4303-9021-5a5c6e9ed458 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
1982024-05-14T04:15:24.549ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
1992024-05-14T04:15:24.549ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-14 04:13:28.927466 UTC dns_group = internal generation = 1
2002024-05-14T04:15:24.549ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 31.994532ms iteration = 3
2012024-05-14T04:15:24.550ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 4ac98832-1cdd-4449-9949-cf5fe247d837 dns_zone_name = oxide.test found_so_far = 1 version = 2
2022024-05-14T04:15:24.550ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
2032024-05-14T04:15:24.550ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-14 04:13:29.424952 UTC dns_group = external generation = 2
2042024-05-14T04:15:24.550ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 32.619875ms iteration = 3
2052024-05-14T04:15:24.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, lookup_type: ById(5a4ac577-a6fd-4dea-aa82-fc399ad2b82d) } result = Ok(())
2062024-05-14T04:15:24.551ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2072024-05-14T04:15:24.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2082024-05-14T04:15:24.552ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 35.072815ms iteration = 5
2092024-05-14T04:15:24.589ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-14T04:13:24.616159Z, time_modified: 2024-05-14T04:13:24.616159Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: b4d2f397-0b2e-4db5-8e33-d295c802134a, db_silo: Silo { identity: SiloIdentity { id: b4d2f397-0b2e-4db5-8e33-d295c802134a, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-14T04:13:29.381647Z, time_modified: 2024-05-14T04:13:29.381647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "52301c060909182b4e1b505a925f9bb8cc086c123ca178be7538af43a1ecc18d" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: b4d2f397-0b2e-4db5-8e33-d295c802134a, db_silo: Silo { identity: SiloIdentity { id: b4d2f397-0b2e-4db5-8e33-d295c802134a, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-14T04:13:29.381647Z, time_modified: 2024-05-14T04:13:29.381647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "52301c060909182b4e1b505a925f9bb8cc086c123ca178be7538af43a1ecc18d" }] }) }
2102024-05-14T04:15:24.591ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 73.778343ms iteration = 3
2112024-05-14T04:15:24.614ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
2122024-05-14T04:15:24.614ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 96.497926ms iteration = 6
2132024-05-14T04:15:24.689ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
2142024-05-14T04:15:24.689ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2152024-05-14T04:15:24.690ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b925976b-5e5d-430f-9926-fa08dcb71834", "content-length": "26", "date": "Tue, 14 May 2024 04:15:24 GMT"} })
2162024-05-14T04:15:24.690ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2172024-05-14T04:15:24.692ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2182024-05-14T04:15:24.809ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
2192024-05-14T04:15:24.819ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
2202024-05-14T04:15:24.827ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
2212024-05-14T04:15:24.827ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2222024-05-14T04:15:24.828ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "73c5d9ba-36a7-41df-b0ee-9ee3cd13b5cd", "content-length": "26", "date": "Tue, 14 May 2024 04:15:24 GMT"} })
2232024-05-14T04:15:24.828ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2242024-05-14T04:15:24.832ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2252024-05-14T04:15:24.883ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 zone_address = fd00:1122:3344:101::2
2262024-05-14T04:15:24.883ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2272024-05-14T04:15:24.883ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a9327d64-365c-47c7-ad16-f17c1259e364", "content-length": "26", "date": "Tue, 14 May 2024 04:15:24 GMT"} })
2282024-05-14T04:15:24.883ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2292024-05-14T04:15:24.885ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
2302024-05-14T04:15:25.052ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
2312024-05-14T04:15:25.099ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 582.268ms iteration = 5
2322024-05-14T04:15:25.175ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
2332024-05-14T04:15:25.176ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "81e4655e-3ace-424c-867e-004233486df1", "date": "Tue, 14 May 2024 04:15:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2342024-05-14T04:15:25.176ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2352024-05-14T04:15:25.177ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "aa600ea7-a825-4700-a071-7b8610625307", "date": "Tue, 14 May 2024 04:15:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2362024-05-14T04:15:25.177ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2372024-05-14T04:15:25.179ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
2382024-05-14T04:15:25.179ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 661.651308ms iteration = 5
2392024-05-14T04:15:25.184ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2402024-05-14T04:15:25.184ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "91c8bd27-e318-4603-aa59-539166f16584", "content-length": "1", "date": "Tue, 14 May 2024 04:15:24 GMT"} })
2412024-05-14T04:15:25.185ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2422024-05-14T04:15:25.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2432024-05-14T04:15:25.187ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 670.094559ms iteration = 5
2442024-05-14T04:15:25.241ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
2452024-05-14T04:15:25.243ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5da3f8ef-7714-4ed0-abdd-66ebc21ee602", "date": "Tue, 14 May 2024 04:15:25 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2462024-05-14T04:15:25.243ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2472024-05-14T04:15:25.244ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c5548e80-2942-4843-818d-5b416dce6448", "date": "Tue, 14 May 2024 04:15:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2482024-05-14T04:15:25.244ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2492024-05-14T04:15:25.246ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "bb5d550e-9aa5-4dee-a408-f7071d450b1e", "date": "Tue, 14 May 2024 04:15:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2502024-05-14T04:15:25.246ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2512024-05-14T04:15:25.248ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2522024-05-14T04:15:25.248ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 731.244095ms iteration = 5
2532024-05-14T04:15:25.289ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2542024-05-14T04:15:25.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2552024-05-14T04:15:25.291ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: 384a5237-ec35-4842-bb57-a29c998a8bde, rack_id: b73a9888-3baa-4f16-9152-a3e1ba42c461, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(854ac53f-66c7-4bae-88ab-8ddb0746c0c4) } rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
2562024-05-14T04:15:25.291ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2572024-05-14T04:15:25.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2582024-05-14T04:15:25.299ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
2592024-05-14T04:15:25.299ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a312a293-3e6f-4829-bf22-c1ecd5f0a957", "content-length": "121", "date": "Tue, 14 May 2024 04:15:24 GMT"} })
2602024-05-14T04:15:25.299ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
2612024-05-14T04:15:25.299ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 854ac53f-66c7-4bae-88ab-8ddb0746c0c4
2622024-05-14T04:15:25.299ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
2632024-05-14T04:15:25.299ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
2642024-05-14T04:15:25.299ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
2652024-05-14T04:15:25.299ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
2662024-05-14T04:15:25.299ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
2672024-05-14T04:15:25.299ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
2682024-05-14T04:15:25.299ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
2692024-05-14T04:15:25.299ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2702024-05-14T04:15:25.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2712024-05-14T04:15:25.302ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
2722024-05-14T04:15:25.302ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0923bb3b-bac1-4047-b59d-3deaa3aac587", "content-length": "2", "date": "Tue, 14 May 2024 04:15:24 GMT"} })
2732024-05-14T04:15:25.302ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
2742024-05-14T04:15:25.303ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "11d4e293-6c5b-413d-a827-496767a2ee8a", "content-length": "51", "date": "Tue, 14 May 2024 04:15:24 GMT"} })
2752024-05-14T04:15:25.303ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
2762024-05-14T04:15:25.303ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
2772024-05-14T04:15:25.331ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 139 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 0a164363-ff72-4126-b161-8a63d3559952 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
2782024-05-14T04:15:25.350ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "42f5de68-9485-40ba-a58e-916ffbbbee80", "date": "Tue, 14 May 2024 04:15:25 GMT"} })
2792024-05-14T04:15:25.350ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch = Switch0
2802024-05-14T04:15:25.350ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
2812024-05-14T04:15:25.350ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ca9ddd70-b955-41ae-bbfe-d20e872eb5f9", "content-length": "467", "date": "Tue, 14 May 2024 04:15:25 GMT"} })
2822024-05-14T04:15:25.350ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2832024-05-14T04:15:25.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2842024-05-14T04:15:25.353ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2852024-05-14T04:15:25.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2862024-05-14T04:15:25.357ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2872024-05-14T04:15:25.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2882024-05-14T04:15:25.359ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2892024-05-14T04:15:25.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2902024-05-14T04:15:25.365ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 847.482233ms iteration = 5
2912024-05-14T04:15:34.517ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
2922024-05-14T04:15:34.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2932024-05-14T04:15:34.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2942024-05-14T04:15:34.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2952024-05-14T04:15:34.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2962024-05-14T04:15:34.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2972024-05-14T04:15:34.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2982024-05-14T04:15:34.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2992024-05-14T04:15:34.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3002024-05-14T04:15:34.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3012024-05-14T04:15:34.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
3022024-05-14T04:15:34.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3032024-05-14T04:15:34.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3042024-05-14T04:15:34.534ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
3052024-05-14T04:15:34.534ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.998897ms iteration = 14
3062024-05-14T04:15:35.331ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 148 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = b136febc-a8c0-4c8b-b938-2a4283490785 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
3072024-05-14T04:15:44.517ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
3082024-05-14T04:15:44.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3092024-05-14T04:15:44.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3102024-05-14T04:15:44.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3112024-05-14T04:15:44.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3122024-05-14T04:15:44.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3132024-05-14T04:15:44.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3142024-05-14T04:15:44.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3152024-05-14T04:15:44.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3162024-05-14T04:15:44.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3172024-05-14T04:15:44.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
3182024-05-14T04:15:44.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3192024-05-14T04:15:44.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3202024-05-14T04:15:44.532ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
3212024-05-14T04:15:44.532ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.650608ms iteration = 15
3222024-05-14T04:15:45.332ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 967 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 9f7a7aa1-8f39-48b4-9bdf-e6bc31650da6 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
3232024-05-14T04:15:54.517ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
3242024-05-14T04:15:54.517ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 7 reason = Timeout
3252024-05-14T04:15:54.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3262024-05-14T04:15:54.517ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 6 reason = Timeout
3272024-05-14T04:15:54.517ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3282024-05-14T04:15:54.517ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3292024-05-14T04:15:54.517ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
3302024-05-14T04:15:54.517ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
3312024-05-14T04:15:54.517ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
3322024-05-14T04:15:54.517ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
3332024-05-14T04:15:54.517ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
3342024-05-14T04:15:54.517ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3352024-05-14T04:15:54.517ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
3362024-05-14T04:15:54.517ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
3372024-05-14T04:15:54.517ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
3382024-05-14T04:15:54.517ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
3392024-05-14T04:15:54.517ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.24\u{b5}s iteration = 6
3402024-05-14T04:15:54.517ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3412024-05-14T04:15:54.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3422024-05-14T04:15:54.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3432024-05-14T04:15:54.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3442024-05-14T04:15:54.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3452024-05-14T04:15:54.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3462024-05-14T04:15:54.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3472024-05-14T04:15:54.519ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3482024-05-14T04:15:54.519ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6779, tv_nsec: 182049516 } })
3492024-05-14T04:15:54.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3502024-05-14T04:15:54.520ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
3512024-05-14T04:15:54.520ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3522024-05-14T04:15:54.520ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6779, tv_nsec: 182472364 } })
3532024-05-14T04:15:54.520ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3542024-05-14T04:15:54.521ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
3552024-05-14T04:15:54.521ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.547242ms iteration = 6
3562024-05-14T04:15:54.521ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
3572024-05-14T04:15:54.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3582024-05-14T04:15:54.521ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3592024-05-14T04:15:54.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3602024-05-14T04:15:54.523ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
3612024-05-14T04:15:54.523ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3622024-05-14T04:15:54.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3632024-05-14T04:15:54.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
3642024-05-14T04:15:54.523ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3652024-05-14T04:15:54.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3662024-05-14T04:15:54.524ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 6 reason = Timeout
3672024-05-14T04:15:54.524ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3682024-05-14T04:15:54.524ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3692024-05-14T04:15:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3702024-05-14T04:15:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
3712024-05-14T04:15:54.524ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3722024-05-14T04:15:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3732024-05-14T04:15:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3742024-05-14T04:15:54.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3752024-05-14T04:15:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3762024-05-14T04:15:54.525ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3772024-05-14T04:15:54.526ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6779, tv_nsec: 188158210 } })
3782024-05-14T04:15:54.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3792024-05-14T04:15:54.526ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3802024-05-14T04:15:54.526ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
3812024-05-14T04:15:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3822024-05-14T04:15:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3832024-05-14T04:15:54.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3842024-05-14T04:15:54.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3852024-05-14T04:15:54.529ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3862024-05-14T04:15:54.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3872024-05-14T04:15:54.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
3882024-05-14T04:15:54.530ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3892024-05-14T04:15:54.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3902024-05-14T04:15:54.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
3912024-05-14T04:15:54.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3922024-05-14T04:15:54.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3932024-05-14T04:15:54.542ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
3942024-05-14T04:15:54.542ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 18.612469ms iteration = 6
3952024-05-14T04:15:54.547ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
3962024-05-14T04:15:54.547ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.173545ms iteration = 16
3972024-05-14T04:15:54.552ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3982024-05-14T04:15:54.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3992024-05-14T04:15:54.554ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4002024-05-14T04:15:54.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4012024-05-14T04:15:54.556ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4022024-05-14T04:15:54.559ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, lookup_type: ById(5a4ac577-a6fd-4dea-aa82-fc399ad2b82d) } result = Ok(())
4032024-05-14T04:15:54.559ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4042024-05-14T04:15:54.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4052024-05-14T04:15:54.560ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 43.222681ms iteration = 6
4062024-05-14T04:15:54.611ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
4072024-05-14T04:15:54.611ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 93.467744ms iteration = 7
4082024-05-14T04:15:54.636ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 zone_address = fd00:1122:3344:101::2
4092024-05-14T04:15:54.636ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4102024-05-14T04:15:54.636ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d6947bc9-b43a-4ac0-b68c-cda3ae3fe755", "content-length": "26", "date": "Tue, 14 May 2024 04:15:54 GMT"} })
4112024-05-14T04:15:54.636ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4122024-05-14T04:15:54.638ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
4132024-05-14T04:15:54.721ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
4142024-05-14T04:15:54.721ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4152024-05-14T04:15:54.722ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cca9a1da-2709-4180-8444-0f1df27f2c4c", "content-length": "26", "date": "Tue, 14 May 2024 04:15:54 GMT"} })
4162024-05-14T04:15:54.722ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4172024-05-14T04:15:54.723ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4182024-05-14T04:15:54.725ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
4192024-05-14T04:15:54.725ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4202024-05-14T04:15:54.726ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ebe47fb0-4cad-4700-bd92-57a2065b573d", "content-length": "26", "date": "Tue, 14 May 2024 04:15:54 GMT"} })
4212024-05-14T04:15:54.726ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4222024-05-14T04:15:54.727ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4232024-05-14T04:15:54.793ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4242024-05-14T04:15:54.794ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a0479fd1-ca3b-4775-a64d-3175e5d28c3a", "content-length": "1", "date": "Tue, 14 May 2024 04:15:53 GMT"} })
4252024-05-14T04:15:54.795ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4262024-05-14T04:15:54.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4272024-05-14T04:15:54.797ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 279.551004ms iteration = 6
4282024-05-14T04:15:54.808ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 290.552349ms iteration = 6
4292024-05-14T04:15:54.834ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4302024-05-14T04:15:54.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4312024-05-14T04:15:54.836ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: 384a5237-ec35-4842-bb57-a29c998a8bde, rack_id: b73a9888-3baa-4f16-9152-a3e1ba42c461, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(854ac53f-66c7-4bae-88ab-8ddb0746c0c4) } rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
4322024-05-14T04:15:54.836ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4332024-05-14T04:15:54.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4342024-05-14T04:15:54.845ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4352024-05-14T04:15:54.846ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "befdef9a-ef50-45fa-b66d-9043d7208d6a", "content-length": "121", "date": "Tue, 14 May 2024 04:15:53 GMT"} })
4362024-05-14T04:15:54.846ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
4372024-05-14T04:15:54.846ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 854ac53f-66c7-4bae-88ab-8ddb0746c0c4
4382024-05-14T04:15:54.846ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
4392024-05-14T04:15:54.846ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
4402024-05-14T04:15:54.846ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
4412024-05-14T04:15:54.846ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
4422024-05-14T04:15:54.846ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
4432024-05-14T04:15:54.846ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
4442024-05-14T04:15:54.846ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
4452024-05-14T04:15:54.846ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4462024-05-14T04:15:54.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4472024-05-14T04:15:54.848ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4482024-05-14T04:15:54.848ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a31147f7-7917-4f9c-a8f7-7c2b8c68a07d", "content-length": "2", "date": "Tue, 14 May 2024 04:15:53 GMT"} })
4492024-05-14T04:15:54.848ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4502024-05-14T04:15:54.848ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f7541a07-01b5-43ff-adfa-52823bdab859", "content-length": "51", "date": "Tue, 14 May 2024 04:15:53 GMT"} })
4512024-05-14T04:15:54.848ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
4522024-05-14T04:15:54.848ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4532024-05-14T04:15:54.894ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2a943dc8-fb9f-49e7-affc-678354867287", "date": "Tue, 14 May 2024 04:15:54 GMT"} })
4542024-05-14T04:15:54.894ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch = Switch0
4552024-05-14T04:15:54.894ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4562024-05-14T04:15:54.898ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2274a8fb-6657-4406-8da7-aa41e1da93c2", "content-length": "467", "date": "Tue, 14 May 2024 04:15:54 GMT"} })
4572024-05-14T04:15:54.898ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4582024-05-14T04:15:54.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4592024-05-14T04:15:54.900ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4602024-05-14T04:15:54.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4612024-05-14T04:15:54.903ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4622024-05-14T04:15:54.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4632024-05-14T04:15:54.905ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4642024-05-14T04:15:54.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4652024-05-14T04:15:54.910ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 392.621747ms iteration = 6
4662024-05-14T04:15:55.331ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 163 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = cef62296-6767-435f-9d1b-a7ad949dc9b6 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
4672024-05-14T04:16:04.518ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
4682024-05-14T04:16:04.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4692024-05-14T04:16:04.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4702024-05-14T04:16:04.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4712024-05-14T04:16:04.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4722024-05-14T04:16:04.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4732024-05-14T04:16:04.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4742024-05-14T04:16:04.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4752024-05-14T04:16:04.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4762024-05-14T04:16:04.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4772024-05-14T04:16:04.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
4782024-05-14T04:16:04.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4792024-05-14T04:16:04.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4802024-05-14T04:16:04.532ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
4812024-05-14T04:16:04.532ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.704119ms iteration = 17
4822024-05-14T04:16:05.331ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 166 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 9a3fd40b-714d-4cb2-bb19-400d2ace2b55 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
4832024-05-14T04:16:14.517ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
4842024-05-14T04:16:14.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4852024-05-14T04:16:14.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4862024-05-14T04:16:14.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4872024-05-14T04:16:14.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4882024-05-14T04:16:14.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4892024-05-14T04:16:14.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4902024-05-14T04:16:14.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4912024-05-14T04:16:14.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4922024-05-14T04:16:14.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4932024-05-14T04:16:14.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
4942024-05-14T04:16:14.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4952024-05-14T04:16:14.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4962024-05-14T04:16:14.532ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
4972024-05-14T04:16:14.532ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.754841ms iteration = 18
4982024-05-14T04:16:15.331ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 169 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = a64c5f14-cafb-4b04-82fa-711bc7242f1c response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
4992024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
5002024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
5012024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5022024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
5032024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
5042024-05-14T04:16:24.517ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
5052024-05-14T04:16:24.517ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
5062024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.4\u{b5}s iteration = 7
5072024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5082024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
5092024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 7 reason = Timeout
5102024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5112024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 8 reason = Timeout
5122024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
5132024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5142024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5152024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
5162024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5172024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
5182024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
5192024-05-14T04:16:24.517ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
5202024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 9.89\u{b5}s iteration = 4
5212024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
5222024-05-14T04:16:24.517ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
5232024-05-14T04:16:24.518ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce
5242024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
5252024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.621\u{b5}s iteration = 5
5262024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5272024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
5282024-05-14T04:16:24.518ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
5292024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
5302024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5312024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
5322024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
5332024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
5342024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5352024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5362024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5372024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5382024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5392024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5402024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5412024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5422024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5432024-05-14T04:16:24.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5442024-05-14T04:16:24.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5452024-05-14T04:16:24.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5462024-05-14T04:16:24.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5472024-05-14T04:16:24.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5482024-05-14T04:16:24.519ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("08a9c078-20b8-4325-90f1-e360e31ca4ab.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("3f6ad63d-f2d7-4b36-a169-d3099a239267.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("3f6ad63d-f2d7-4b36-a169-d3099a239267.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 6809, tv_nsec: 181597827 } })
5492024-05-14T04:16:24.519ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0dd5d513-ee28-4da3-83f2-7a3d7e3b267d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6aa90bd6-c3db-4dfa-88c0-97308b03c15f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9a8a4960-0835-4c5e-b472-b2521417e003.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9a8a4960-0835-4c5e-b472-b2521417e003.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6809, tv_nsec: 181643227 } })
5502024-05-14T04:16:24.520ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5512024-05-14T04:16:24.520ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5522024-05-14T04:16:24.521ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5532024-05-14T04:16:24.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5542024-05-14T04:16:24.521ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
5552024-05-14T04:16:24.521ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.47468ms iteration = 7
5562024-05-14T04:16:24.521ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6809, tv_nsec: 183661239 } })
5572024-05-14T04:16:24.521ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5582024-05-14T04:16:24.521ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
5592024-05-14T04:16:24.521ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.990711ms iteration = 4
5602024-05-14T04:16:24.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5612024-05-14T04:16:24.522ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5622024-05-14T04:16:24.522ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5632024-05-14T04:16:24.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5642024-05-14T04:16:24.523ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
5652024-05-14T04:16:24.523ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 5.143704ms iteration = 4
5662024-05-14T04:16:24.523ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
5672024-05-14T04:16:24.523ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5682024-05-14T04:16:24.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5692024-05-14T04:16:24.524ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6809, tv_nsec: 186236471 } })
5702024-05-14T04:16:24.524ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
5712024-05-14T04:16:24.524ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5722024-05-14T04:16:24.524ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5732024-05-14T04:16:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5742024-05-14T04:16:24.525ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5752024-05-14T04:16:24.525ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5762024-05-14T04:16:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5772024-05-14T04:16:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5782024-05-14T04:16:24.526ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
5792024-05-14T04:16:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5802024-05-14T04:16:24.526ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5812024-05-14T04:16:24.526ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5822024-05-14T04:16:24.526ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6809, tv_nsec: 188729642 } })
5832024-05-14T04:16:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5842024-05-14T04:16:24.527ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
5852024-05-14T04:16:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5862024-05-14T04:16:24.527ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5872024-05-14T04:16:24.527ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
5882024-05-14T04:16:24.527ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5892024-05-14T04:16:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5902024-05-14T04:16:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5912024-05-14T04:16:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5922024-05-14T04:16:24.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5932024-05-14T04:16:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5942024-05-14T04:16:24.529ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5952024-05-14T04:16:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5962024-05-14T04:16:24.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5972024-05-14T04:16:24.529ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5982024-05-14T04:16:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5992024-05-14T04:16:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6002024-05-14T04:16:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6012024-05-14T04:16:24.529ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6022024-05-14T04:16:24.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6032024-05-14T04:16:24.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6042024-05-14T04:16:24.530ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
6052024-05-14T04:16:24.530ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
6062024-05-14T04:16:24.530ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
6072024-05-14T04:16:24.530ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6082024-05-14T04:16:24.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6092024-05-14T04:16:24.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6102024-05-14T04:16:24.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
6112024-05-14T04:16:24.530ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6122024-05-14T04:16:24.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6132024-05-14T04:16:24.531ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6142024-05-14T04:16:24.532ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6152024-05-14T04:16:24.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6162024-05-14T04:16:24.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6172024-05-14T04:16:24.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6182024-05-14T04:16:24.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6192024-05-14T04:16:24.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
6202024-05-14T04:16:24.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6212024-05-14T04:16:24.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6222024-05-14T04:16:24.534ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6232024-05-14T04:16:24.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6242024-05-14T04:16:24.534ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6252024-05-14T04:16:24.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6262024-05-14T04:16:24.535ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
6272024-05-14T04:16:24.535ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 11.697848ms iteration = 7
6282024-05-14T04:16:24.535ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6292024-05-14T04:16:24.535ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6302024-05-14T04:16:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6312024-05-14T04:16:24.535ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
6322024-05-14T04:16:24.536ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 18.15949ms iteration = 8
6332024-05-14T04:16:24.536ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
6342024-05-14T04:16:24.536ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = f07c863b-7137-4303-9021-5a5c6e9ed458 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
6352024-05-14T04:16:24.536ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6362024-05-14T04:16:24.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6372024-05-14T04:16:24.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6382024-05-14T04:16:24.539ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
6392024-05-14T04:16:24.539ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.762904ms iteration = 19
6402024-05-14T04:16:24.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6412024-05-14T04:16:24.542ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6422024-05-14T04:16:24.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6432024-05-14T04:16:24.546ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6442024-05-14T04:16:24.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6452024-05-14T04:16:24.550ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6462024-05-14T04:16:24.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6472024-05-14T04:16:24.550ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = f07c863b-7137-4303-9021-5a5c6e9ed458 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
6482024-05-14T04:16:24.550ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
6492024-05-14T04:16:24.550ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-14 04:13:28.927466 UTC dns_group = internal generation = 1
6502024-05-14T04:16:24.550ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 33.006413ms iteration = 4
6512024-05-14T04:16:24.552ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6522024-05-14T04:16:24.555ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, lookup_type: ById(5a4ac577-a6fd-4dea-aa82-fc399ad2b82d) } result = Ok(())
6532024-05-14T04:16:24.555ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6542024-05-14T04:16:24.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6552024-05-14T04:16:24.556ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 39.214519ms iteration = 7
6562024-05-14T04:16:24.569ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
6572024-05-14T04:16:24.569ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
6582024-05-14T04:16:24.569ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
6592024-05-14T04:16:24.569ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6602024-05-14T04:16:24.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6612024-05-14T04:16:24.571ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6622024-05-14T04:16:24.573ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6632024-05-14T04:16:24.573ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6642024-05-14T04:16:24.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6652024-05-14T04:16:24.575ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
6662024-05-14T04:16:24.575ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 4ac98832-1cdd-4449-9949-cf5fe247d837 dns_zone_name = oxide.test found_so_far = 0 version = 2
6672024-05-14T04:16:24.575ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6682024-05-14T04:16:24.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6692024-05-14T04:16:24.576ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6702024-05-14T04:16:24.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6712024-05-14T04:16:24.579ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6722024-05-14T04:16:24.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6732024-05-14T04:16:24.581ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 4ac98832-1cdd-4449-9949-cf5fe247d837 dns_zone_name = oxide.test found_so_far = 1 version = 2
6742024-05-14T04:16:24.581ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
6752024-05-14T04:16:24.581ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-14 04:13:29.424952 UTC dns_group = external generation = 2
6762024-05-14T04:16:24.581ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 63.30107ms iteration = 4
6772024-05-14T04:16:24.584ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-14T04:13:24.616159Z, time_modified: 2024-05-14T04:13:24.616159Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: b4d2f397-0b2e-4db5-8e33-d295c802134a, db_silo: Silo { identity: SiloIdentity { id: b4d2f397-0b2e-4db5-8e33-d295c802134a, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-14T04:13:29.381647Z, time_modified: 2024-05-14T04:13:29.381647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "52301c060909182b4e1b505a925f9bb8cc086c123ca178be7538af43a1ecc18d" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: b4d2f397-0b2e-4db5-8e33-d295c802134a, db_silo: Silo { identity: SiloIdentity { id: b4d2f397-0b2e-4db5-8e33-d295c802134a, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-14T04:13:29.381647Z, time_modified: 2024-05-14T04:13:29.381647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "52301c060909182b4e1b505a925f9bb8cc086c123ca178be7538af43a1ecc18d" }] }) }
6782024-05-14T04:16:24.584ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 66.254119ms iteration = 4
6792024-05-14T04:16:24.820ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
6802024-05-14T04:16:24.820ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6812024-05-14T04:16:24.822ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "962173df-f371-468b-90b7-e2b2965a9702", "content-length": "26", "date": "Tue, 14 May 2024 04:16:24 GMT"} })
6822024-05-14T04:16:24.822ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6832024-05-14T04:16:24.823ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6842024-05-14T04:16:24.832ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 zone_address = fd00:1122:3344:101::2
6852024-05-14T04:16:24.832ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6862024-05-14T04:16:24.832ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8c63bb03-a9b3-4139-bbec-e77d85c1a628", "content-length": "26", "date": "Tue, 14 May 2024 04:16:24 GMT"} })
6872024-05-14T04:16:24.832ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6882024-05-14T04:16:24.836ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
6892024-05-14T04:16:24.902ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
6902024-05-14T04:16:24.902ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6912024-05-14T04:16:24.902ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d4a53765-0946-4860-a3f2-5be292492cfe", "content-length": "26", "date": "Tue, 14 May 2024 04:16:24 GMT"} })
6922024-05-14T04:16:24.902ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6932024-05-14T04:16:24.904ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6942024-05-14T04:16:24.907ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
6952024-05-14T04:16:24.907ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
6962024-05-14T04:16:24.969ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
6972024-05-14T04:16:24.970ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "35135b2e-330b-4523-8757-75eb4984da5b", "content-length": "1", "date": "Tue, 14 May 2024 04:16:24 GMT"} })
6982024-05-14T04:16:24.971ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6992024-05-14T04:16:24.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7002024-05-14T04:16:24.973ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 455.918507ms iteration = 7
7012024-05-14T04:16:25.046ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 528.345752ms iteration = 7
7022024-05-14T04:16:25.144ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
7032024-05-14T04:16:25.157ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
7042024-05-14T04:16:25.158ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "60a3a6dc-9c8d-4300-b425-fecd7775b702", "date": "Tue, 14 May 2024 04:16:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7052024-05-14T04:16:25.158ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7062024-05-14T04:16:25.160ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b4db8248-9eff-4bf4-9326-84c0362b3a7f", "date": "Tue, 14 May 2024 04:16:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7072024-05-14T04:16:25.160ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7082024-05-14T04:16:25.162ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7092024-05-14T04:16:25.162ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 644.413837ms iteration = 6
7102024-05-14T04:16:25.182ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7112024-05-14T04:16:25.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7122024-05-14T04:16:25.184ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: 384a5237-ec35-4842-bb57-a29c998a8bde, rack_id: b73a9888-3baa-4f16-9152-a3e1ba42c461, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(854ac53f-66c7-4bae-88ab-8ddb0746c0c4) } rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
7132024-05-14T04:16:25.184ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7142024-05-14T04:16:25.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7152024-05-14T04:16:25.193ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
7162024-05-14T04:16:25.193ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c327a381-894b-42b1-83ed-cd5ceff63de2", "content-length": "121", "date": "Tue, 14 May 2024 04:16:25 GMT"} })
7172024-05-14T04:16:25.193ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
7182024-05-14T04:16:25.193ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 854ac53f-66c7-4bae-88ab-8ddb0746c0c4
7192024-05-14T04:16:25.193ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
7202024-05-14T04:16:25.193ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
7212024-05-14T04:16:25.193ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
7222024-05-14T04:16:25.193ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
7232024-05-14T04:16:25.193ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
7242024-05-14T04:16:25.193ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
7252024-05-14T04:16:25.193ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
7262024-05-14T04:16:25.193ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7272024-05-14T04:16:25.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7282024-05-14T04:16:25.194ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
7292024-05-14T04:16:25.195ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fd68ba75-b8e1-4520-8c6b-d0aed6a61997", "content-length": "2", "date": "Tue, 14 May 2024 04:16:25 GMT"} })
7302024-05-14T04:16:25.195ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
7312024-05-14T04:16:25.195ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ac5fd067-a2e5-4234-b561-f5c0ab23bef9", "content-length": "51", "date": "Tue, 14 May 2024 04:16:25 GMT"} })
7322024-05-14T04:16:25.195ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
7332024-05-14T04:16:25.195ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
7342024-05-14T04:16:25.198ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
7352024-05-14T04:16:25.199ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f41af3bc-5f32-44cc-9bbb-5c6c268c1d3d", "date": "Tue, 14 May 2024 04:16:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7362024-05-14T04:16:25.199ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7372024-05-14T04:16:25.200ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "dbf30fcc-4ac5-42af-8161-9a3cad119054", "date": "Tue, 14 May 2024 04:16:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7382024-05-14T04:16:25.200ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7392024-05-14T04:16:25.201ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "45ef5280-e078-4ac0-bdbd-ef59665ebd52", "date": "Tue, 14 May 2024 04:16:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7402024-05-14T04:16:25.201ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7412024-05-14T04:16:25.202ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7422024-05-14T04:16:25.202ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 685.088415ms iteration = 6
7432024-05-14T04:16:25.244ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6f31431e-e08e-4b71-b516-b9435bd14691", "date": "Tue, 14 May 2024 04:16:25 GMT"} })
7442024-05-14T04:16:25.244ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch = Switch0
7452024-05-14T04:16:25.244ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
7462024-05-14T04:16:25.244ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "16a087d2-f8e0-447c-807c-f174f8a27d4b", "content-length": "467", "date": "Tue, 14 May 2024 04:16:25 GMT"} })
7472024-05-14T04:16:25.244ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7482024-05-14T04:16:25.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7492024-05-14T04:16:25.247ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7502024-05-14T04:16:25.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7512024-05-14T04:16:25.249ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7522024-05-14T04:16:25.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7532024-05-14T04:16:25.251ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7542024-05-14T04:16:25.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7552024-05-14T04:16:25.257ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 739.844331ms iteration = 7
7562024-05-14T04:16:25.331ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 155 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 763cc7c0-b2ea-4bfb-81ee-4f4c7bcfa257 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
7572024-05-14T04:16:34.518ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
7582024-05-14T04:16:34.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7592024-05-14T04:16:34.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7602024-05-14T04:16:34.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7612024-05-14T04:16:34.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7622024-05-14T04:16:34.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7632024-05-14T04:16:34.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7642024-05-14T04:16:34.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7652024-05-14T04:16:34.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7662024-05-14T04:16:34.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7672024-05-14T04:16:34.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
7682024-05-14T04:16:34.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7692024-05-14T04:16:34.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7702024-05-14T04:16:34.532ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
7712024-05-14T04:16:34.532ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.203659ms iteration = 20
7722024-05-14T04:16:35.331ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 250 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 2bc9c464-75d4-4e07-9a5d-e256e7cbf5fe response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
7732024-05-14T04:16:44.518ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
7742024-05-14T04:16:44.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7752024-05-14T04:16:44.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7762024-05-14T04:16:44.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7772024-05-14T04:16:44.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7782024-05-14T04:16:44.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7792024-05-14T04:16:44.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7802024-05-14T04:16:44.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7812024-05-14T04:16:44.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7822024-05-14T04:16:44.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7832024-05-14T04:16:44.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
7842024-05-14T04:16:44.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7852024-05-14T04:16:44.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7862024-05-14T04:16:44.533ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
7872024-05-14T04:16:44.533ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.67645ms iteration = 21
7882024-05-14T04:16:45.332ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 215 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 0169930b-ff4e-4c48-bb90-ba0eda8c961c response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
7892024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
7902024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
7912024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7922024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7932024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
7942024-05-14T04:16:54.518ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
7952024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 9 reason = Timeout
7962024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
7972024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7982024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
7992024-05-14T04:16:54.518ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
8002024-05-14T04:16:54.518ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
8012024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
8022024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.29\u{b5}s iteration = 8
8032024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
8042024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 8 reason = Timeout
8052024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8062024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8072024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8082024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8092024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8102024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8112024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8122024-05-14T04:16:54.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
8132024-05-14T04:16:54.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8142024-05-14T04:16:54.520ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8152024-05-14T04:16:54.520ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8162024-05-14T04:16:54.521ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8172024-05-14T04:16:54.521ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6839, tv_nsec: 182735800 } })
8182024-05-14T04:16:54.521ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
8192024-05-14T04:16:54.521ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
8202024-05-14T04:16:54.521ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.398249ms iteration = 8
8212024-05-14T04:16:54.522ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6839, tv_nsec: 183951565 } })
8222024-05-14T04:16:54.522ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
8232024-05-14T04:16:54.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
8242024-05-14T04:16:54.523ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8252024-05-14T04:16:54.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8262024-05-14T04:16:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8272024-05-14T04:16:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8282024-05-14T04:16:54.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8292024-05-14T04:16:54.524ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8302024-05-14T04:16:54.524ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
8312024-05-14T04:16:54.524ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8322024-05-14T04:16:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8332024-05-14T04:16:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8342024-05-14T04:16:54.524ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6839, tv_nsec: 186435965 } })
8352024-05-14T04:16:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8362024-05-14T04:16:54.524ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8372024-05-14T04:16:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8382024-05-14T04:16:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8392024-05-14T04:16:54.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
8402024-05-14T04:16:54.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8412024-05-14T04:16:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8422024-05-14T04:16:54.526ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8432024-05-14T04:16:54.526ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
8442024-05-14T04:16:54.526ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8452024-05-14T04:16:54.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8462024-05-14T04:16:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8472024-05-14T04:16:54.526ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8482024-05-14T04:16:54.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8492024-05-14T04:16:54.527ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8502024-05-14T04:16:54.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
8512024-05-14T04:16:54.529ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8522024-05-14T04:16:54.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8532024-05-14T04:16:54.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8542024-05-14T04:16:54.530ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8552024-05-14T04:16:54.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8562024-05-14T04:16:54.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
8572024-05-14T04:16:54.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8582024-05-14T04:16:54.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8592024-05-14T04:16:54.533ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
8602024-05-14T04:16:54.533ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 9.280999ms iteration = 8
8612024-05-14T04:16:54.537ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
8622024-05-14T04:16:54.537ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.286623ms iteration = 22
8632024-05-14T04:16:54.540ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8642024-05-14T04:16:54.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8652024-05-14T04:16:54.542ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8662024-05-14T04:16:54.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8672024-05-14T04:16:54.543ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8682024-05-14T04:16:54.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, lookup_type: ById(5a4ac577-a6fd-4dea-aa82-fc399ad2b82d) } result = Ok(())
8692024-05-14T04:16:54.546ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8702024-05-14T04:16:54.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8712024-05-14T04:16:54.547ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 29.169154ms iteration = 8
8722024-05-14T04:16:54.558ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
8732024-05-14T04:16:54.558ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 40.24166ms iteration = 9
8742024-05-14T04:16:54.592ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
8752024-05-14T04:16:54.592ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8762024-05-14T04:16:54.593ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "af4ffc54-f47f-4c23-aaac-8a070d25be6a", "content-length": "26", "date": "Tue, 14 May 2024 04:16:54 GMT"} })
8772024-05-14T04:16:54.593ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8782024-05-14T04:16:54.594ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8792024-05-14T04:16:54.596ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
8802024-05-14T04:16:54.596ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8812024-05-14T04:16:54.596ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "33db5eb3-1711-46f0-ad40-e863d3522252", "content-length": "26", "date": "Tue, 14 May 2024 04:16:54 GMT"} })
8822024-05-14T04:16:54.596ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8832024-05-14T04:16:54.597ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8842024-05-14T04:16:54.609ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 zone_address = fd00:1122:3344:101::2
8852024-05-14T04:16:54.609ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8862024-05-14T04:16:54.609ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b8d102b2-750d-472d-a391-f451f8160cc2", "content-length": "26", "date": "Tue, 14 May 2024 04:16:54 GMT"} })
8872024-05-14T04:16:54.609ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8882024-05-14T04:16:54.610ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
8892024-05-14T04:16:54.672ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 153.668481ms iteration = 8
8902024-05-14T04:16:54.678ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
8912024-05-14T04:16:54.679ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3ad0ffa7-954f-43af-995a-42ace15add32", "content-length": "1", "date": "Tue, 14 May 2024 04:16:54 GMT"} })
8922024-05-14T04:16:54.681ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8932024-05-14T04:16:54.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8942024-05-14T04:16:54.683ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 164.643934ms iteration = 8
8952024-05-14T04:16:54.738ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8962024-05-14T04:16:54.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8972024-05-14T04:16:54.739ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: 384a5237-ec35-4842-bb57-a29c998a8bde, rack_id: b73a9888-3baa-4f16-9152-a3e1ba42c461, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(854ac53f-66c7-4bae-88ab-8ddb0746c0c4) } rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
8982024-05-14T04:16:54.739ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8992024-05-14T04:16:54.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9002024-05-14T04:16:54.747ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
9012024-05-14T04:16:54.747ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5064afc4-5ecf-420a-8714-5800df86441f", "content-length": "121", "date": "Tue, 14 May 2024 04:16:54 GMT"} })
9022024-05-14T04:16:54.747ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
9032024-05-14T04:16:54.747ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 854ac53f-66c7-4bae-88ab-8ddb0746c0c4
9042024-05-14T04:16:54.747ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
9052024-05-14T04:16:54.747ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
9062024-05-14T04:16:54.747ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
9072024-05-14T04:16:54.747ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
9082024-05-14T04:16:54.747ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
9092024-05-14T04:16:54.747ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
9102024-05-14T04:16:54.747ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
9112024-05-14T04:16:54.747ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9122024-05-14T04:16:54.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9132024-05-14T04:16:54.749ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
9142024-05-14T04:16:54.749ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ebcbd758-d9b0-45b1-a3b7-cbd247cc7853", "content-length": "2", "date": "Tue, 14 May 2024 04:16:54 GMT"} })
9152024-05-14T04:16:54.749ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
9162024-05-14T04:16:54.749ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6511889a-c681-4512-847f-1ae8a8b2ded0", "content-length": "51", "date": "Tue, 14 May 2024 04:16:54 GMT"} })
9172024-05-14T04:16:54.749ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
9182024-05-14T04:16:54.749ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
9192024-05-14T04:16:54.796ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1f3645b8-2a78-404d-96b3-08b1c68908c7", "date": "Tue, 14 May 2024 04:16:54 GMT"} })
9202024-05-14T04:16:54.796ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch = Switch0
9212024-05-14T04:16:54.796ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
9222024-05-14T04:16:54.797ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7fa34af2-a550-458f-8fe7-ab8c854d3a05", "content-length": "467", "date": "Tue, 14 May 2024 04:16:54 GMT"} })
9232024-05-14T04:16:54.797ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9242024-05-14T04:16:54.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9252024-05-14T04:16:54.799ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9262024-05-14T04:16:54.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9272024-05-14T04:16:54.802ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9282024-05-14T04:16:54.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9292024-05-14T04:16:54.803ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9302024-05-14T04:16:54.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9312024-05-14T04:16:54.807ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 289.015627ms iteration = 8
9322024-05-14T04:16:55.301ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
9332024-05-14T04:16:55.301ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
9342024-05-14T04:16:55.330ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
9352024-05-14T04:16:55.330ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:61238
9362024-05-14T04:16:55.331ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:61238 req_id = 4da15ce5-bea6-420e-abf0-9699a31b833a roles = RoleSet { roles: {} } uri = /metrics/producers
9372024-05-14T04:16:55.331ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:61238 req_id = 4da15ce5-bea6-420e-abf0-9699a31b833a resource = Database result = Ok(()) uri = /metrics/producers
9382024-05-14T04:16:55.331ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 171 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 659c98ad-22d7-437b-906b-34c7eeb65d4e response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
9392024-05-14T04:16:55.362ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 90873ca0-52f3-4dd5-a08c-9897fa8723c5
9402024-05-14T04:16:55.362ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:61238 req_id = 4da15ce5-bea6-420e-abf0-9699a31b833a roles = RoleSet { roles: {} } uri = /metrics/producers
9412024-05-14T04:16:55.362ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:61238 req_id = 4da15ce5-bea6-420e-abf0-9699a31b833a resource = Database result = Ok(()) uri = /metrics/producers
9422024-05-14T04:16:55.390ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 90873ca0-52f3-4dd5-a08c-9897fa8723c5 uri = http://[fd00:1122:3344:101::d]:12223/producers
9432024-05-14T04:16:55.390ZDEBGnexus: client response oximeter-collector = 90873ca0-52f3-4dd5-a08c-9897fa8723c5 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e588f94c-5a01-469f-917e-f96abb5bbcb4", "date": "Tue, 14 May 2024 04:16:55 GMT"} })
9442024-05-14T04:16:55.390ZINFOnexus: assigned collector to new producer collector_id = 90873ca0-52f3-4dd5-a08c-9897fa8723c5 file = nexus/src/app/oximeter.rs:125 producer_id = ef1c9f55-f835-41e6-8d3a-81e26da276c7
9452024-05-14T04:16:55.392ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 61074 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:61238 req_id = 4da15ce5-bea6-420e-abf0-9699a31b833a response_code = 201 uri = /metrics/producers
9462024-05-14T04:16:55.392ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::c)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "4da15ce5-bea6-420e-abf0-9699a31b833a", "content-length": "41", "date": "Tue, 14 May 2024 04:16:55 GMT"} })
9472024-05-14T04:16:55.393ZDEBGnexus (producer-registration-task): registered with nexus successfully
9482024-05-14T04:16:55.393ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
9492024-05-14T04:17:04.518ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
9502024-05-14T04:17:04.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9512024-05-14T04:17:04.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9522024-05-14T04:17:04.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9532024-05-14T04:17:04.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9542024-05-14T04:17:04.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9552024-05-14T04:17:04.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9562024-05-14T04:17:04.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9572024-05-14T04:17:04.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9582024-05-14T04:17:04.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9592024-05-14T04:17:04.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
9602024-05-14T04:17:04.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9612024-05-14T04:17:04.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9622024-05-14T04:17:04.532ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
9632024-05-14T04:17:04.532ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.018756ms iteration = 23
9642024-05-14T04:17:05.392ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 173 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = f88a3fa7-af77-4c16-85b3-983580093e12 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
9652024-05-14T04:17:14.518ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
9662024-05-14T04:17:14.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9672024-05-14T04:17:14.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9682024-05-14T04:17:14.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9692024-05-14T04:17:14.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9702024-05-14T04:17:14.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9712024-05-14T04:17:14.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9722024-05-14T04:17:14.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9732024-05-14T04:17:14.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9742024-05-14T04:17:14.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9752024-05-14T04:17:14.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
9762024-05-14T04:17:14.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9772024-05-14T04:17:14.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9782024-05-14T04:17:14.532ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
9792024-05-14T04:17:14.532ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.360992ms iteration = 24
9802024-05-14T04:17:15.391ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 164 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 5eab518a-348f-41e0-ba7b-9b4e5e773028 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
9812024-05-14T04:17:24.518ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
9822024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
9832024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9842024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
9852024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
9862024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9872024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
9882024-05-14T04:17:24.519ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce
9892024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
9902024-05-14T04:17:24.519ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
9912024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 16.47\u{b5}s iteration = 6
9922024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 8.63\u{b5}s iteration = 5
9932024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
9942024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
9952024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
9962024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9972024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9982024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
9992024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
10002024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10012024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 10 reason = Timeout
10022024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10032024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
10042024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
10052024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10062024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
10072024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
10082024-05-14T04:17:24.519ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
10092024-05-14T04:17:24.519ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
10102024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10112024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
10122024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.07\u{b5}s iteration = 9
10132024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10142024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
10152024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
10162024-05-14T04:17:24.519ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
10172024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10182024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10192024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10202024-05-14T04:17:24.519ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0dd5d513-ee28-4da3-83f2-7a3d7e3b267d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6aa90bd6-c3db-4dfa-88c0-97308b03c15f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9a8a4960-0835-4c5e-b472-b2521417e003.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9a8a4960-0835-4c5e-b472-b2521417e003.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6869, tv_nsec: 180942273 } })
10212024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
10222024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10232024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10242024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10252024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10262024-05-14T04:17:24.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10272024-05-14T04:17:24.520ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("08a9c078-20b8-4325-90f1-e360e31ca4ab.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("3f6ad63d-f2d7-4b36-a169-d3099a239267.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("3f6ad63d-f2d7-4b36-a169-d3099a239267.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 6869, tv_nsec: 181935913 } })
10282024-05-14T04:17:24.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10292024-05-14T04:17:24.520ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10302024-05-14T04:17:24.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10312024-05-14T04:17:24.521ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10322024-05-14T04:17:24.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10332024-05-14T04:17:24.521ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
10342024-05-14T04:17:24.521ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.730815ms iteration = 5
10352024-05-14T04:17:24.521ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
10362024-05-14T04:17:24.521ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.638104ms iteration = 9
10372024-05-14T04:17:24.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10382024-05-14T04:17:24.521ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10392024-05-14T04:17:24.522ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10402024-05-14T04:17:24.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
10412024-05-14T04:17:24.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.797278ms iteration = 5
10422024-05-14T04:17:24.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10432024-05-14T04:17:24.523ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10442024-05-14T04:17:24.523ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10452024-05-14T04:17:24.524ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6869, tv_nsec: 185354243 } })
10462024-05-14T04:17:24.524ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6869, tv_nsec: 185355873 } })
10472024-05-14T04:17:24.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
10482024-05-14T04:17:24.524ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10492024-05-14T04:17:24.525ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
10502024-05-14T04:17:24.525ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10512024-05-14T04:17:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10522024-05-14T04:17:24.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
10532024-05-14T04:17:24.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
10542024-05-14T04:17:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10552024-05-14T04:17:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
10562024-05-14T04:17:24.525ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10572024-05-14T04:17:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10582024-05-14T04:17:24.526ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
10592024-05-14T04:17:24.526ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10602024-05-14T04:17:24.526ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10612024-05-14T04:17:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10622024-05-14T04:17:24.526ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6869, tv_nsec: 188095409 } })
10632024-05-14T04:17:24.527ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10642024-05-14T04:17:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10652024-05-14T04:17:24.527ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10662024-05-14T04:17:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10672024-05-14T04:17:24.527ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
10682024-05-14T04:17:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10692024-05-14T04:17:24.528ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10702024-05-14T04:17:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10712024-05-14T04:17:24.529ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
10722024-05-14T04:17:24.529ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10732024-05-14T04:17:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10742024-05-14T04:17:24.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10752024-05-14T04:17:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10762024-05-14T04:17:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
10772024-05-14T04:17:24.529ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10782024-05-14T04:17:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10792024-05-14T04:17:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10802024-05-14T04:17:24.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10812024-05-14T04:17:24.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10822024-05-14T04:17:24.530ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10832024-05-14T04:17:24.530ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10842024-05-14T04:17:24.531ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
10852024-05-14T04:17:24.531ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
10862024-05-14T04:17:24.531ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
10872024-05-14T04:17:24.531ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10882024-05-14T04:17:24.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10892024-05-14T04:17:24.531ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
10902024-05-14T04:17:24.531ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
10912024-05-14T04:17:24.531ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
10922024-05-14T04:17:24.531ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10932024-05-14T04:17:24.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10942024-05-14T04:17:24.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10952024-05-14T04:17:24.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10962024-05-14T04:17:24.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10972024-05-14T04:17:24.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10982024-05-14T04:17:24.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10992024-05-14T04:17:24.532ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11002024-05-14T04:17:24.533ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11012024-05-14T04:17:24.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11022024-05-14T04:17:24.533ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
11032024-05-14T04:17:24.533ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 8.488683ms iteration = 9
11042024-05-14T04:17:24.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
11052024-05-14T04:17:24.534ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11062024-05-14T04:17:24.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11072024-05-14T04:17:24.535ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
11082024-05-14T04:17:24.535ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 16.046647ms iteration = 10
11092024-05-14T04:17:24.536ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11102024-05-14T04:17:24.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11112024-05-14T04:17:24.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
11122024-05-14T04:17:24.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11132024-05-14T04:17:24.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11142024-05-14T04:17:24.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11152024-05-14T04:17:24.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11162024-05-14T04:17:24.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11172024-05-14T04:17:24.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11182024-05-14T04:17:24.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11192024-05-14T04:17:24.538ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11202024-05-14T04:17:24.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11212024-05-14T04:17:24.539ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
11222024-05-14T04:17:24.539ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = f07c863b-7137-4303-9021-5a5c6e9ed458 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
11232024-05-14T04:17:24.539ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11242024-05-14T04:17:24.539ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
11252024-05-14T04:17:24.539ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 4ac98832-1cdd-4449-9949-cf5fe247d837 dns_zone_name = oxide.test found_so_far = 0 version = 2
11262024-05-14T04:17:24.539ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11272024-05-14T04:17:24.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11282024-05-14T04:17:24.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11292024-05-14T04:17:24.540ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
11302024-05-14T04:17:24.540ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11312024-05-14T04:17:24.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11322024-05-14T04:17:24.540ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11332024-05-14T04:17:24.541ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11342024-05-14T04:17:24.542ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
11352024-05-14T04:17:24.542ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.781444ms iteration = 25
11362024-05-14T04:17:24.544ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11372024-05-14T04:17:24.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11382024-05-14T04:17:24.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11392024-05-14T04:17:24.545ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11402024-05-14T04:17:24.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11412024-05-14T04:17:24.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11422024-05-14T04:17:24.545ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11432024-05-14T04:17:24.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11442024-05-14T04:17:24.546ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11452024-05-14T04:17:24.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11462024-05-14T04:17:24.546ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 4ac98832-1cdd-4449-9949-cf5fe247d837 dns_zone_name = oxide.test found_so_far = 1 version = 2
11472024-05-14T04:17:24.546ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
11482024-05-14T04:17:24.546ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-14 04:13:29.424952 UTC dns_group = external generation = 2
11492024-05-14T04:17:24.546ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 27.880918ms iteration = 5
11502024-05-14T04:17:24.547ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = f07c863b-7137-4303-9021-5a5c6e9ed458 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
11512024-05-14T04:17:24.547ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
11522024-05-14T04:17:24.547ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-14 04:13:28.927466 UTC dns_group = internal generation = 1
11532024-05-14T04:17:24.547ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 28.361448ms iteration = 5
11542024-05-14T04:17:24.547ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11552024-05-14T04:17:24.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, lookup_type: ById(5a4ac577-a6fd-4dea-aa82-fc399ad2b82d) } result = Ok(())
11562024-05-14T04:17:24.550ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11572024-05-14T04:17:24.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11582024-05-14T04:17:24.552ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 33.718717ms iteration = 9
11592024-05-14T04:17:24.579ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-14T04:13:24.616159Z, time_modified: 2024-05-14T04:13:24.616159Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: b4d2f397-0b2e-4db5-8e33-d295c802134a, db_silo: Silo { identity: SiloIdentity { id: b4d2f397-0b2e-4db5-8e33-d295c802134a, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-14T04:13:29.381647Z, time_modified: 2024-05-14T04:13:29.381647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "52301c060909182b4e1b505a925f9bb8cc086c123ca178be7538af43a1ecc18d" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: b4d2f397-0b2e-4db5-8e33-d295c802134a, db_silo: Silo { identity: SiloIdentity { id: b4d2f397-0b2e-4db5-8e33-d295c802134a, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-14T04:13:29.381647Z, time_modified: 2024-05-14T04:13:29.381647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "52301c060909182b4e1b505a925f9bb8cc086c123ca178be7538af43a1ecc18d" }] }) }
11602024-05-14T04:17:24.579ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 60.36735ms iteration = 5
11612024-05-14T04:17:24.650ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
11622024-05-14T04:17:24.653ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
11632024-05-14T04:17:24.653ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11642024-05-14T04:17:24.653ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "30657257-3f4c-446a-b553-e76e454a0698", "content-length": "26", "date": "Tue, 14 May 2024 04:17:24 GMT"} })
11652024-05-14T04:17:24.653ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11662024-05-14T04:17:24.654ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11672024-05-14T04:17:24.660ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
11682024-05-14T04:17:24.660ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11692024-05-14T04:17:24.661ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9e6318a3-d917-4735-9882-15cef070ca70", "content-length": "26", "date": "Tue, 14 May 2024 04:17:24 GMT"} })
11702024-05-14T04:17:24.661ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11712024-05-14T04:17:24.664ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11722024-05-14T04:17:24.768ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
11732024-05-14T04:17:24.813ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 zone_address = fd00:1122:3344:101::2
11742024-05-14T04:17:24.813ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11752024-05-14T04:17:24.813ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "373ab2bd-74b9-4248-8f51-45d9b0f83123", "content-length": "26", "date": "Tue, 14 May 2024 04:17:24 GMT"} })
11762024-05-14T04:17:24.813ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11772024-05-14T04:17:24.814ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
11782024-05-14T04:17:24.820ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
11792024-05-14T04:17:24.820ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6b7cd8d8-bf1d-444d-bfac-65b016725f12", "content-length": "1", "date": "Tue, 14 May 2024 04:17:24 GMT"} })
11802024-05-14T04:17:24.821ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
11812024-05-14T04:17:24.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
11822024-05-14T04:17:24.823ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 304.575463ms iteration = 9
11832024-05-14T04:17:24.920ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
11842024-05-14T04:17:24.921ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c4b01f4f-2c8f-4139-a1d7-fdd121ad7a3d", "date": "Tue, 14 May 2024 04:17:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
11852024-05-14T04:17:24.921ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
11862024-05-14T04:17:24.923ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "831d0cef-d6f0-46ae-8b75-e83aed8d19d0", "date": "Tue, 14 May 2024 04:17:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
11872024-05-14T04:17:24.923ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
11882024-05-14T04:17:24.924ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
11892024-05-14T04:17:24.924ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 405.458999ms iteration = 7
11902024-05-14T04:17:24.925ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
11912024-05-14T04:17:24.925ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 406.895948ms iteration = 9
11922024-05-14T04:17:24.964ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
11932024-05-14T04:17:24.965ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "776fddd6-0005-4928-ae12-658ec137ba76", "date": "Tue, 14 May 2024 04:17:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
11942024-05-14T04:17:24.965ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
11952024-05-14T04:17:24.968ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "bff985a4-e488-4a49-bdf4-583a2db5c2b7", "date": "Tue, 14 May 2024 04:17:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
11962024-05-14T04:17:24.968ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
11972024-05-14T04:17:24.970ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b858a54d-ea1a-4c3d-a28d-1072da8506cb", "date": "Tue, 14 May 2024 04:17:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
11982024-05-14T04:17:24.970ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
11992024-05-14T04:17:24.973ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
12002024-05-14T04:17:24.973ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 453.975117ms iteration = 7
12012024-05-14T04:17:25.008ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12022024-05-14T04:17:25.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12032024-05-14T04:17:25.009ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: 384a5237-ec35-4842-bb57-a29c998a8bde, rack_id: b73a9888-3baa-4f16-9152-a3e1ba42c461, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(854ac53f-66c7-4bae-88ab-8ddb0746c0c4) } rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
12042024-05-14T04:17:25.009ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12052024-05-14T04:17:25.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12062024-05-14T04:17:25.016ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
12072024-05-14T04:17:25.016ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "94ce9022-aa87-4524-b9f0-f1f1f3173d67", "content-length": "121", "date": "Tue, 14 May 2024 04:17:24 GMT"} })
12082024-05-14T04:17:25.016ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
12092024-05-14T04:17:25.016ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 854ac53f-66c7-4bae-88ab-8ddb0746c0c4
12102024-05-14T04:17:25.017ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
12112024-05-14T04:17:25.017ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
12122024-05-14T04:17:25.017ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
12132024-05-14T04:17:25.017ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
12142024-05-14T04:17:25.017ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
12152024-05-14T04:17:25.017ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
12162024-05-14T04:17:25.017ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
12172024-05-14T04:17:25.017ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12182024-05-14T04:17:25.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12192024-05-14T04:17:25.019ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
12202024-05-14T04:17:25.019ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "da39701e-492c-4f6a-978e-3d53fb68beae", "content-length": "2", "date": "Tue, 14 May 2024 04:17:24 GMT"} })
12212024-05-14T04:17:25.019ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
12222024-05-14T04:17:25.019ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c5f45e0c-baea-4902-911d-af6be97be441", "content-length": "51", "date": "Tue, 14 May 2024 04:17:24 GMT"} })
12232024-05-14T04:17:25.019ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
12242024-05-14T04:17:25.019ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
12252024-05-14T04:17:25.063ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ca30119a-f9d2-4f8e-afe2-190c632141d2", "date": "Tue, 14 May 2024 04:17:24 GMT"} })
12262024-05-14T04:17:25.063ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch = Switch0
12272024-05-14T04:17:25.063ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
12282024-05-14T04:17:25.063ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ad10783d-2bd3-4bdc-a2dc-850a274c4c76", "content-length": "467", "date": "Tue, 14 May 2024 04:17:24 GMT"} })
12292024-05-14T04:17:25.063ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12302024-05-14T04:17:25.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12312024-05-14T04:17:25.065ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12322024-05-14T04:17:25.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12332024-05-14T04:17:25.068ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12342024-05-14T04:17:25.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12352024-05-14T04:17:25.069ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12362024-05-14T04:17:25.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12372024-05-14T04:17:25.075ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 555.884983ms iteration = 9
12382024-05-14T04:17:25.392ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 163 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 09647e75-1ff8-431a-898e-96afefa81dbd response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
12392024-05-14T04:17:34.518ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
12402024-05-14T04:17:34.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12412024-05-14T04:17:34.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12422024-05-14T04:17:34.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12432024-05-14T04:17:34.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12442024-05-14T04:17:34.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12452024-05-14T04:17:34.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12462024-05-14T04:17:34.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12472024-05-14T04:17:34.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12482024-05-14T04:17:34.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12492024-05-14T04:17:34.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
12502024-05-14T04:17:34.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12512024-05-14T04:17:34.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12522024-05-14T04:17:34.538ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
12532024-05-14T04:17:34.538ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.092329ms iteration = 26
12542024-05-14T04:17:35.392ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 189 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 9d110e56-1ded-484b-9ac1-793a7806a536 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
12552024-05-14T04:17:44.518ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
12562024-05-14T04:17:44.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12572024-05-14T04:17:44.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12582024-05-14T04:17:44.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12592024-05-14T04:17:44.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12602024-05-14T04:17:44.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12612024-05-14T04:17:44.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12622024-05-14T04:17:44.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12632024-05-14T04:17:44.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12642024-05-14T04:17:44.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12652024-05-14T04:17:44.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
12662024-05-14T04:17:44.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12672024-05-14T04:17:44.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12682024-05-14T04:17:44.534ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
12692024-05-14T04:17:44.534ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.517636ms iteration = 27
12702024-05-14T04:17:45.392ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 165 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 9eb088cc-a21c-4a26-be5a-ab2c2a5eb64d response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
12712024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
12722024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
12732024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12742024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12752024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
12762024-05-14T04:17:54.519ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
12772024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
12782024-05-14T04:17:54.519ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
12792024-05-14T04:17:54.519ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
12802024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 11 reason = Timeout
12812024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 11.46\u{b5}s iteration = 10
12822024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
12832024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12842024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
12852024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12862024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
12872024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
12882024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12892024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12902024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12912024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12922024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
12932024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12942024-05-14T04:17:54.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12952024-05-14T04:17:54.521ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12962024-05-14T04:17:54.521ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12972024-05-14T04:17:54.521ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12982024-05-14T04:17:54.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12992024-05-14T04:17:54.522ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
13002024-05-14T04:17:54.522ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.915189ms iteration = 10
13012024-05-14T04:17:54.523ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6899, tv_nsec: 184704690 } })
13022024-05-14T04:17:54.523ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6899, tv_nsec: 184822502 } })
13032024-05-14T04:17:54.523ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
13042024-05-14T04:17:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
13052024-05-14T04:17:54.524ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13062024-05-14T04:17:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13072024-05-14T04:17:54.524ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
13082024-05-14T04:17:54.524ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13092024-05-14T04:17:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
13102024-05-14T04:17:54.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
13112024-05-14T04:17:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
13122024-05-14T04:17:54.525ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13132024-05-14T04:17:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13142024-05-14T04:17:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
13152024-05-14T04:17:54.525ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13162024-05-14T04:17:54.525ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
13172024-05-14T04:17:54.525ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13182024-05-14T04:17:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13192024-05-14T04:17:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13202024-05-14T04:17:54.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13212024-05-14T04:17:54.526ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13222024-05-14T04:17:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13232024-05-14T04:17:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13242024-05-14T04:17:54.527ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6899, tv_nsec: 188329864 } })
13252024-05-14T04:17:54.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13262024-05-14T04:17:54.527ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13272024-05-14T04:17:54.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13282024-05-14T04:17:54.527ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13292024-05-14T04:17:54.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13302024-05-14T04:17:54.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
13312024-05-14T04:17:54.528ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13322024-05-14T04:17:54.528ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
13332024-05-14T04:17:54.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
13342024-05-14T04:17:54.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13352024-05-14T04:17:54.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
13362024-05-14T04:17:54.529ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13372024-05-14T04:17:54.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13382024-05-14T04:17:54.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
13392024-05-14T04:17:54.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13402024-05-14T04:17:54.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13412024-05-14T04:17:54.532ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
13422024-05-14T04:17:54.532ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 8.122235ms iteration = 10
13432024-05-14T04:17:54.533ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
13442024-05-14T04:17:54.533ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 14.415134ms iteration = 11
13452024-05-14T04:17:54.537ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
13462024-05-14T04:17:54.537ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.623459ms iteration = 28
13472024-05-14T04:17:54.540ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13482024-05-14T04:17:54.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13492024-05-14T04:17:54.542ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13502024-05-14T04:17:54.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13512024-05-14T04:17:54.543ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13522024-05-14T04:17:54.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, lookup_type: ById(5a4ac577-a6fd-4dea-aa82-fc399ad2b82d) } result = Ok(())
13532024-05-14T04:17:54.546ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13542024-05-14T04:17:54.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13552024-05-14T04:17:54.547ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.430479ms iteration = 10
13562024-05-14T04:17:54.593ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
13572024-05-14T04:17:54.593ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13582024-05-14T04:17:54.593ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0d4d209f-5d42-4979-9067-23e784b8bd04", "content-length": "26", "date": "Tue, 14 May 2024 04:17:54 GMT"} })
13592024-05-14T04:17:54.593ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13602024-05-14T04:17:54.594ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
13612024-05-14T04:17:54.594ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13622024-05-14T04:17:54.594ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9a2901e4-264e-4772-a899-63947c24f990", "content-length": "26", "date": "Tue, 14 May 2024 04:17:54 GMT"} })
13632024-05-14T04:17:54.594ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13642024-05-14T04:17:54.596ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13652024-05-14T04:17:54.596ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13662024-05-14T04:17:54.616ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 zone_address = fd00:1122:3344:101::2
13672024-05-14T04:17:54.616ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13682024-05-14T04:17:54.616ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fd05718e-23d8-4952-9e5b-5b47ad37f554", "content-length": "26", "date": "Tue, 14 May 2024 04:17:54 GMT"} })
13692024-05-14T04:17:54.616ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13702024-05-14T04:17:54.618ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
13712024-05-14T04:17:54.674ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 155.132449ms iteration = 10
13722024-05-14T04:17:54.679ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
13732024-05-14T04:17:54.680ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a4611b0e-261a-4440-ab38-c7b76b79abbb", "content-length": "1", "date": "Tue, 14 May 2024 04:17:54 GMT"} })
13742024-05-14T04:17:54.681ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13752024-05-14T04:17:54.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13762024-05-14T04:17:54.682ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 163.565612ms iteration = 10
13772024-05-14T04:17:54.742ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13782024-05-14T04:17:54.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13792024-05-14T04:17:54.743ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: 384a5237-ec35-4842-bb57-a29c998a8bde, rack_id: b73a9888-3baa-4f16-9152-a3e1ba42c461, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(854ac53f-66c7-4bae-88ab-8ddb0746c0c4) } rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
13802024-05-14T04:17:54.743ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13812024-05-14T04:17:54.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13822024-05-14T04:17:54.749ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
13832024-05-14T04:17:54.749ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8e4e7b86-a6b1-455c-9e26-4daba9035ad6", "content-length": "121", "date": "Tue, 14 May 2024 04:17:54 GMT"} })
13842024-05-14T04:17:54.750ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
13852024-05-14T04:17:54.750ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 854ac53f-66c7-4bae-88ab-8ddb0746c0c4
13862024-05-14T04:17:54.750ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
13872024-05-14T04:17:54.750ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
13882024-05-14T04:17:54.750ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
13892024-05-14T04:17:54.750ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
13902024-05-14T04:17:54.750ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
13912024-05-14T04:17:54.750ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
13922024-05-14T04:17:54.750ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
13932024-05-14T04:17:54.750ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13942024-05-14T04:17:54.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13952024-05-14T04:17:54.757ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
13962024-05-14T04:17:54.757ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "04a49301-f18b-40de-b84b-3866d9cbcd50", "content-length": "2", "date": "Tue, 14 May 2024 04:17:54 GMT"} })
13972024-05-14T04:17:54.757ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
13982024-05-14T04:17:54.758ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b6584480-48ee-4226-b1e1-720f89ff9b08", "content-length": "51", "date": "Tue, 14 May 2024 04:17:54 GMT"} })
13992024-05-14T04:17:54.758ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
14002024-05-14T04:17:54.758ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
14012024-05-14T04:17:54.805ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d4d58fe0-3f49-4aab-8d98-e75cd96e3d27", "date": "Tue, 14 May 2024 04:17:54 GMT"} })
14022024-05-14T04:17:54.805ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch = Switch0
14032024-05-14T04:17:54.805ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
14042024-05-14T04:17:54.805ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "546974c2-b6e6-44d2-93fb-d747eb192544", "content-length": "467", "date": "Tue, 14 May 2024 04:17:54 GMT"} })
14052024-05-14T04:17:54.805ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14062024-05-14T04:17:54.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14072024-05-14T04:17:54.808ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14082024-05-14T04:17:54.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14092024-05-14T04:17:54.811ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14102024-05-14T04:17:54.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14112024-05-14T04:17:54.812ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14122024-05-14T04:17:54.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14132024-05-14T04:17:54.817ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 298.235516ms iteration = 10
14142024-05-14T04:17:55.392ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 202 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = baca154f-3606-4273-96ca-c762c1ffef8f response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
14152024-05-14T04:18:04.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
14162024-05-14T04:18:04.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14172024-05-14T04:18:04.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14182024-05-14T04:18:04.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14192024-05-14T04:18:04.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14202024-05-14T04:18:04.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14212024-05-14T04:18:04.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14222024-05-14T04:18:04.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14232024-05-14T04:18:04.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14242024-05-14T04:18:04.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14252024-05-14T04:18:04.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
14262024-05-14T04:18:04.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14272024-05-14T04:18:04.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14282024-05-14T04:18:04.534ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
14292024-05-14T04:18:04.534ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.809102ms iteration = 29
14302024-05-14T04:18:05.393ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 186 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 3f4e9c14-be75-4539-aa08-7b0084bbea5d response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
14312024-05-14T04:18:14.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
14322024-05-14T04:18:14.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14332024-05-14T04:18:14.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14342024-05-14T04:18:14.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14352024-05-14T04:18:14.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14362024-05-14T04:18:14.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14372024-05-14T04:18:14.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14382024-05-14T04:18:14.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14392024-05-14T04:18:14.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14402024-05-14T04:18:14.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14412024-05-14T04:18:14.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
14422024-05-14T04:18:14.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14432024-05-14T04:18:14.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14442024-05-14T04:18:14.534ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
14452024-05-14T04:18:14.534ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.495815ms iteration = 30
14462024-05-14T04:18:15.392ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 161 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 39661756-a563-4422-8c59-56f71cf75cc7 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
14472024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
14482024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
14492024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14502024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
14512024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
14522024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14532024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
14542024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
14552024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
14562024-05-14T04:18:24.519ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
14572024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 9.94\u{b5}s iteration = 6
14582024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
14592024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
14602024-05-14T04:18:24.519ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce
14612024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.391\u{b5}s iteration = 7
14622024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
14632024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14642024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
14652024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
14662024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
14672024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14682024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
14692024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
14702024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14712024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14722024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
14732024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14742024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
14752024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
14762024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14772024-05-14T04:18:24.519ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
14782024-05-14T04:18:24.519ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
14792024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.71\u{b5}s iteration = 11
14802024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
14812024-05-14T04:18:24.519ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
14822024-05-14T04:18:24.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
14832024-05-14T04:18:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14842024-05-14T04:18:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14852024-05-14T04:18:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14862024-05-14T04:18:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14872024-05-14T04:18:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14882024-05-14T04:18:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
14892024-05-14T04:18:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14902024-05-14T04:18:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14912024-05-14T04:18:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14922024-05-14T04:18:24.520ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("08a9c078-20b8-4325-90f1-e360e31ca4ab.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("3f6ad63d-f2d7-4b36-a169-d3099a239267.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("3f6ad63d-f2d7-4b36-a169-d3099a239267.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 6929, tv_nsec: 180999474 } })
14932024-05-14T04:18:24.520ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0dd5d513-ee28-4da3-83f2-7a3d7e3b267d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6aa90bd6-c3db-4dfa-88c0-97308b03c15f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9a8a4960-0835-4c5e-b472-b2521417e003.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9a8a4960-0835-4c5e-b472-b2521417e003.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6929, tv_nsec: 181007495 } })
14942024-05-14T04:18:24.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14952024-05-14T04:18:24.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14962024-05-14T04:18:24.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14972024-05-14T04:18:24.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14982024-05-14T04:18:24.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14992024-05-14T04:18:24.521ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15002024-05-14T04:18:24.522ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
15012024-05-14T04:18:24.522ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.681064ms iteration = 11
15022024-05-14T04:18:24.522ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15032024-05-14T04:18:24.523ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15042024-05-14T04:18:24.523ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
15052024-05-14T04:18:24.523ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.586713ms iteration = 6
15062024-05-14T04:18:24.523ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6929, tv_nsec: 183967405 } })
15072024-05-14T04:18:24.523ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
15082024-05-14T04:18:24.523ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.681444ms iteration = 6
15092024-05-14T04:18:24.523ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15102024-05-14T04:18:24.523ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15112024-05-14T04:18:24.524ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
15122024-05-14T04:18:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
15132024-05-14T04:18:24.525ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15142024-05-14T04:18:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15152024-05-14T04:18:24.525ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
15162024-05-14T04:18:24.525ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15172024-05-14T04:18:24.525ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
15182024-05-14T04:18:24.525ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
15192024-05-14T04:18:24.525ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15202024-05-14T04:18:24.525ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6929, tv_nsec: 186392564 } })
15212024-05-14T04:18:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
15222024-05-14T04:18:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15232024-05-14T04:18:24.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
15242024-05-14T04:18:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15252024-05-14T04:18:24.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15262024-05-14T04:18:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15272024-05-14T04:18:24.527ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
15282024-05-14T04:18:24.527ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15292024-05-14T04:18:24.527ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15302024-05-14T04:18:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15312024-05-14T04:18:24.527ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6929, tv_nsec: 188338894 } })
15322024-05-14T04:18:24.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15332024-05-14T04:18:24.527ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15342024-05-14T04:18:24.527ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
15352024-05-14T04:18:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15362024-05-14T04:18:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15372024-05-14T04:18:24.528ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15382024-05-14T04:18:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15392024-05-14T04:18:24.528ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15402024-05-14T04:18:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
15412024-05-14T04:18:24.528ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15422024-05-14T04:18:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15432024-05-14T04:18:24.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15442024-05-14T04:18:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15452024-05-14T04:18:24.529ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15462024-05-14T04:18:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15472024-05-14T04:18:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
15482024-05-14T04:18:24.529ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15492024-05-14T04:18:24.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15502024-05-14T04:18:24.530ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
15512024-05-14T04:18:24.530ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15522024-05-14T04:18:24.530ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15532024-05-14T04:18:24.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15542024-05-14T04:18:24.531ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15552024-05-14T04:18:24.531ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15562024-05-14T04:18:24.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15572024-05-14T04:18:24.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15582024-05-14T04:18:24.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15592024-05-14T04:18:24.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
15602024-05-14T04:18:24.533ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15612024-05-14T04:18:24.533ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15622024-05-14T04:18:24.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15632024-05-14T04:18:24.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
15642024-05-14T04:18:24.534ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15652024-05-14T04:18:24.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15662024-05-14T04:18:24.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15672024-05-14T04:18:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
15682024-05-14T04:18:24.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15692024-05-14T04:18:24.535ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
15702024-05-14T04:18:24.535ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
15712024-05-14T04:18:24.535ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
15722024-05-14T04:18:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15732024-05-14T04:18:24.535ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15742024-05-14T04:18:24.535ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15752024-05-14T04:18:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15762024-05-14T04:18:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15772024-05-14T04:18:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15782024-05-14T04:18:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
15792024-05-14T04:18:24.535ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15802024-05-14T04:18:24.536ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15812024-05-14T04:18:24.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15822024-05-14T04:18:24.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15832024-05-14T04:18:24.537ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
15842024-05-14T04:18:24.537ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 12.385103ms iteration = 11
15852024-05-14T04:18:24.538ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15862024-05-14T04:18:24.538ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
15872024-05-14T04:18:24.538ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 19.299293ms iteration = 12
15882024-05-14T04:18:24.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15892024-05-14T04:18:24.540ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15902024-05-14T04:18:24.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15912024-05-14T04:18:24.540ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15922024-05-14T04:18:24.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15932024-05-14T04:18:24.541ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
15942024-05-14T04:18:24.541ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.891786ms iteration = 31
15952024-05-14T04:18:24.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
15962024-05-14T04:18:24.541ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15972024-05-14T04:18:24.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15982024-05-14T04:18:24.542ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
15992024-05-14T04:18:24.542ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = f07c863b-7137-4303-9021-5a5c6e9ed458 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
16002024-05-14T04:18:24.542ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16012024-05-14T04:18:24.542ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
16022024-05-14T04:18:24.542ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16032024-05-14T04:18:24.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16042024-05-14T04:18:24.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16052024-05-14T04:18:24.543ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16062024-05-14T04:18:24.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16072024-05-14T04:18:24.543ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16082024-05-14T04:18:24.544ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16092024-05-14T04:18:24.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16102024-05-14T04:18:24.545ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16112024-05-14T04:18:24.545ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16122024-05-14T04:18:24.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16132024-05-14T04:18:24.547ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16142024-05-14T04:18:24.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16152024-05-14T04:18:24.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
16162024-05-14T04:18:24.548ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16172024-05-14T04:18:24.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16182024-05-14T04:18:24.548ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16192024-05-14T04:18:24.550ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = f07c863b-7137-4303-9021-5a5c6e9ed458 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
16202024-05-14T04:18:24.550ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
16212024-05-14T04:18:24.550ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-14 04:13:28.927466 UTC dns_group = internal generation = 1
16222024-05-14T04:18:24.550ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 30.609584ms iteration = 6
16232024-05-14T04:18:24.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
16242024-05-14T04:18:24.550ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16252024-05-14T04:18:24.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16262024-05-14T04:18:24.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, lookup_type: ById(5a4ac577-a6fd-4dea-aa82-fc399ad2b82d) } result = Ok(())
16272024-05-14T04:18:24.551ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16282024-05-14T04:18:24.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16292024-05-14T04:18:24.553ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 33.942012ms iteration = 11
16302024-05-14T04:18:24.553ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16312024-05-14T04:18:24.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16322024-05-14T04:18:24.555ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16332024-05-14T04:18:24.555ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
16342024-05-14T04:18:24.555ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
16352024-05-14T04:18:24.555ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
16362024-05-14T04:18:24.555ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16372024-05-14T04:18:24.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16382024-05-14T04:18:24.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16392024-05-14T04:18:24.557ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16402024-05-14T04:18:24.557ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16412024-05-14T04:18:24.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16422024-05-14T04:18:24.558ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16432024-05-14T04:18:24.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16442024-05-14T04:18:24.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
16452024-05-14T04:18:24.560ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16462024-05-14T04:18:24.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16472024-05-14T04:18:24.560ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16482024-05-14T04:18:24.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16492024-05-14T04:18:24.561ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16502024-05-14T04:18:24.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16512024-05-14T04:18:24.561ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
16522024-05-14T04:18:24.562ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 4ac98832-1cdd-4449-9949-cf5fe247d837 dns_zone_name = oxide.test found_so_far = 0 version = 2
16532024-05-14T04:18:24.562ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16542024-05-14T04:18:24.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16552024-05-14T04:18:24.563ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16562024-05-14T04:18:24.563ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16572024-05-14T04:18:24.566ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
16582024-05-14T04:18:24.566ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16592024-05-14T04:18:24.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16602024-05-14T04:18:24.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
16612024-05-14T04:18:24.568ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 4ac98832-1cdd-4449-9949-cf5fe247d837 dns_zone_name = oxide.test found_so_far = 1 version = 2
16622024-05-14T04:18:24.568ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
16632024-05-14T04:18:24.568ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-14 04:13:29.424952 UTC dns_group = external generation = 2
16642024-05-14T04:18:24.568ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 48.676272ms iteration = 6
16652024-05-14T04:18:24.570ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16662024-05-14T04:18:24.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16672024-05-14T04:18:24.572ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16682024-05-14T04:18:24.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16692024-05-14T04:18:24.573ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16702024-05-14T04:18:24.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16712024-05-14T04:18:24.577ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16722024-05-14T04:18:24.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16732024-05-14T04:18:24.581ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-14T04:13:24.616159Z, time_modified: 2024-05-14T04:13:24.616159Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: b4d2f397-0b2e-4db5-8e33-d295c802134a, db_silo: Silo { identity: SiloIdentity { id: b4d2f397-0b2e-4db5-8e33-d295c802134a, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-14T04:13:29.381647Z, time_modified: 2024-05-14T04:13:29.381647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "52301c060909182b4e1b505a925f9bb8cc086c123ca178be7538af43a1ecc18d" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: b4d2f397-0b2e-4db5-8e33-d295c802134a, db_silo: Silo { identity: SiloIdentity { id: b4d2f397-0b2e-4db5-8e33-d295c802134a, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-14T04:13:29.381647Z, time_modified: 2024-05-14T04:13:29.381647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "52301c060909182b4e1b505a925f9bb8cc086c123ca178be7538af43a1ecc18d" }] }) }
16742024-05-14T04:18:24.581ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 61.537544ms iteration = 6
16752024-05-14T04:18:24.582ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16762024-05-14T04:18:24.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16772024-05-14T04:18:24.583ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16782024-05-14T04:18:24.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16792024-05-14T04:18:24.584ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16802024-05-14T04:18:24.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16812024-05-14T04:18:24.585ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16822024-05-14T04:18:24.590ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
16832024-05-14T04:18:24.590ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16842024-05-14T04:18:24.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16852024-05-14T04:18:24.591ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16862024-05-14T04:18:24.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16872024-05-14T04:18:24.593ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16882024-05-14T04:18:24.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16892024-05-14T04:18:24.594ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16902024-05-14T04:18:24.598ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
16912024-05-14T04:18:24.598ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16922024-05-14T04:18:24.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16932024-05-14T04:18:24.606ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16942024-05-14T04:18:24.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16952024-05-14T04:18:24.607ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16962024-05-14T04:18:24.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16972024-05-14T04:18:24.608ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16982024-05-14T04:18:24.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16992024-05-14T04:18:24.610ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17002024-05-14T04:18:24.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17012024-05-14T04:18:24.622ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17022024-05-14T04:18:24.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17032024-05-14T04:18:24.626ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17042024-05-14T04:18:24.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17052024-05-14T04:18:24.627ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17062024-05-14T04:18:24.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17072024-05-14T04:18:24.629ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17082024-05-14T04:18:24.636ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
17092024-05-14T04:18:24.636ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17102024-05-14T04:18:24.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17112024-05-14T04:18:24.637ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17122024-05-14T04:18:24.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17132024-05-14T04:18:24.639ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17142024-05-14T04:18:24.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17152024-05-14T04:18:24.640ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17162024-05-14T04:18:24.641ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
17172024-05-14T04:18:24.645ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
17182024-05-14T04:18:24.645ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17192024-05-14T04:18:24.645ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
17202024-05-14T04:18:24.645ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17212024-05-14T04:18:24.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17222024-05-14T04:18:24.645ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e68276c2-c725-4227-8678-de2a4e02ae78", "content-length": "26", "date": "Tue, 14 May 2024 04:18:24 GMT"} })
17232024-05-14T04:18:24.645ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17242024-05-14T04:18:24.647ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
17252024-05-14T04:18:24.649ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
17262024-05-14T04:18:24.649ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17272024-05-14T04:18:24.649ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8bf7526b-05b4-4777-b37d-132088e1188f", "content-length": "26", "date": "Tue, 14 May 2024 04:18:24 GMT"} })
17282024-05-14T04:18:24.649ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17292024-05-14T04:18:24.651ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: e62da211-12c5-4692-8c20-db5bc7fe07c1, kind: Service { id: 3f6ad63d-f2d7-4b36-a169-d3099a239267 }, name: Name("external-dns-3f6ad63d-f2d7-4b36-a169-d3099a239267"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 144, 234])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: f57799ac-aa6e-4ba0-bc52-58682a5e6565, kind: Service { id: 08a9c078-20b8-4325-90f1-e360e31ca4ab }, name: Name("external-dns-08a9c078-20b8-4325-90f1-e360e31ca4ab"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 205])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 39434b96-9afb-4275-820d-5b03cc9bd22e, kind: Service { id: d4fac703-f5ba-4b34-aac1-7fc066e8abb3 }, name: Name("nexus-d4fac703-f5ba-4b34-aac1-7fc066e8abb3"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 227, 112])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9c5fd2f0-c15e-4b9a-a187-1219adc66af4, kind: Service { id: 4db81567-f1e3-4377-871b-92d735132e59 }, name: Name("nexus-4db81567-f1e3-4377-871b-92d735132e59"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 196, 247])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: baa7aa7d-6397-4839-b691-60f701e425ea, kind: Service { id: ef1c9f55-f835-41e6-8d3a-81e26da276c7 }, name: Name("nexus-ef1c9f55-f835-41e6-8d3a-81e26da276c7"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 122])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
17302024-05-14T04:18:24.651ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17312024-05-14T04:18:24.651ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
17322024-05-14T04:18:24.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17332024-05-14T04:18:24.652ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17342024-05-14T04:18:24.654ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
17352024-05-14T04:18:24.654ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17362024-05-14T04:18:24.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17372024-05-14T04:18:24.654ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
17382024-05-14T04:18:24.658ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
17392024-05-14T04:18:24.658ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [VpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: e62da211-12c5-4692-8c20-db5bc7fe07c1, kind: Service { id: 3f6ad63d-f2d7-4b36-a169-d3099a239267 }, name: Name("external-dns-3f6ad63d-f2d7-4b36-a169-d3099a239267"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 144, 234])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: f57799ac-aa6e-4ba0-bc52-58682a5e6565, kind: Service { id: 08a9c078-20b8-4325-90f1-e360e31ca4ab }, name: Name("external-dns-08a9c078-20b8-4325-90f1-e360e31ca4ab"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 205])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }] }, VpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 39434b96-9afb-4275-820d-5b03cc9bd22e, kind: Service { id: d4fac703-f5ba-4b34-aac1-7fc066e8abb3 }, name: Name("nexus-d4fac703-f5ba-4b34-aac1-7fc066e8abb3"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 227, 112])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9c5fd2f0-c15e-4b9a-a187-1219adc66af4, kind: Service { id: 4db81567-f1e3-4377-871b-92d735132e59 }, name: Name("nexus-4db81567-f1e3-4377-871b-92d735132e59"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 196, 247])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: baa7aa7d-6397-4839-b691-60f701e425ea, kind: Service { id: ef1c9f55-f835-41e6-8d3a-81e26da276c7 }, name: Name("nexus-ef1c9f55-f835-41e6-8d3a-81e26da276c7"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 122])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }] }]
17402024-05-14T04:18:24.658ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
17412024-05-14T04:18:24.686ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 zone_address = fd00:1122:3344:101::2
17422024-05-14T04:18:24.686ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17432024-05-14T04:18:24.686ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1abd299f-5fef-45e7-8e35-08be1fdbbabf", "content-length": "26", "date": "Tue, 14 May 2024 04:18:24 GMT"} })
17442024-05-14T04:18:24.686ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17452024-05-14T04:18:24.688ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
17462024-05-14T04:18:24.798ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
17472024-05-14T04:18:24.798ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "15ef0698-2735-4e84-9c71-dacec0e931bf", "date": "Tue, 14 May 2024 04:18:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
17482024-05-14T04:18:24.798ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
17492024-05-14T04:18:24.800ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e7b6cf73-f4c4-47ff-8b54-2c43fcb6b481", "date": "Tue, 14 May 2024 04:18:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
17502024-05-14T04:18:24.800ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
17512024-05-14T04:18:24.803ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
17522024-05-14T04:18:24.803ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 284.136657ms iteration = 8
17532024-05-14T04:18:24.805ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 285.969915ms iteration = 11
17542024-05-14T04:18:24.806ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
17552024-05-14T04:18:24.806ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0d96869d-2c27-4011-b995-29e88e171acc", "content-length": "1", "date": "Tue, 14 May 2024 04:18:24 GMT"} })
17562024-05-14T04:18:24.807ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
17572024-05-14T04:18:24.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
17582024-05-14T04:18:24.809ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 290.054009ms iteration = 11
17592024-05-14T04:18:24.857ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
17602024-05-14T04:18:25.004ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
17612024-05-14T04:18:25.004ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a0b77696-d9de-4c9a-b893-01ee81cb7390", "date": "Tue, 14 May 2024 04:18:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
17622024-05-14T04:18:25.004ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
17632024-05-14T04:18:25.006ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "91568cc6-37ff-4626-a97f-c22f5007eeb2", "date": "Tue, 14 May 2024 04:18:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
17642024-05-14T04:18:25.006ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
17652024-05-14T04:18:25.008ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5f212279-418d-4e18-954c-8781bda40557", "date": "Tue, 14 May 2024 04:18:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
17662024-05-14T04:18:25.008ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
17672024-05-14T04:18:25.010ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
17682024-05-14T04:18:25.010ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 490.648744ms iteration = 8
17692024-05-14T04:18:25.020ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, time_created: 2024-05-14T04:13:27.542920Z, time_modified: 2024-05-14T04:13:27.542920Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b73a9888-3baa-4f16-9152-a3e1ba42c461, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: 0, usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
17702024-05-14T04:18:25.020ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
17712024-05-14T04:18:25.020ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17722024-05-14T04:18:25.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17732024-05-14T04:18:25.023ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17742024-05-14T04:18:25.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17752024-05-14T04:18:25.024ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17762024-05-14T04:18:25.027ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, lookup_type: ById(5a4ac577-a6fd-4dea-aa82-fc399ad2b82d) } result = Ok(())
17772024-05-14T04:18:25.066ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17782024-05-14T04:18:25.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17792024-05-14T04:18:25.067ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: 384a5237-ec35-4842-bb57-a29c998a8bde, rack_id: b73a9888-3baa-4f16-9152-a3e1ba42c461, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(854ac53f-66c7-4bae-88ab-8ddb0746c0c4) } rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
17802024-05-14T04:18:25.067ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17812024-05-14T04:18:25.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17822024-05-14T04:18:25.073ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
17832024-05-14T04:18:25.074ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "451e262f-2ee0-43c8-b805-54992e19cc5a", "content-length": "121", "date": "Tue, 14 May 2024 04:18:24 GMT"} })
17842024-05-14T04:18:25.074ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
17852024-05-14T04:18:25.074ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 854ac53f-66c7-4bae-88ab-8ddb0746c0c4
17862024-05-14T04:18:25.075ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
17872024-05-14T04:18:25.075ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
17882024-05-14T04:18:25.075ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
17892024-05-14T04:18:25.075ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
17902024-05-14T04:18:25.075ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
17912024-05-14T04:18:25.075ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
17922024-05-14T04:18:25.075ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
17932024-05-14T04:18:25.075ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17942024-05-14T04:18:25.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17952024-05-14T04:18:25.076ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
17962024-05-14T04:18:25.076ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "38bf0d2f-d823-48ef-8aba-2759871b5848", "content-length": "2", "date": "Tue, 14 May 2024 04:18:24 GMT"} })
17972024-05-14T04:18:25.076ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
17982024-05-14T04:18:25.076ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2f21afcc-a1e8-497b-ace9-121a60fdfd9b", "content-length": "51", "date": "Tue, 14 May 2024 04:18:24 GMT"} })
17992024-05-14T04:18:25.076ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
18002024-05-14T04:18:25.076ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
18012024-05-14T04:18:25.091ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18022024-05-14T04:18:25.130ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "80e1dd3a-d16a-44a6-a2d0-a3ed9ff2c7c4", "date": "Tue, 14 May 2024 04:18:25 GMT"} })
18032024-05-14T04:18:25.130ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch = Switch0
18042024-05-14T04:18:25.130ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
18052024-05-14T04:18:25.130ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e2f7d412-6bc7-4f6b-ba02-b71b74053894", "content-length": "467", "date": "Tue, 14 May 2024 04:18:24 GMT"} })
18062024-05-14T04:18:25.131ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18072024-05-14T04:18:25.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18082024-05-14T04:18:25.131ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d background_task = service_firewall_rule_propagation result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", query: None, fragment: None }, status: 204, headers: {"x-request-id": "866e4b5d-4ef5-4312-8561-7edcaf2dd421", "date": "Tue, 14 May 2024 04:18:25 GMT"} })
18092024-05-14T04:18:25.132ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:486
18102024-05-14T04:18:25.132ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 607.231599ms
18112024-05-14T04:18:25.132ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 607.238469ms iteration = 2
18122024-05-14T04:18:25.133ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18132024-05-14T04:18:25.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18142024-05-14T04:18:25.137ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18152024-05-14T04:18:25.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18162024-05-14T04:18:25.138ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18172024-05-14T04:18:25.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18182024-05-14T04:18:25.144ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 624.807127ms iteration = 11
18192024-05-14T04:18:25.393ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 162 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = feb1f699-b793-4460-8861-fa7ca50606d5 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
18202024-05-14T04:18:34.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
18212024-05-14T04:18:34.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18222024-05-14T04:18:34.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18232024-05-14T04:18:34.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18242024-05-14T04:18:34.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18252024-05-14T04:18:34.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18262024-05-14T04:18:34.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18272024-05-14T04:18:34.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18282024-05-14T04:18:34.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18292024-05-14T04:18:34.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18302024-05-14T04:18:34.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
18312024-05-14T04:18:34.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18322024-05-14T04:18:34.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18332024-05-14T04:18:34.533ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
18342024-05-14T04:18:34.533ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.863312ms iteration = 32
18352024-05-14T04:18:35.393ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 862 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 0b110b4f-d0ce-4660-adcf-43dfe387df72 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
18362024-05-14T04:18:44.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
18372024-05-14T04:18:44.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18382024-05-14T04:18:44.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18392024-05-14T04:18:44.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18402024-05-14T04:18:44.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18412024-05-14T04:18:44.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18422024-05-14T04:18:44.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18432024-05-14T04:18:44.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18442024-05-14T04:18:44.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18452024-05-14T04:18:44.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18462024-05-14T04:18:44.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
18472024-05-14T04:18:44.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18482024-05-14T04:18:44.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18492024-05-14T04:18:44.534ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
18502024-05-14T04:18:44.534ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.073297ms iteration = 33
18512024-05-14T04:18:45.392ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 141 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 6af8b42e-a824-4f75-ac48-e0522f5a25b6 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
18522024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
18532024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
18542024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18552024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
18562024-05-14T04:18:54.519ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
18572024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18582024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
18592024-05-14T04:18:54.519ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
18602024-05-14T04:18:54.519ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
18612024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.21\u{b5}s iteration = 12
18622024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
18632024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
18642024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
18652024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18662024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
18672024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
18682024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
18692024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18702024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18712024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
18722024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
18732024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
18742024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18752024-05-14T04:18:54.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18762024-05-14T04:18:54.520ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18772024-05-14T04:18:54.521ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18782024-05-14T04:18:54.521ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18792024-05-14T04:18:54.521ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
18802024-05-14T04:18:54.521ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.603373ms iteration = 12
18812024-05-14T04:18:54.521ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6959, tv_nsec: 182499715 } })
18822024-05-14T04:18:54.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18832024-05-14T04:18:54.523ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
18842024-05-14T04:18:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
18852024-05-14T04:18:54.524ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18862024-05-14T04:18:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
18872024-05-14T04:18:54.525ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18882024-05-14T04:18:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18892024-05-14T04:18:54.525ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
18902024-05-14T04:18:54.525ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
18912024-05-14T04:18:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
18922024-05-14T04:18:54.525ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6959, tv_nsec: 186039727 } })
18932024-05-14T04:18:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
18942024-05-14T04:18:54.525ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18952024-05-14T04:18:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
18962024-05-14T04:18:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18972024-05-14T04:18:54.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
18982024-05-14T04:18:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18992024-05-14T04:18:54.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19002024-05-14T04:18:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19012024-05-14T04:18:54.526ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
19022024-05-14T04:18:54.526ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19032024-05-14T04:18:54.527ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6959, tv_nsec: 187787332 } })
19042024-05-14T04:18:54.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19052024-05-14T04:18:54.527ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19062024-05-14T04:18:54.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19072024-05-14T04:18:54.527ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19082024-05-14T04:18:54.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19092024-05-14T04:18:54.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19102024-05-14T04:18:54.528ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
19112024-05-14T04:18:54.528ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19122024-05-14T04:18:54.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19132024-05-14T04:18:54.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
19142024-05-14T04:18:54.531ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
19152024-05-14T04:18:54.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19162024-05-14T04:18:54.531ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19172024-05-14T04:18:54.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19182024-05-14T04:18:54.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
19192024-05-14T04:18:54.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
19202024-05-14T04:18:54.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19212024-05-14T04:18:54.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19222024-05-14T04:18:54.547ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
19232024-05-14T04:18:54.547ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.728365ms iteration = 34
19242024-05-14T04:18:54.549ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19252024-05-14T04:18:54.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19262024-05-14T04:18:54.551ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19272024-05-14T04:18:54.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19282024-05-14T04:18:54.553ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19292024-05-14T04:18:54.555ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, lookup_type: ById(5a4ac577-a6fd-4dea-aa82-fc399ad2b82d) } result = Ok(())
19302024-05-14T04:18:54.555ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19312024-05-14T04:18:54.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19322024-05-14T04:18:54.558ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 39.211849ms iteration = 12
19332024-05-14T04:18:54.567ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
19342024-05-14T04:18:54.567ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 48.039369ms iteration = 13
19352024-05-14T04:18:54.635ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 zone_address = fd00:1122:3344:101::2
19362024-05-14T04:18:54.635ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19372024-05-14T04:18:54.636ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "946083c4-374c-4195-9407-a28197e4162c", "content-length": "26", "date": "Tue, 14 May 2024 04:18:54 GMT"} })
19382024-05-14T04:18:54.636ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19392024-05-14T04:18:54.638ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
19402024-05-14T04:18:54.718ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
19412024-05-14T04:18:54.718ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19422024-05-14T04:18:54.718ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "51f3385d-978e-44bf-b4e4-6e01190a09f5", "content-length": "26", "date": "Tue, 14 May 2024 04:18:54 GMT"} })
19432024-05-14T04:18:54.718ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19442024-05-14T04:18:54.721ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
19452024-05-14T04:18:54.725ZDEBGnexus (BackgroundTasks): client request SledAgent = 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/instances/81352410-feca-4767-8735-ac7a16a7c095/state
19462024-05-14T04:18:54.759ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
19472024-05-14T04:18:54.759ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19482024-05-14T04:18:54.760ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f28030a4-4dc4-4ad3-9d01-7da0bec63da6", "content-length": "26", "date": "Tue, 14 May 2024 04:18:54 GMT"} })
19492024-05-14T04:18:54.760ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19502024-05-14T04:18:54.762ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
19512024-05-14T04:18:54.924ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
19522024-05-14T04:18:54.925ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2f734070-153c-4812-88a2-99976c78771d", "content-length": "1", "date": "Tue, 14 May 2024 04:18:54 GMT"} })
19532024-05-14T04:18:54.926ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19542024-05-14T04:18:54.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19552024-05-14T04:18:54.928ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 409.551572ms iteration = 12
19562024-05-14T04:18:54.945ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 426.201282ms iteration = 12
19572024-05-14T04:18:54.949ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19582024-05-14T04:18:54.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19592024-05-14T04:18:54.951ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: 384a5237-ec35-4842-bb57-a29c998a8bde, rack_id: b73a9888-3baa-4f16-9152-a3e1ba42c461, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(854ac53f-66c7-4bae-88ab-8ddb0746c0c4) } rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
19602024-05-14T04:18:54.951ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19612024-05-14T04:18:54.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19622024-05-14T04:18:54.957ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
19632024-05-14T04:18:54.958ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0f4c1b30-d7ba-4a67-b0fa-b7320aa9ca8d", "content-length": "121", "date": "Tue, 14 May 2024 04:18:54 GMT"} })
19642024-05-14T04:18:54.958ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
19652024-05-14T04:18:54.958ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 854ac53f-66c7-4bae-88ab-8ddb0746c0c4
19662024-05-14T04:18:54.958ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
19672024-05-14T04:18:54.959ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
19682024-05-14T04:18:54.959ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
19692024-05-14T04:18:54.959ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
19702024-05-14T04:18:54.959ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
19712024-05-14T04:18:54.959ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
19722024-05-14T04:18:54.959ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
19732024-05-14T04:18:54.959ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19742024-05-14T04:18:54.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19752024-05-14T04:18:54.960ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
19762024-05-14T04:18:54.960ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5a05c3eb-d51f-4202-9a61-d4fe9199071e", "content-length": "2", "date": "Tue, 14 May 2024 04:18:54 GMT"} })
19772024-05-14T04:18:54.960ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
19782024-05-14T04:18:54.961ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5c754135-b405-41a2-afc3-bf859b923fe7", "content-length": "51", "date": "Tue, 14 May 2024 04:18:54 GMT"} })
19792024-05-14T04:18:54.961ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
19802024-05-14T04:18:54.961ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
19812024-05-14T04:18:55.008ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ff3fb169-ea8c-476b-ac0d-8edf34be7a42", "date": "Tue, 14 May 2024 04:18:54 GMT"} })
19822024-05-14T04:18:55.008ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch = Switch0
19832024-05-14T04:18:55.008ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
19842024-05-14T04:18:55.008ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5c717ae1-39f1-46b3-9ada-6e51e8a24371", "content-length": "467", "date": "Tue, 14 May 2024 04:18:54 GMT"} })
19852024-05-14T04:18:55.008ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19862024-05-14T04:18:55.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19872024-05-14T04:18:55.012ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19882024-05-14T04:18:55.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19892024-05-14T04:18:55.015ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19902024-05-14T04:18:55.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19912024-05-14T04:18:55.016ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19922024-05-14T04:18:55.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19932024-05-14T04:18:55.023ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 504.389534ms iteration = 12
19942024-05-14T04:18:55.393ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 170 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 9a467e0f-f01c-42f1-9958-fc79e1b27db3 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
19952024-05-14T04:18:58.803ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1:0]:36620
19962024-05-14T04:18:58.804ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:36620 req_id = 8c790928-d369-4095-b2c8-203beeef5220 roles = RoleSet { roles: {} } uri = /metrics/producers
19972024-05-14T04:18:58.804ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:36620 req_id = 8c790928-d369-4095-b2c8-203beeef5220 resource = Database result = Ok(()) uri = /metrics/producers
19982024-05-14T04:18:58.833ZDEBGnexus (BackgroundTasks): client response SledAgent = 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d background_task = instance_watcher result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/instances/81352410-feca-4767-8735-ac7a16a7c095/state", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c1b2a4e3-faa9-4563-b350-c8cc2784dab2", "content-length": "311", "date": "Tue, 14 May 2024 04:18:58 GMT"} })
19992024-05-14T04:18:58.837ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 90873ca0-52f3-4dd5-a08c-9897fa8723c5
20002024-05-14T04:18:58.837ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:36620 req_id = 8c790928-d369-4095-b2c8-203beeef5220 roles = RoleSet { roles: {} } uri = /metrics/producers
20012024-05-14T04:18:58.837ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:36620 req_id = 8c790928-d369-4095-b2c8-203beeef5220 resource = Database result = Ok(()) uri = /metrics/producers
20022024-05-14T04:18:58.838ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 state = Starting
20032024-05-14T04:18:58.838ZINFOnexus (BackgroundTasks): received new runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:1979 instance_id = 81352410-feca-4767-8735-ac7a16a7c095 instance_state = InstanceRuntimeState { propolis_id: Some(658dcf17-14c1-4107-80a5-1311c4e1c50a), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-05-14T04:18:50.952660066Z } propolis_id = 658dcf17-14c1-4107-80a5-1311c4e1c50a vmm_state = VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2024-05-14T04:18:50.927585Z }
20042024-05-14T04:18:58.838ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 roles = RoleSet { roles: {} }
20052024-05-14T04:18:58.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 resource = Database result = Ok(())
20062024-05-14T04:18:58.840ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 roles = RoleSet { roles: {} }
20072024-05-14T04:18:58.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 resource = Database result = Ok(())
20082024-05-14T04:18:58.842ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 roles = RoleSet { roles: {} }
20092024-05-14T04:18:58.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 resource = Database result = Ok(())
20102024-05-14T04:18:58.846ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 roles = RoleSet { roles: {} }
20112024-05-14T04:18:58.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 resource = Database result = Ok(())
20122024-05-14T04:18:58.847ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 roles = RoleSet { roles: {} }
20132024-05-14T04:18:58.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 resource = Database result = Ok(())
20142024-05-14T04:18:58.849ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 roles = RoleSet { roles: {} }
20152024-05-14T04:18:58.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 resource = Database result = Ok(())
20162024-05-14T04:18:58.850ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20172024-05-14T04:18:58.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: b4d2f397-0b2e-4db5-8e33-d295c802134a, lookup_type: ById(b4d2f397-0b2e-4db5-8e33-d295c802134a) }, key: 50703023-9ceb-49c3-96d8-d7a97bbf9851, lookup_type: ById(50703023-9ceb-49c3-96d8-d7a97bbf9851) }, key: 81352410-feca-4767-8735-ac7a16a7c095, lookup_type: ById(81352410-feca-4767-8735-ac7a16a7c095) } result = Ok(())
20182024-05-14T04:18:58.855ZDEBGnexus (BackgroundTasks): instance state generation already advanced, won't touch network config background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095
20192024-05-14T04:18:58.878ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 90873ca0-52f3-4dd5-a08c-9897fa8723c5 uri = http://[fd00:1122:3344:101::d]:12223/producers
20202024-05-14T04:18:58.879ZDEBGnexus: client response oximeter-collector = 90873ca0-52f3-4dd5-a08c-9897fa8723c5 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "97c275b7-a0e6-4a8c-a254-1aa00b8b1e2f", "date": "Tue, 14 May 2024 04:18:58 GMT"} })
20212024-05-14T04:18:58.879ZINFOnexus: assigned collector to new producer collector_id = 90873ca0-52f3-4dd5-a08c-9897fa8723c5 file = nexus/src/app/oximeter.rs:125 producer_id = 81352410-feca-4767-8735-ac7a16a7c095
20222024-05-14T04:18:58.881ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 76992 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:36620 req_id = 8c790928-d369-4095-b2c8-203beeef5220 response_code = 201 uri = /metrics/producers
20232024-05-14T04:18:58.899ZINFOnexus (BackgroundTasks): instance and vmm updated by sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2112 instance_id = 81352410-feca-4767-8735-ac7a16a7c095 instance_updated = false propolis_id = 658dcf17-14c1-4107-80a5-1311c4e1c50a vmm_updated = false
20242024-05-14T04:18:58.899ZDEBGnexus (BackgroundTasks): update successful background_task = instance_watcher instance_id = 81352410-feca-4767-8735-ac7a16a7c095 instance_updated = false state = Starting vmm_updated = false
20252024-05-14T04:18:58.900ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1
20262024-05-14T04:18:58.900ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 4.375404206s iteration = 12
20272024-05-14T04:19:04.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
20282024-05-14T04:19:04.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20292024-05-14T04:19:04.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20302024-05-14T04:19:04.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20312024-05-14T04:19:04.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20322024-05-14T04:19:04.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20332024-05-14T04:19:04.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20342024-05-14T04:19:04.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20352024-05-14T04:19:04.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20362024-05-14T04:19:04.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20372024-05-14T04:19:04.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
20382024-05-14T04:19:04.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20392024-05-14T04:19:04.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20402024-05-14T04:19:04.535ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
20412024-05-14T04:19:04.535ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.549317ms iteration = 35
20422024-05-14T04:19:05.393ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 164 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = d92d03aa-eb3d-4e8e-b0c5-f613f22f7033 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
20432024-05-14T04:19:14.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
20442024-05-14T04:19:14.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20452024-05-14T04:19:14.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20462024-05-14T04:19:14.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20472024-05-14T04:19:14.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20482024-05-14T04:19:14.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20492024-05-14T04:19:14.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20502024-05-14T04:19:14.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20512024-05-14T04:19:14.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20522024-05-14T04:19:14.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20532024-05-14T04:19:14.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
20542024-05-14T04:19:14.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20552024-05-14T04:19:14.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20562024-05-14T04:19:14.534ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
20572024-05-14T04:19:14.534ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.72879ms iteration = 36
20582024-05-14T04:19:15.393ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 189 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 4f4ac6e3-10e0-4ac0-b1cd-a236a6a4a80b response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
20592024-05-14T04:19:24.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
20602024-05-14T04:19:24.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
20612024-05-14T04:19:24.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20622024-05-14T04:19:24.519ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
20632024-05-14T04:19:24.519ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20642024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
20652024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
20662024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
20672024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
20682024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
20692024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
20702024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20712024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20722024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
20732024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
20742024-05-14T04:19:24.520ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce
20752024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
20762024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 18.76\u{b5}s iteration = 8
20772024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
20782024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
20792024-05-14T04:19:24.520ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
20802024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.44\u{b5}s iteration = 7
20812024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
20822024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20832024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
20842024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
20852024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20862024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
20872024-05-14T04:19:24.520ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
20882024-05-14T04:19:24.520ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
20892024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.59\u{b5}s iteration = 13
20902024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
20912024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
20922024-05-14T04:19:24.520ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
20932024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
20942024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20952024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20962024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20972024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20982024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
20992024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
21002024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21012024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21022024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21032024-05-14T04:19:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21042024-05-14T04:19:24.520ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0dd5d513-ee28-4da3-83f2-7a3d7e3b267d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6aa90bd6-c3db-4dfa-88c0-97308b03c15f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9a8a4960-0835-4c5e-b472-b2521417e003.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9a8a4960-0835-4c5e-b472-b2521417e003.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6989, tv_nsec: 180930863 } })
21052024-05-14T04:19:24.520ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("08a9c078-20b8-4325-90f1-e360e31ca4ab.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("3f6ad63d-f2d7-4b36-a169-d3099a239267.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("3f6ad63d-f2d7-4b36-a169-d3099a239267.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 6989, tv_nsec: 180936023 } })
21062024-05-14T04:19:24.521ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
21072024-05-14T04:19:24.521ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.122113ms iteration = 7
21082024-05-14T04:19:24.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21092024-05-14T04:19:24.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21102024-05-14T04:19:24.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21112024-05-14T04:19:24.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21122024-05-14T04:19:24.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21132024-05-14T04:19:24.522ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21142024-05-14T04:19:24.522ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21152024-05-14T04:19:24.522ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
21162024-05-14T04:19:24.522ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.797617ms iteration = 13
21172024-05-14T04:19:24.523ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
21182024-05-14T04:19:24.523ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.104103ms iteration = 7
21192024-05-14T04:19:24.523ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21202024-05-14T04:19:24.523ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21212024-05-14T04:19:24.524ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6989, tv_nsec: 184501656 } })
21222024-05-14T04:19:24.524ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6989, tv_nsec: 184704280 } })
21232024-05-14T04:19:24.525ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
21242024-05-14T04:19:24.525ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
21252024-05-14T04:19:24.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
21262024-05-14T04:19:24.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
21272024-05-14T04:19:24.525ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
21282024-05-14T04:19:24.525ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21292024-05-14T04:19:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
21302024-05-14T04:19:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21312024-05-14T04:19:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21322024-05-14T04:19:24.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21332024-05-14T04:19:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
21342024-05-14T04:19:24.526ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21352024-05-14T04:19:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21362024-05-14T04:19:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21372024-05-14T04:19:24.526ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21382024-05-14T04:19:24.526ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21392024-05-14T04:19:24.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21402024-05-14T04:19:24.527ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21412024-05-14T04:19:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21422024-05-14T04:19:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21432024-05-14T04:19:24.527ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21442024-05-14T04:19:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
21452024-05-14T04:19:24.528ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21462024-05-14T04:19:24.528ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6989, tv_nsec: 188537668 } })
21472024-05-14T04:19:24.528ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
21482024-05-14T04:19:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21492024-05-14T04:19:24.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21502024-05-14T04:19:24.528ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21512024-05-14T04:19:24.529ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
21522024-05-14T04:19:24.529ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21532024-05-14T04:19:24.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21542024-05-14T04:19:24.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
21552024-05-14T04:19:24.530ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
21562024-05-14T04:19:24.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
21572024-05-14T04:19:24.531ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21582024-05-14T04:19:24.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
21592024-05-14T04:19:24.531ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21602024-05-14T04:19:24.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21612024-05-14T04:19:24.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
21622024-05-14T04:19:24.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21632024-05-14T04:19:24.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21642024-05-14T04:19:24.533ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
21652024-05-14T04:19:24.533ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
21662024-05-14T04:19:24.533ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
21672024-05-14T04:19:24.533ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21682024-05-14T04:19:24.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21692024-05-14T04:19:24.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
21702024-05-14T04:19:24.534ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21712024-05-14T04:19:24.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21722024-05-14T04:19:24.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21732024-05-14T04:19:24.534ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
21742024-05-14T04:19:24.534ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
21752024-05-14T04:19:24.534ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
21762024-05-14T04:19:24.534ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21772024-05-14T04:19:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21782024-05-14T04:19:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
21792024-05-14T04:19:24.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21802024-05-14T04:19:24.535ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
21812024-05-14T04:19:24.535ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 10.201608ms iteration = 13
21822024-05-14T04:19:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21832024-05-14T04:19:24.535ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21842024-05-14T04:19:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
21852024-05-14T04:19:24.535ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21862024-05-14T04:19:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21872024-05-14T04:19:24.535ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21882024-05-14T04:19:24.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21892024-05-14T04:19:24.536ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21902024-05-14T04:19:24.537ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21912024-05-14T04:19:24.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
21922024-05-14T04:19:24.539ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21932024-05-14T04:19:24.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21942024-05-14T04:19:24.539ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
21952024-05-14T04:19:24.539ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21962024-05-14T04:19:24.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21972024-05-14T04:19:24.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21982024-05-14T04:19:24.540ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21992024-05-14T04:19:24.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22002024-05-14T04:19:24.540ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
22012024-05-14T04:19:24.540ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.975637ms iteration = 37
22022024-05-14T04:19:24.541ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
22032024-05-14T04:19:24.541ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = f07c863b-7137-4303-9021-5a5c6e9ed458 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
22042024-05-14T04:19:24.541ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22052024-05-14T04:19:24.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22062024-05-14T04:19:24.541ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
22072024-05-14T04:19:24.541ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 4ac98832-1cdd-4449-9949-cf5fe247d837 dns_zone_name = oxide.test found_so_far = 0 version = 2
22082024-05-14T04:19:24.541ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22092024-05-14T04:19:24.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22102024-05-14T04:19:24.542ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22112024-05-14T04:19:24.543ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22122024-05-14T04:19:24.545ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22132024-05-14T04:19:24.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22142024-05-14T04:19:24.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22152024-05-14T04:19:24.546ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22162024-05-14T04:19:24.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22172024-05-14T04:19:24.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22182024-05-14T04:19:24.546ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22192024-05-14T04:19:24.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22202024-05-14T04:19:24.548ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 4ac98832-1cdd-4449-9949-cf5fe247d837 dns_zone_name = oxide.test found_so_far = 1 version = 2
22212024-05-14T04:19:24.548ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
22222024-05-14T04:19:24.548ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-14 04:13:29.424952 UTC dns_group = external generation = 2
22232024-05-14T04:19:24.548ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22242024-05-14T04:19:24.548ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 28.403389ms iteration = 7
22252024-05-14T04:19:24.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22262024-05-14T04:19:24.549ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = f07c863b-7137-4303-9021-5a5c6e9ed458 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
22272024-05-14T04:19:24.549ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
22282024-05-14T04:19:24.549ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-14 04:13:28.927466 UTC dns_group = internal generation = 1
22292024-05-14T04:19:24.549ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 29.036561ms iteration = 7
22302024-05-14T04:19:24.549ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22312024-05-14T04:19:24.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, lookup_type: ById(5a4ac577-a6fd-4dea-aa82-fc399ad2b82d) } result = Ok(())
22322024-05-14T04:19:24.552ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22332024-05-14T04:19:24.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22342024-05-14T04:19:24.552ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-14T04:13:24.616159Z, time_modified: 2024-05-14T04:13:24.616159Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: b4d2f397-0b2e-4db5-8e33-d295c802134a, db_silo: Silo { identity: SiloIdentity { id: b4d2f397-0b2e-4db5-8e33-d295c802134a, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-14T04:13:29.381647Z, time_modified: 2024-05-14T04:13:29.381647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "52301c060909182b4e1b505a925f9bb8cc086c123ca178be7538af43a1ecc18d" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: b4d2f397-0b2e-4db5-8e33-d295c802134a, db_silo: Silo { identity: SiloIdentity { id: b4d2f397-0b2e-4db5-8e33-d295c802134a, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-14T04:13:29.381647Z, time_modified: 2024-05-14T04:13:29.381647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "52301c060909182b4e1b505a925f9bb8cc086c123ca178be7538af43a1ecc18d" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
22352024-05-14T04:19:24.552ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 32.648435ms iteration = 7
22362024-05-14T04:19:24.553ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 33.670596ms iteration = 13
22372024-05-14T04:19:24.563ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
22382024-05-14T04:19:24.563ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 43.934235ms iteration = 14
22392024-05-14T04:19:24.645ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
22402024-05-14T04:19:24.657ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
22412024-05-14T04:19:24.661ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 zone_address = fd00:1122:3344:101::2
22422024-05-14T04:19:24.661ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22432024-05-14T04:19:24.662ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ff6c5805-9c96-4ab4-bdde-c47041117c05", "content-length": "26", "date": "Tue, 14 May 2024 04:19:24 GMT"} })
22442024-05-14T04:19:24.662ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22452024-05-14T04:19:24.664ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
22462024-05-14T04:19:24.670ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
22472024-05-14T04:19:24.670ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22482024-05-14T04:19:24.670ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3c29c9d8-842f-4723-841d-4d3b0af4907b", "content-length": "26", "date": "Tue, 14 May 2024 04:19:24 GMT"} })
22492024-05-14T04:19:24.670ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22502024-05-14T04:19:24.672ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
22512024-05-14T04:19:24.672ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22522024-05-14T04:19:24.672ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "53952f3c-a863-4267-9f78-1fb604771fbf", "content-length": "26", "date": "Tue, 14 May 2024 04:19:24 GMT"} })
22532024-05-14T04:19:24.672ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22542024-05-14T04:19:24.672ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22552024-05-14T04:19:24.673ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22562024-05-14T04:19:24.810ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
22572024-05-14T04:19:24.813ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
22582024-05-14T04:19:24.814ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e6b212f8-658f-495e-b153-6a7c3e563dbf", "content-length": "2", "date": "Tue, 14 May 2024 04:19:24 GMT"} })
22592024-05-14T04:19:24.815ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
22602024-05-14T04:19:24.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
22612024-05-14T04:19:24.817ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 297.392909ms iteration = 13
22622024-05-14T04:19:24.818ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 298.114933ms iteration = 13
22632024-05-14T04:19:24.871ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
22642024-05-14T04:19:24.871ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7ed30bd2-9350-47c8-ab2c-f90868225f37", "date": "Tue, 14 May 2024 04:19:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22652024-05-14T04:19:24.871ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22662024-05-14T04:19:24.873ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "79f82089-49c0-4905-be76-89a3c4fc5e4e", "date": "Tue, 14 May 2024 04:19:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22672024-05-14T04:19:24.873ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22682024-05-14T04:19:24.874ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22692024-05-14T04:19:24.874ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 354.609743ms iteration = 9
22702024-05-14T04:19:24.898ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
22712024-05-14T04:19:24.898ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8ff7e7b9-8bf7-4371-b96d-4224d2febb9c", "date": "Tue, 14 May 2024 04:19:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
22722024-05-14T04:19:24.898ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
22732024-05-14T04:19:24.900ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "383d0a7b-55c3-4ad7-8c05-e18c602d0ae4", "date": "Tue, 14 May 2024 04:19:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
22742024-05-14T04:19:24.900ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
22752024-05-14T04:19:24.901ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "32795c4b-12d3-44f3-8935-cbc5d91ef83a", "date": "Tue, 14 May 2024 04:19:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
22762024-05-14T04:19:24.901ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
22772024-05-14T04:19:24.903ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
22782024-05-14T04:19:24.903ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 382.982081ms iteration = 9
22792024-05-14T04:19:24.951ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22802024-05-14T04:19:24.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22812024-05-14T04:19:24.953ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: 384a5237-ec35-4842-bb57-a29c998a8bde, rack_id: b73a9888-3baa-4f16-9152-a3e1ba42c461, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(854ac53f-66c7-4bae-88ab-8ddb0746c0c4) } rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
22822024-05-14T04:19:24.953ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22832024-05-14T04:19:24.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22842024-05-14T04:19:24.960ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
22852024-05-14T04:19:24.960ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9f7373df-a7fb-4472-92cd-37fc2d4e999e", "content-length": "121", "date": "Tue, 14 May 2024 04:19:24 GMT"} })
22862024-05-14T04:19:24.960ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
22872024-05-14T04:19:24.960ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 854ac53f-66c7-4bae-88ab-8ddb0746c0c4
22882024-05-14T04:19:24.960ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
22892024-05-14T04:19:24.960ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
22902024-05-14T04:19:24.960ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
22912024-05-14T04:19:24.960ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
22922024-05-14T04:19:24.960ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
22932024-05-14T04:19:24.960ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
22942024-05-14T04:19:24.960ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
22952024-05-14T04:19:24.960ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22962024-05-14T04:19:24.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22972024-05-14T04:19:24.962ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
22982024-05-14T04:19:24.962ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "90c286eb-cb7a-45b5-8a01-e0ad71a6cd00", "content-length": "2", "date": "Tue, 14 May 2024 04:19:24 GMT"} })
22992024-05-14T04:19:24.962ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
23002024-05-14T04:19:24.962ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "656c05f8-05a4-459c-9311-09d7f479d2da", "content-length": "51", "date": "Tue, 14 May 2024 04:19:24 GMT"} })
23012024-05-14T04:19:24.962ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
23022024-05-14T04:19:24.962ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
23032024-05-14T04:19:25.010ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e4613765-ed8a-44d6-82c0-266ebe3d93f6", "date": "Tue, 14 May 2024 04:19:24 GMT"} })
23042024-05-14T04:19:25.010ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch = Switch0
23052024-05-14T04:19:25.010ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
23062024-05-14T04:19:25.010ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1a92333f-ad7c-4ee7-a01d-8ed72db18591", "content-length": "467", "date": "Tue, 14 May 2024 04:19:25 GMT"} })
23072024-05-14T04:19:25.010ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23082024-05-14T04:19:25.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23092024-05-14T04:19:25.013ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23102024-05-14T04:19:25.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23112024-05-14T04:19:25.016ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23122024-05-14T04:19:25.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23132024-05-14T04:19:25.017ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23142024-05-14T04:19:25.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23152024-05-14T04:19:25.023ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 503.954175ms iteration = 13
23162024-05-14T04:19:25.393ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 673 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 82bd89e6-c65c-4d9f-8471-1d2c75d62c13 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
23172024-05-14T04:19:25.396ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
23182024-05-14T04:19:25.397ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
23192024-05-14T04:19:25.426ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
23202024-05-14T04:19:25.426ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:37460
23212024-05-14T04:19:25.427ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:37460 req_id = 96bd3e2a-f88a-4b02-9601-11ff03f31c38 roles = RoleSet { roles: {} } uri = /metrics/producers
23222024-05-14T04:19:25.427ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:37460 req_id = 96bd3e2a-f88a-4b02-9601-11ff03f31c38 resource = Database result = Ok(()) uri = /metrics/producers
23232024-05-14T04:19:25.458ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 90873ca0-52f3-4dd5-a08c-9897fa8723c5
23242024-05-14T04:19:25.458ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:37460 req_id = 96bd3e2a-f88a-4b02-9601-11ff03f31c38 roles = RoleSet { roles: {} } uri = /metrics/producers
23252024-05-14T04:19:25.459ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:37460 req_id = 96bd3e2a-f88a-4b02-9601-11ff03f31c38 resource = Database result = Ok(()) uri = /metrics/producers
23262024-05-14T04:19:25.490ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 90873ca0-52f3-4dd5-a08c-9897fa8723c5 uri = http://[fd00:1122:3344:101::d]:12223/producers
23272024-05-14T04:19:25.490ZDEBGnexus: client response oximeter-collector = 90873ca0-52f3-4dd5-a08c-9897fa8723c5 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "75282174-691d-4d99-aa8b-f3c6daf6616e", "date": "Tue, 14 May 2024 04:19:25 GMT"} })
23282024-05-14T04:19:25.490ZINFOnexus: assigned collector to new producer collector_id = 90873ca0-52f3-4dd5-a08c-9897fa8723c5 file = nexus/src/app/oximeter.rs:125 producer_id = ef1c9f55-f835-41e6-8d3a-81e26da276c7
23292024-05-14T04:19:25.491ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 64708 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:37460 req_id = 96bd3e2a-f88a-4b02-9601-11ff03f31c38 response_code = 201 uri = /metrics/producers
23302024-05-14T04:19:25.491ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::c)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "96bd3e2a-f88a-4b02-9601-11ff03f31c38", "content-length": "41", "date": "Tue, 14 May 2024 04:19:25 GMT"} })
23312024-05-14T04:19:25.493ZDEBGnexus (producer-registration-task): registered with nexus successfully
23322024-05-14T04:19:25.493ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
23332024-05-14T04:19:34.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
23342024-05-14T04:19:34.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23352024-05-14T04:19:34.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23362024-05-14T04:19:34.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23372024-05-14T04:19:34.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23382024-05-14T04:19:34.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23392024-05-14T04:19:34.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23402024-05-14T04:19:34.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23412024-05-14T04:19:34.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23422024-05-14T04:19:34.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23432024-05-14T04:19:34.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
23442024-05-14T04:19:34.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23452024-05-14T04:19:34.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23462024-05-14T04:19:34.536ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
23472024-05-14T04:19:34.536ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.744151ms iteration = 38
23482024-05-14T04:19:35.491ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 135 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 6dac6026-7ddd-4054-a456-98326e96a16c response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
23492024-05-14T04:19:44.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
23502024-05-14T04:19:44.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23512024-05-14T04:19:44.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23522024-05-14T04:19:44.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23532024-05-14T04:19:44.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23542024-05-14T04:19:44.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23552024-05-14T04:19:44.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23562024-05-14T04:19:44.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23572024-05-14T04:19:44.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23582024-05-14T04:19:44.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23592024-05-14T04:19:44.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
23602024-05-14T04:19:44.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23612024-05-14T04:19:44.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23622024-05-14T04:19:44.538ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
23632024-05-14T04:19:44.538ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.547118ms iteration = 39
23642024-05-14T04:19:45.491ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 139 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 038b104d-8ea6-4693-85c5-65385f559674 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
23652024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
23662024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
23672024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23682024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23692024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
23702024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
23712024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
23722024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23732024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
23742024-05-14T04:19:54.520ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
23752024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
23762024-05-14T04:19:54.520ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
23772024-05-14T04:19:54.520ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
23782024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
23792024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.1\u{b5}s iteration = 14
23802024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
23812024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23822024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
23832024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23842024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
23852024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
23862024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23872024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23882024-05-14T04:19:54.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23892024-05-14T04:19:54.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23902024-05-14T04:19:54.522ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23912024-05-14T04:19:54.522ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23922024-05-14T04:19:54.522ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23932024-05-14T04:19:54.522ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
23942024-05-14T04:19:54.522ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.342637ms iteration = 14
23952024-05-14T04:19:54.523ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 7019, tv_nsec: 183229220 } })
23962024-05-14T04:19:54.523ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 7019, tv_nsec: 183557636 } })
23972024-05-14T04:19:54.524ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
23982024-05-14T04:19:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
23992024-05-14T04:19:54.525ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24002024-05-14T04:19:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24012024-05-14T04:19:54.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
24022024-05-14T04:19:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24032024-05-14T04:19:54.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24042024-05-14T04:19:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
24052024-05-14T04:19:54.525ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24062024-05-14T04:19:54.525ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
24072024-05-14T04:19:54.525ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
24082024-05-14T04:19:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24092024-05-14T04:19:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24102024-05-14T04:19:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
24112024-05-14T04:19:54.525ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24122024-05-14T04:19:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
24132024-05-14T04:19:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24142024-05-14T04:19:54.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24152024-05-14T04:19:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24162024-05-14T04:19:54.526ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
24172024-05-14T04:19:54.526ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24182024-05-14T04:19:54.526ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24192024-05-14T04:19:54.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24202024-05-14T04:19:54.527ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 7019, tv_nsec: 187795873 } })
24212024-05-14T04:19:54.527ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24222024-05-14T04:19:54.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24232024-05-14T04:19:54.528ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24242024-05-14T04:19:54.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24252024-05-14T04:19:54.528ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
24262024-05-14T04:19:54.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
24272024-05-14T04:19:54.528ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
24282024-05-14T04:19:54.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
24292024-05-14T04:19:54.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
24302024-05-14T04:19:54.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24312024-05-14T04:19:54.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24322024-05-14T04:19:54.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
24332024-05-14T04:19:54.531ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24342024-05-14T04:19:54.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24352024-05-14T04:19:54.536ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
24362024-05-14T04:19:54.536ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.997586ms iteration = 40
24372024-05-14T04:19:54.539ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24382024-05-14T04:19:54.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24392024-05-14T04:19:54.541ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24402024-05-14T04:19:54.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24412024-05-14T04:19:54.542ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24422024-05-14T04:19:54.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, lookup_type: ById(5a4ac577-a6fd-4dea-aa82-fc399ad2b82d) } result = Ok(())
24432024-05-14T04:19:54.545ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24442024-05-14T04:19:54.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24452024-05-14T04:19:54.547ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.645993ms iteration = 14
24462024-05-14T04:19:54.564ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
24472024-05-14T04:19:54.564ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 43.999176ms iteration = 15
24482024-05-14T04:19:54.581ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
24492024-05-14T04:19:54.581ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 55.535511ms iteration = 14
24502024-05-14T04:19:54.625ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
24512024-05-14T04:19:54.625ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24522024-05-14T04:19:54.626ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fa65ffdd-7a00-41ae-9038-41b777a9fa81", "content-length": "26", "date": "Tue, 14 May 2024 04:19:54 GMT"} })
24532024-05-14T04:19:54.626ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24542024-05-14T04:19:54.628ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
24552024-05-14T04:19:54.714ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
24562024-05-14T04:19:54.714ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24572024-05-14T04:19:54.714ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "54171d4f-d5bf-4344-a385-6462d7bbb64b", "content-length": "26", "date": "Tue, 14 May 2024 04:19:54 GMT"} })
24582024-05-14T04:19:54.714ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24592024-05-14T04:19:54.716ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
24602024-05-14T04:19:54.720ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
24612024-05-14T04:19:54.720ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cdab352f-890c-449d-a9c7-a925780c3fb1", "content-length": "2", "date": "Tue, 14 May 2024 04:19:54 GMT"} })
24622024-05-14T04:19:54.723ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
24632024-05-14T04:19:54.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
24642024-05-14T04:19:54.723ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 zone_address = fd00:1122:3344:101::2
24652024-05-14T04:19:54.723ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24662024-05-14T04:19:54.725ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 204.720561ms iteration = 14
24672024-05-14T04:19:54.725ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0f6b7846-02c0-4ef8-aa44-10330167baea", "content-length": "26", "date": "Tue, 14 May 2024 04:19:54 GMT"} })
24682024-05-14T04:19:54.725ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24692024-05-14T04:19:54.727ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
24702024-05-14T04:19:54.782ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 261.817284ms iteration = 14
24712024-05-14T04:19:54.853ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24722024-05-14T04:19:54.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24732024-05-14T04:19:54.855ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: 384a5237-ec35-4842-bb57-a29c998a8bde, rack_id: b73a9888-3baa-4f16-9152-a3e1ba42c461, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(854ac53f-66c7-4bae-88ab-8ddb0746c0c4) } rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
24742024-05-14T04:19:54.855ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24752024-05-14T04:19:54.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24762024-05-14T04:19:54.862ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
24772024-05-14T04:19:54.862ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "006908c4-1717-4884-9240-c6a17aa8f5cc", "content-length": "121", "date": "Tue, 14 May 2024 04:19:54 GMT"} })
24782024-05-14T04:19:54.862ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
24792024-05-14T04:19:54.862ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 854ac53f-66c7-4bae-88ab-8ddb0746c0c4
24802024-05-14T04:19:54.863ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
24812024-05-14T04:19:54.863ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
24822024-05-14T04:19:54.863ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
24832024-05-14T04:19:54.863ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
24842024-05-14T04:19:54.863ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
24852024-05-14T04:19:54.863ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
24862024-05-14T04:19:54.863ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
24872024-05-14T04:19:54.863ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24882024-05-14T04:19:54.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24892024-05-14T04:19:54.865ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
24902024-05-14T04:19:54.866ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9cf75ddf-7c4e-4da5-bf3c-c1a69d647ae3", "content-length": "2", "date": "Tue, 14 May 2024 04:19:54 GMT"} })
24912024-05-14T04:19:54.866ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
24922024-05-14T04:19:54.866ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "40622232-559e-4de5-81d8-abd890fdca1b", "content-length": "51", "date": "Tue, 14 May 2024 04:19:54 GMT"} })
24932024-05-14T04:19:54.866ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
24942024-05-14T04:19:54.866ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
24952024-05-14T04:19:54.911ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "32f04d92-a551-4d6f-b109-4fdaa06247aa", "date": "Tue, 14 May 2024 04:19:54 GMT"} })
24962024-05-14T04:19:54.911ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch = Switch0
24972024-05-14T04:19:54.911ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
24982024-05-14T04:19:54.911ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ab8b913c-1fef-425f-818f-305a5d6beeda", "content-length": "467", "date": "Tue, 14 May 2024 04:19:54 GMT"} })
24992024-05-14T04:19:54.911ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25002024-05-14T04:19:54.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25012024-05-14T04:19:54.914ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25022024-05-14T04:19:54.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25032024-05-14T04:19:54.916ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25042024-05-14T04:19:54.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25052024-05-14T04:19:54.918ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25062024-05-14T04:19:54.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25072024-05-14T04:19:54.925ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 404.888558ms iteration = 14
25082024-05-14T04:19:55.491ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 288 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 99d2d17a-dcab-4495-af6c-91c6ef2683fa response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
25092024-05-14T04:20:04.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
25102024-05-14T04:20:04.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25112024-05-14T04:20:04.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25122024-05-14T04:20:04.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25132024-05-14T04:20:04.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25142024-05-14T04:20:04.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25152024-05-14T04:20:04.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25162024-05-14T04:20:04.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25172024-05-14T04:20:04.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25182024-05-14T04:20:04.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25192024-05-14T04:20:04.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
25202024-05-14T04:20:04.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25212024-05-14T04:20:04.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25222024-05-14T04:20:04.534ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
25232024-05-14T04:20:04.535ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.340442ms iteration = 41
25242024-05-14T04:20:05.491ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 259 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = ed8b4027-f7eb-481c-a2fc-198958805bb3 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
25252024-05-14T04:20:14.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
25262024-05-14T04:20:14.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25272024-05-14T04:20:14.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25282024-05-14T04:20:14.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25292024-05-14T04:20:14.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25302024-05-14T04:20:14.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25312024-05-14T04:20:14.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25322024-05-14T04:20:14.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25332024-05-14T04:20:14.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25342024-05-14T04:20:14.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25352024-05-14T04:20:14.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
25362024-05-14T04:20:14.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25372024-05-14T04:20:14.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25382024-05-14T04:20:14.536ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
25392024-05-14T04:20:14.536ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.126639ms iteration = 42
25402024-05-14T04:20:15.492ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 953 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 77f70331-41c2-4552-912a-f0e55cf1f8ec response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
25412024-05-14T04:20:24.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
25422024-05-14T04:20:24.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
25432024-05-14T04:20:24.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25442024-05-14T04:20:24.519ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
25452024-05-14T04:20:24.519ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25462024-05-14T04:20:24.519ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
25472024-05-14T04:20:24.519ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
25482024-05-14T04:20:24.519ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
25492024-05-14T04:20:24.519ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
25502024-05-14T04:20:24.519ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
25512024-05-14T04:20:24.519ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
25522024-05-14T04:20:24.519ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25532024-05-14T04:20:24.519ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25542024-05-14T04:20:24.519ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
25552024-05-14T04:20:24.519ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
25562024-05-14T04:20:24.519ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
25572024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.98\u{b5}s iteration = 15
25582024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
25592024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
25602024-05-14T04:20:24.520ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
25612024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
25622024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
25632024-05-14T04:20:24.520ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
25642024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.94\u{b5}s iteration = 8
25652024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
25662024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
25672024-05-14T04:20:24.520ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce
25682024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.46\u{b5}s iteration = 9
25692024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
25702024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
25712024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
25722024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25732024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
25742024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
25752024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25762024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
25772024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
25782024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
25792024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25802024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25812024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25822024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25832024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25842024-05-14T04:20:24.520ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("08a9c078-20b8-4325-90f1-e360e31ca4ab.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("3f6ad63d-f2d7-4b36-a169-d3099a239267.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("3f6ad63d-f2d7-4b36-a169-d3099a239267.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 7049, tv_nsec: 180490044 } })
25852024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25862024-05-14T04:20:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
25872024-05-14T04:20:24.520ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0dd5d513-ee28-4da3-83f2-7a3d7e3b267d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6aa90bd6-c3db-4dfa-88c0-97308b03c15f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9a8a4960-0835-4c5e-b472-b2521417e003.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9a8a4960-0835-4c5e-b472-b2521417e003.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 7049, tv_nsec: 180588496 } })
25882024-05-14T04:20:24.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25892024-05-14T04:20:24.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25902024-05-14T04:20:24.522ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25912024-05-14T04:20:24.522ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25922024-05-14T04:20:24.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25932024-05-14T04:20:24.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
25942024-05-14T04:20:24.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.839858ms iteration = 8
25952024-05-14T04:20:24.522ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
25962024-05-14T04:20:24.522ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.92353ms iteration = 15
25972024-05-14T04:20:24.523ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25982024-05-14T04:20:24.523ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25992024-05-14T04:20:24.523ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26002024-05-14T04:20:24.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26012024-05-14T04:20:24.523ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26022024-05-14T04:20:24.523ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
26032024-05-14T04:20:24.523ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.005321ms iteration = 8
26042024-05-14T04:20:24.523ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 7049, tv_nsec: 183958405 } })
26052024-05-14T04:20:24.524ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26062024-05-14T04:20:24.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
26072024-05-14T04:20:24.525ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
26082024-05-14T04:20:24.525ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26092024-05-14T04:20:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26102024-05-14T04:20:24.526ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
26112024-05-14T04:20:24.526ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
26122024-05-14T04:20:24.526ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 7049, tv_nsec: 186269462 } })
26132024-05-14T04:20:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
26142024-05-14T04:20:24.526ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
26152024-05-14T04:20:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
26162024-05-14T04:20:24.527ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26172024-05-14T04:20:24.527ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26182024-05-14T04:20:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26192024-05-14T04:20:24.527ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26202024-05-14T04:20:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
26212024-05-14T04:20:24.527ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26222024-05-14T04:20:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26232024-05-14T04:20:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26242024-05-14T04:20:24.528ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
26252024-05-14T04:20:24.528ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26262024-05-14T04:20:24.529ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26272024-05-14T04:20:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
26282024-05-14T04:20:24.529ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26292024-05-14T04:20:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26302024-05-14T04:20:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26312024-05-14T04:20:24.529ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 7049, tv_nsec: 189763713 } })
26322024-05-14T04:20:24.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
26332024-05-14T04:20:24.530ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26342024-05-14T04:20:24.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26352024-05-14T04:20:24.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26362024-05-14T04:20:24.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26372024-05-14T04:20:24.530ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
26382024-05-14T04:20:24.530ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
26392024-05-14T04:20:24.530ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
26402024-05-14T04:20:24.530ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26412024-05-14T04:20:24.531ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26422024-05-14T04:20:24.531ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
26432024-05-14T04:20:24.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26442024-05-14T04:20:24.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
26452024-05-14T04:20:24.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26462024-05-14T04:20:24.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26472024-05-14T04:20:24.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26482024-05-14T04:20:24.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
26492024-05-14T04:20:24.532ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
26502024-05-14T04:20:24.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
26512024-05-14T04:20:24.532ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26522024-05-14T04:20:24.532ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
26532024-05-14T04:20:24.532ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
26542024-05-14T04:20:24.532ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
26552024-05-14T04:20:24.532ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26562024-05-14T04:20:24.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26572024-05-14T04:20:24.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26582024-05-14T04:20:24.533ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26592024-05-14T04:20:24.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
26602024-05-14T04:20:24.534ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26612024-05-14T04:20:24.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26622024-05-14T04:20:24.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
26632024-05-14T04:20:24.534ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26642024-05-14T04:20:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26652024-05-14T04:20:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
26662024-05-14T04:20:24.535ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26672024-05-14T04:20:24.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26682024-05-14T04:20:24.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
26692024-05-14T04:20:24.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26702024-05-14T04:20:24.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26712024-05-14T04:20:24.536ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26722024-05-14T04:20:24.536ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
26732024-05-14T04:20:24.537ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 10.869292ms iteration = 15
26742024-05-14T04:20:24.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26752024-05-14T04:20:24.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
26762024-05-14T04:20:24.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26772024-05-14T04:20:24.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26782024-05-14T04:20:24.537ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
26792024-05-14T04:20:24.537ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = f07c863b-7137-4303-9021-5a5c6e9ed458 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
26802024-05-14T04:20:24.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26812024-05-14T04:20:24.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26822024-05-14T04:20:24.538ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26832024-05-14T04:20:24.538ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
26842024-05-14T04:20:24.538ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 4ac98832-1cdd-4449-9949-cf5fe247d837 dns_zone_name = oxide.test found_so_far = 0 version = 2
26852024-05-14T04:20:24.538ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26862024-05-14T04:20:24.538ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26872024-05-14T04:20:24.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26882024-05-14T04:20:24.539ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
26892024-05-14T04:20:24.539ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 19.480647ms iteration = 16
26902024-05-14T04:20:24.539ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
26912024-05-14T04:20:24.539ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26922024-05-14T04:20:24.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26932024-05-14T04:20:24.540ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26942024-05-14T04:20:24.542ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
26952024-05-14T04:20:24.542ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.515099ms iteration = 43
26962024-05-14T04:20:24.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
26972024-05-14T04:20:24.542ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26982024-05-14T04:20:24.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26992024-05-14T04:20:24.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
27002024-05-14T04:20:24.543ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
27012024-05-14T04:20:24.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
27022024-05-14T04:20:24.544ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = f07c863b-7137-4303-9021-5a5c6e9ed458 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
27032024-05-14T04:20:24.544ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
27042024-05-14T04:20:24.544ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-14 04:13:28.927466 UTC dns_group = internal generation = 1
27052024-05-14T04:20:24.544ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 24.963759ms iteration = 8
27062024-05-14T04:20:24.545ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 4ac98832-1cdd-4449-9949-cf5fe247d837 dns_zone_name = oxide.test found_so_far = 1 version = 2
27072024-05-14T04:20:24.545ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
27082024-05-14T04:20:24.545ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-14 04:13:29.424952 UTC dns_group = external generation = 2
27092024-05-14T04:20:24.545ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 25.380277ms iteration = 8
27102024-05-14T04:20:24.548ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27112024-05-14T04:20:24.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27122024-05-14T04:20:24.549ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-14T04:13:24.616159Z, time_modified: 2024-05-14T04:13:24.616159Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: b4d2f397-0b2e-4db5-8e33-d295c802134a, db_silo: Silo { identity: SiloIdentity { id: b4d2f397-0b2e-4db5-8e33-d295c802134a, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-14T04:13:29.381647Z, time_modified: 2024-05-14T04:13:29.381647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "52301c060909182b4e1b505a925f9bb8cc086c123ca178be7538af43a1ecc18d" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: b4d2f397-0b2e-4db5-8e33-d295c802134a, db_silo: Silo { identity: SiloIdentity { id: b4d2f397-0b2e-4db5-8e33-d295c802134a, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-14T04:13:29.381647Z, time_modified: 2024-05-14T04:13:29.381647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "52301c060909182b4e1b505a925f9bb8cc086c123ca178be7538af43a1ecc18d" }] }) }
27132024-05-14T04:20:24.549ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 29.167234ms iteration = 8
27142024-05-14T04:20:24.549ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27152024-05-14T04:20:24.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27162024-05-14T04:20:24.551ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27172024-05-14T04:20:24.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, lookup_type: ById(5a4ac577-a6fd-4dea-aa82-fc399ad2b82d) } result = Ok(())
27182024-05-14T04:20:24.554ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27192024-05-14T04:20:24.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27202024-05-14T04:20:24.555ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 36.174957ms iteration = 15
27212024-05-14T04:20:24.663ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
27222024-05-14T04:20:24.665ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
27232024-05-14T04:20:24.679ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 zone_address = fd00:1122:3344:101::2
27242024-05-14T04:20:24.679ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27252024-05-14T04:20:24.679ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e1052c15-db29-44d5-8583-db0a9c5f1bfd", "content-length": "26", "date": "Tue, 14 May 2024 04:20:24 GMT"} })
27262024-05-14T04:20:24.679ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27272024-05-14T04:20:24.680ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
27282024-05-14T04:20:24.807ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
27292024-05-14T04:20:24.807ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27302024-05-14T04:20:24.808ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ae7affed-fe34-43b8-9790-e67e99dc4362", "content-length": "26", "date": "Tue, 14 May 2024 04:20:24 GMT"} })
27312024-05-14T04:20:24.808ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27322024-05-14T04:20:24.810ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
27332024-05-14T04:20:24.812ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
27342024-05-14T04:20:24.812ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
27352024-05-14T04:20:24.812ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27362024-05-14T04:20:24.813ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "579030f9-cf20-4393-80cf-0356cf69011a", "content-length": "26", "date": "Tue, 14 May 2024 04:20:24 GMT"} })
27372024-05-14T04:20:24.813ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27382024-05-14T04:20:24.813ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d2d16023-10de-426a-b0e4-fb24681d8040", "date": "Tue, 14 May 2024 04:20:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
27392024-05-14T04:20:24.813ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
27402024-05-14T04:20:24.816ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
27412024-05-14T04:20:24.816ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ca410f4d-2e4e-49ae-a34f-4a0098f1defb", "date": "Tue, 14 May 2024 04:20:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
27422024-05-14T04:20:24.816ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
27432024-05-14T04:20:24.816ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
27442024-05-14T04:20:24.817ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
27452024-05-14T04:20:24.817ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 298.165164ms iteration = 10
27462024-05-14T04:20:24.921ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 401.57341ms iteration = 15
27472024-05-14T04:20:24.923ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
27482024-05-14T04:20:24.923ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2e3606ce-b4c0-4e29-a968-8d3884850151", "content-length": "2", "date": "Tue, 14 May 2024 04:20:24 GMT"} })
27492024-05-14T04:20:24.924ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
27502024-05-14T04:20:24.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
27512024-05-14T04:20:24.926ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 406.444249ms iteration = 15
27522024-05-14T04:20:24.926ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
27532024-05-14T04:20:24.926ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "08fb76bb-b715-4034-8191-ae2af65e90f4", "date": "Tue, 14 May 2024 04:20:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
27542024-05-14T04:20:24.927ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
27552024-05-14T04:20:24.929ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d316e9c7-fc08-43bc-9917-54fb1c7c5f16", "date": "Tue, 14 May 2024 04:20:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
27562024-05-14T04:20:24.929ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
27572024-05-14T04:20:24.931ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1553a05d-7730-4fd1-8978-ffa87dc0fc42", "date": "Tue, 14 May 2024 04:20:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
27582024-05-14T04:20:24.931ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
27592024-05-14T04:20:24.933ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
27602024-05-14T04:20:24.933ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 413.36111ms iteration = 10
27612024-05-14T04:20:24.956ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27622024-05-14T04:20:24.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27632024-05-14T04:20:24.957ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: 384a5237-ec35-4842-bb57-a29c998a8bde, rack_id: b73a9888-3baa-4f16-9152-a3e1ba42c461, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(854ac53f-66c7-4bae-88ab-8ddb0746c0c4) } rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
27642024-05-14T04:20:24.957ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27652024-05-14T04:20:24.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27662024-05-14T04:20:24.964ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
27672024-05-14T04:20:24.964ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "33c92b57-cd59-4328-8365-7ddedf7f3f12", "content-length": "121", "date": "Tue, 14 May 2024 04:20:24 GMT"} })
27682024-05-14T04:20:24.964ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
27692024-05-14T04:20:24.964ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 854ac53f-66c7-4bae-88ab-8ddb0746c0c4
27702024-05-14T04:20:24.964ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
27712024-05-14T04:20:24.964ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
27722024-05-14T04:20:24.964ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
27732024-05-14T04:20:24.964ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
27742024-05-14T04:20:24.964ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
27752024-05-14T04:20:24.964ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
27762024-05-14T04:20:24.964ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
27772024-05-14T04:20:24.964ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27782024-05-14T04:20:24.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27792024-05-14T04:20:24.966ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
27802024-05-14T04:20:24.967ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f891dad8-1922-491f-9b08-ae68db567ac9", "content-length": "2", "date": "Tue, 14 May 2024 04:20:24 GMT"} })
27812024-05-14T04:20:24.967ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
27822024-05-14T04:20:24.967ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e48a86ec-d27f-45df-b696-05951436e1e2", "content-length": "51", "date": "Tue, 14 May 2024 04:20:24 GMT"} })
27832024-05-14T04:20:24.967ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
27842024-05-14T04:20:24.967ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
27852024-05-14T04:20:25.017ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "17e1577e-66fe-4c45-96d6-a443fb0911aa", "date": "Tue, 14 May 2024 04:20:24 GMT"} })
27862024-05-14T04:20:25.017ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch = Switch0
27872024-05-14T04:20:25.017ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
27882024-05-14T04:20:25.018ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7de139b8-3e0b-481a-8280-ef0892625f64", "content-length": "467", "date": "Tue, 14 May 2024 04:20:24 GMT"} })
27892024-05-14T04:20:25.018ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27902024-05-14T04:20:25.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27912024-05-14T04:20:25.020ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27922024-05-14T04:20:25.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27932024-05-14T04:20:25.022ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27942024-05-14T04:20:25.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27952024-05-14T04:20:25.024ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27962024-05-14T04:20:25.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27972024-05-14T04:20:25.029ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 510.121841ms iteration = 15
27982024-05-14T04:20:25.491ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 243 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 5ee3c762-f153-4919-95b5-c91c69bf78de response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
27992024-05-14T04:20:34.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
28002024-05-14T04:20:34.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28012024-05-14T04:20:34.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28022024-05-14T04:20:34.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28032024-05-14T04:20:34.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28042024-05-14T04:20:34.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28052024-05-14T04:20:34.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28062024-05-14T04:20:34.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28072024-05-14T04:20:34.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28082024-05-14T04:20:34.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28092024-05-14T04:20:34.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
28102024-05-14T04:20:34.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28112024-05-14T04:20:34.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28122024-05-14T04:20:34.534ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
28132024-05-14T04:20:34.534ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.069886ms iteration = 44
28142024-05-14T04:20:35.491ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 239 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 1dde23a2-42c8-451b-ad08-7d78d73a69df response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
28152024-05-14T04:20:44.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
28162024-05-14T04:20:44.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28172024-05-14T04:20:44.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28182024-05-14T04:20:44.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28192024-05-14T04:20:44.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28202024-05-14T04:20:44.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28212024-05-14T04:20:44.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28222024-05-14T04:20:44.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28232024-05-14T04:20:44.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28242024-05-14T04:20:44.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28252024-05-14T04:20:44.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
28262024-05-14T04:20:44.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28272024-05-14T04:20:44.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28282024-05-14T04:20:44.534ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
28292024-05-14T04:20:44.534ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.808241ms iteration = 45
28302024-05-14T04:20:45.491ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 140 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 985d2c5e-1323-4e21-ab10-15200e6b6115 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
28312024-05-14T04:20:54.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
28322024-05-14T04:20:54.519ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
28332024-05-14T04:20:54.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28342024-05-14T04:20:54.519ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
28352024-05-14T04:20:54.519ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
28362024-05-14T04:20:54.519ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28372024-05-14T04:20:54.519ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
28382024-05-14T04:20:54.519ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
28392024-05-14T04:20:54.519ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
28402024-05-14T04:20:54.519ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
28412024-05-14T04:20:54.519ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 4.93\u{b5}s iteration = 16
28422024-05-14T04:20:54.519ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
28432024-05-14T04:20:54.519ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28442024-05-14T04:20:54.520ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
28452024-05-14T04:20:54.520ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
28462024-05-14T04:20:54.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
28472024-05-14T04:20:54.520ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
28482024-05-14T04:20:54.520ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28492024-05-14T04:20:54.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
28502024-05-14T04:20:54.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28512024-05-14T04:20:54.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28522024-05-14T04:20:54.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
28532024-05-14T04:20:54.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28542024-05-14T04:20:54.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28552024-05-14T04:20:54.521ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28562024-05-14T04:20:54.521ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 7079, tv_nsec: 181681098 } })
28572024-05-14T04:20:54.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28582024-05-14T04:20:54.522ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28592024-05-14T04:20:54.522ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28602024-05-14T04:20:54.522ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
28612024-05-14T04:20:54.522ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.725606ms iteration = 16
28622024-05-14T04:20:54.522ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
28632024-05-14T04:20:54.522ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 7079, tv_nsec: 182871273 } })
28642024-05-14T04:20:54.523ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
28652024-05-14T04:20:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
28662024-05-14T04:20:54.524ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28672024-05-14T04:20:54.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28682024-05-14T04:20:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28692024-05-14T04:20:54.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28702024-05-14T04:20:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28712024-05-14T04:20:54.526ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
28722024-05-14T04:20:54.526ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
28732024-05-14T04:20:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
28742024-05-14T04:20:54.526ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28752024-05-14T04:20:54.526ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 7079, tv_nsec: 186431295 } })
28762024-05-14T04:20:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
28772024-05-14T04:20:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28782024-05-14T04:20:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
28792024-05-14T04:20:54.526ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28802024-05-14T04:20:54.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28812024-05-14T04:20:54.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28822024-05-14T04:20:54.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28832024-05-14T04:20:54.527ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
28842024-05-14T04:20:54.527ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28852024-05-14T04:20:54.527ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
28862024-05-14T04:20:54.528ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28872024-05-14T04:20:54.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28882024-05-14T04:20:54.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28892024-05-14T04:20:54.528ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28902024-05-14T04:20:54.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28912024-05-14T04:20:54.529ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28922024-05-14T04:20:54.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
28932024-05-14T04:20:54.530ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
28942024-05-14T04:20:54.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
28952024-05-14T04:20:54.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
28962024-05-14T04:20:54.531ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28972024-05-14T04:20:54.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28982024-05-14T04:20:54.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
28992024-05-14T04:20:54.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29002024-05-14T04:20:54.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29012024-05-14T04:20:54.534ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
29022024-05-14T04:20:54.534ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 14.430474ms iteration = 17
29032024-05-14T04:20:54.535ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
29042024-05-14T04:20:54.535ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 9.504473ms iteration = 16
29052024-05-14T04:20:54.537ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
29062024-05-14T04:20:54.537ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.704781ms iteration = 46
29072024-05-14T04:20:54.540ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29082024-05-14T04:20:54.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29092024-05-14T04:20:54.542ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29102024-05-14T04:20:54.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29112024-05-14T04:20:54.543ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29122024-05-14T04:20:54.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, lookup_type: ById(5a4ac577-a6fd-4dea-aa82-fc399ad2b82d) } result = Ok(())
29132024-05-14T04:20:54.546ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29142024-05-14T04:20:54.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29152024-05-14T04:20:54.547ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 27.557621ms iteration = 16
29162024-05-14T04:20:54.570ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
29172024-05-14T04:20:54.570ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29182024-05-14T04:20:54.570ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
29192024-05-14T04:20:54.571ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29202024-05-14T04:20:54.571ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2a81f505-213a-4342-bc54-9e271038ff88", "content-length": "26", "date": "Tue, 14 May 2024 04:20:54 GMT"} })
29212024-05-14T04:20:54.571ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29222024-05-14T04:20:54.571ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f0ccda34-cf1f-4fa5-87ae-5847f962ec58", "content-length": "26", "date": "Tue, 14 May 2024 04:20:54 GMT"} })
29232024-05-14T04:20:54.571ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29242024-05-14T04:20:54.572ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 zone_address = fd00:1122:3344:101::2
29252024-05-14T04:20:54.572ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29262024-05-14T04:20:54.572ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29272024-05-14T04:20:54.572ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1bd7c11e-e6db-4be1-897a-32e013ed6f02", "content-length": "26", "date": "Tue, 14 May 2024 04:20:54 GMT"} })
29282024-05-14T04:20:54.572ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29292024-05-14T04:20:54.574ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29302024-05-14T04:20:54.574ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
29312024-05-14T04:20:54.642ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
29322024-05-14T04:20:54.643ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "424c4742-3974-4737-8691-a2b58045e61d", "content-length": "2", "date": "Tue, 14 May 2024 04:20:53 GMT"} })
29332024-05-14T04:20:54.645ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
29342024-05-14T04:20:54.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
29352024-05-14T04:20:54.646ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 127.018847ms iteration = 16
29362024-05-14T04:20:54.659ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 139.41901ms iteration = 16
29372024-05-14T04:20:54.709ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29382024-05-14T04:20:54.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29392024-05-14T04:20:54.711ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: 384a5237-ec35-4842-bb57-a29c998a8bde, rack_id: b73a9888-3baa-4f16-9152-a3e1ba42c461, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(854ac53f-66c7-4bae-88ab-8ddb0746c0c4) } rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
29402024-05-14T04:20:54.711ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29412024-05-14T04:20:54.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29422024-05-14T04:20:54.717ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
29432024-05-14T04:20:54.718ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ddb2321c-33d4-4458-97cf-d2c48d47297f", "content-length": "121", "date": "Tue, 14 May 2024 04:20:53 GMT"} })
29442024-05-14T04:20:54.718ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
29452024-05-14T04:20:54.718ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 854ac53f-66c7-4bae-88ab-8ddb0746c0c4
29462024-05-14T04:20:54.718ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
29472024-05-14T04:20:54.718ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
29482024-05-14T04:20:54.718ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
29492024-05-14T04:20:54.718ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
29502024-05-14T04:20:54.718ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
29512024-05-14T04:20:54.718ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
29522024-05-14T04:20:54.718ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
29532024-05-14T04:20:54.718ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29542024-05-14T04:20:54.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29552024-05-14T04:20:54.719ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
29562024-05-14T04:20:54.720ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "39f2b076-d0bb-44c0-9588-ee380680f4e5", "content-length": "2", "date": "Tue, 14 May 2024 04:20:53 GMT"} })
29572024-05-14T04:20:54.720ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
29582024-05-14T04:20:54.720ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c2094ca7-cfdc-43cf-823b-959618c20d6c", "content-length": "51", "date": "Tue, 14 May 2024 04:20:53 GMT"} })
29592024-05-14T04:20:54.720ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
29602024-05-14T04:20:54.720ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
29612024-05-14T04:20:54.799ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e231d15c-2059-48a6-bcd1-76b58e9bf4e6", "date": "Tue, 14 May 2024 04:20:54 GMT"} })
29622024-05-14T04:20:54.799ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch = Switch0
29632024-05-14T04:20:54.799ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
29642024-05-14T04:20:54.800ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bea503c2-3e4f-45ff-b4c2-fd51fa180f99", "content-length": "467", "date": "Tue, 14 May 2024 04:20:54 GMT"} })
29652024-05-14T04:20:54.800ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29662024-05-14T04:20:54.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29672024-05-14T04:20:54.802ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29682024-05-14T04:20:54.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29692024-05-14T04:20:54.804ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29702024-05-14T04:20:54.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29712024-05-14T04:20:54.806ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29722024-05-14T04:20:54.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29732024-05-14T04:20:54.812ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 292.453568ms iteration = 16
29742024-05-14T04:20:55.491ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 190 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = dc20454f-6283-4a37-adc6-972a98ebb900 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
29752024-05-14T04:21:04.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
29762024-05-14T04:21:04.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29772024-05-14T04:21:04.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29782024-05-14T04:21:04.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29792024-05-14T04:21:04.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29802024-05-14T04:21:04.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29812024-05-14T04:21:04.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29822024-05-14T04:21:04.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29832024-05-14T04:21:04.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29842024-05-14T04:21:04.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29852024-05-14T04:21:04.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
29862024-05-14T04:21:04.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29872024-05-14T04:21:04.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29882024-05-14T04:21:04.537ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
29892024-05-14T04:21:04.537ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.359954ms iteration = 47
29902024-05-14T04:21:05.491ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 150 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 2656524b-acb0-4851-9504-4090048af296 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
29912024-05-14T04:21:14.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
29922024-05-14T04:21:14.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29932024-05-14T04:21:14.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29942024-05-14T04:21:14.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29952024-05-14T04:21:14.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29962024-05-14T04:21:14.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29972024-05-14T04:21:14.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29982024-05-14T04:21:14.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29992024-05-14T04:21:14.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30002024-05-14T04:21:14.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30012024-05-14T04:21:14.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
30022024-05-14T04:21:14.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30032024-05-14T04:21:14.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30042024-05-14T04:21:14.534ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
30052024-05-14T04:21:14.534ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.919854ms iteration = 48
30062024-05-14T04:21:15.491ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 239 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 80e866b7-f15a-4f82-974a-e5635009fea1 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
30072024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
30082024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
30092024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30102024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
30112024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
30122024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
30132024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30142024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
30152024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
30162024-05-14T04:21:24.520ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce
30172024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30182024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.54\u{b5}s iteration = 10
30192024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
30202024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
30212024-05-14T04:21:24.520ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
30222024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.52\u{b5}s iteration = 9
30232024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
30242024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30252024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
30262024-05-14T04:21:24.520ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
30272024-05-14T04:21:24.520ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
30282024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
30292024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.271\u{b5}s iteration = 17
30302024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
30312024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
30322024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
30332024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
30342024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30352024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
30362024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
30372024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
30382024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
30392024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30402024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30412024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
30422024-05-14T04:21:24.520ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
30432024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30442024-05-14T04:21:24.520ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0dd5d513-ee28-4da3-83f2-7a3d7e3b267d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6aa90bd6-c3db-4dfa-88c0-97308b03c15f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9a8a4960-0835-4c5e-b472-b2521417e003.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("9a8a4960-0835-4c5e-b472-b2521417e003.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 7109, tv_nsec: 180371662 } })
30452024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30462024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30472024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30482024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
30492024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
30502024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30512024-05-14T04:21:24.520ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("08a9c078-20b8-4325-90f1-e360e31ca4ab.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("3f6ad63d-f2d7-4b36-a169-d3099a239267.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("3f6ad63d-f2d7-4b36-a169-d3099a239267.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 7109, tv_nsec: 180484854 } })
30522024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30532024-05-14T04:21:24.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30542024-05-14T04:21:24.522ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30552024-05-14T04:21:24.522ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30562024-05-14T04:21:24.522ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30572024-05-14T04:21:24.522ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30582024-05-14T04:21:24.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30592024-05-14T04:21:24.522ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30602024-05-14T04:21:24.523ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30612024-05-14T04:21:24.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30622024-05-14T04:21:24.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30632024-05-14T04:21:24.524ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
30642024-05-14T04:21:24.524ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.96695ms iteration = 9
30652024-05-14T04:21:24.524ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
30662024-05-14T04:21:24.524ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.890659ms iteration = 17
30672024-05-14T04:21:24.524ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
30682024-05-14T04:21:24.524ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.120574ms iteration = 9
30692024-05-14T04:21:24.524ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 7109, tv_nsec: 183942914 } })
30702024-05-14T04:21:24.524ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30712024-05-14T04:21:24.524ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 7109, tv_nsec: 184003396 } })
30722024-05-14T04:21:24.524ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30732024-05-14T04:21:24.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
30742024-05-14T04:21:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
30752024-05-14T04:21:24.525ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30762024-05-14T04:21:24.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30772024-05-14T04:21:24.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
30782024-05-14T04:21:24.525ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
30792024-05-14T04:21:24.525ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
30802024-05-14T04:21:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
30812024-05-14T04:21:24.526ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
30822024-05-14T04:21:24.526ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30832024-05-14T04:21:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30842024-05-14T04:21:24.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
30852024-05-14T04:21:24.526ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30862024-05-14T04:21:24.526ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
30872024-05-14T04:21:24.526ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30882024-05-14T04:21:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
30892024-05-14T04:21:24.527ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30902024-05-14T04:21:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30912024-05-14T04:21:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30922024-05-14T04:21:24.527ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30932024-05-14T04:21:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30942024-05-14T04:21:24.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30952024-05-14T04:21:24.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30962024-05-14T04:21:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30972024-05-14T04:21:24.528ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30982024-05-14T04:21:24.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30992024-05-14T04:21:24.528ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31002024-05-14T04:21:24.528ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31012024-05-14T04:21:24.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31022024-05-14T04:21:24.529ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 7109, tv_nsec: 188704302 } })
31032024-05-14T04:21:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31042024-05-14T04:21:24.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31052024-05-14T04:21:24.529ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31062024-05-14T04:21:24.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31072024-05-14T04:21:24.530ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
31082024-05-14T04:21:24.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31092024-05-14T04:21:24.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
31102024-05-14T04:21:24.530ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
31112024-05-14T04:21:24.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
31122024-05-14T04:21:24.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31132024-05-14T04:21:24.531ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
31142024-05-14T04:21:24.531ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
31152024-05-14T04:21:24.531ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
31162024-05-14T04:21:24.531ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31172024-05-14T04:21:24.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31182024-05-14T04:21:24.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
31192024-05-14T04:21:24.531ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31202024-05-14T04:21:24.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31212024-05-14T04:21:24.532ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
31222024-05-14T04:21:24.532ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
31232024-05-14T04:21:24.532ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
31242024-05-14T04:21:24.532ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31252024-05-14T04:21:24.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31262024-05-14T04:21:24.532ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31272024-05-14T04:21:24.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
31282024-05-14T04:21:24.533ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31292024-05-14T04:21:24.533ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31302024-05-14T04:21:24.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31312024-05-14T04:21:24.534ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
31322024-05-14T04:21:24.534ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 9.031794ms iteration = 17
31332024-05-14T04:21:24.535ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31342024-05-14T04:21:24.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31352024-05-14T04:21:24.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31362024-05-14T04:21:24.536ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31372024-05-14T04:21:24.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31382024-05-14T04:21:24.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
31392024-05-14T04:21:24.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31402024-05-14T04:21:24.536ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31412024-05-14T04:21:24.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31422024-05-14T04:21:24.536ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
31432024-05-14T04:21:24.536ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 16.572787ms iteration = 18
31442024-05-14T04:21:24.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31452024-05-14T04:21:24.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31462024-05-14T04:21:24.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31472024-05-14T04:21:24.537ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
31482024-05-14T04:21:24.537ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = f07c863b-7137-4303-9021-5a5c6e9ed458 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
31492024-05-14T04:21:24.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31502024-05-14T04:21:24.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31512024-05-14T04:21:24.538ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
31522024-05-14T04:21:24.538ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31532024-05-14T04:21:24.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31542024-05-14T04:21:24.539ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31552024-05-14T04:21:24.539ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
31562024-05-14T04:21:24.539ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 4ac98832-1cdd-4449-9949-cf5fe247d837 dns_zone_name = oxide.test found_so_far = 0 version = 2
31572024-05-14T04:21:24.539ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31582024-05-14T04:21:24.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31592024-05-14T04:21:24.540ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31602024-05-14T04:21:24.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31612024-05-14T04:21:24.541ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31622024-05-14T04:21:24.542ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
31632024-05-14T04:21:24.542ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.997758ms iteration = 49
31642024-05-14T04:21:24.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31652024-05-14T04:21:24.543ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31662024-05-14T04:21:24.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31672024-05-14T04:21:24.543ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = f07c863b-7137-4303-9021-5a5c6e9ed458 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
31682024-05-14T04:21:24.543ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
31692024-05-14T04:21:24.543ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-14 04:13:28.927466 UTC dns_group = internal generation = 1
31702024-05-14T04:21:24.544ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 23.788095ms iteration = 9
31712024-05-14T04:21:24.544ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31722024-05-14T04:21:24.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31732024-05-14T04:21:24.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31742024-05-14T04:21:24.545ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31752024-05-14T04:21:24.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31762024-05-14T04:21:24.546ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-14T04:13:24.616159Z, time_modified: 2024-05-14T04:13:24.616159Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: b4d2f397-0b2e-4db5-8e33-d295c802134a, db_silo: Silo { identity: SiloIdentity { id: b4d2f397-0b2e-4db5-8e33-d295c802134a, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-14T04:13:29.381647Z, time_modified: 2024-05-14T04:13:29.381647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "52301c060909182b4e1b505a925f9bb8cc086c123ca178be7538af43a1ecc18d" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: b4d2f397-0b2e-4db5-8e33-d295c802134a, db_silo: Silo { identity: SiloIdentity { id: b4d2f397-0b2e-4db5-8e33-d295c802134a, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-14T04:13:29.381647Z, time_modified: 2024-05-14T04:13:29.381647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "52301c060909182b4e1b505a925f9bb8cc086c123ca178be7538af43a1ecc18d" }] }) }
31772024-05-14T04:21:24.546ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 25.987109ms iteration = 9
31782024-05-14T04:21:24.546ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31792024-05-14T04:21:24.546ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 4ac98832-1cdd-4449-9949-cf5fe247d837 dns_zone_name = oxide.test found_so_far = 1 version = 2
31802024-05-14T04:21:24.546ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
31812024-05-14T04:21:24.546ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-14 04:13:29.424952 UTC dns_group = external generation = 2
31822024-05-14T04:21:24.546ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 26.775116ms iteration = 9
31832024-05-14T04:21:24.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, lookup_type: ById(5a4ac577-a6fd-4dea-aa82-fc399ad2b82d) } result = Ok(())
31842024-05-14T04:21:24.548ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31852024-05-14T04:21:24.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31862024-05-14T04:21:24.549ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 29.638933ms iteration = 17
31872024-05-14T04:21:24.656ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
31882024-05-14T04:21:24.668ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
31892024-05-14T04:21:24.674ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 zone_address = fd00:1122:3344:101::2
31902024-05-14T04:21:24.674ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
31912024-05-14T04:21:24.675ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3895773b-e0a6-4071-aab4-c838433c0381", "content-length": "26", "date": "Tue, 14 May 2024 04:21:24 GMT"} })
31922024-05-14T04:21:24.675ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
31932024-05-14T04:21:24.676ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
31942024-05-14T04:21:24.794ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
31952024-05-14T04:21:24.794ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8aa92a52-47b8-4bc4-ba52-cdd4b09998e1", "date": "Tue, 14 May 2024 04:21:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
31962024-05-14T04:21:24.794ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
31972024-05-14T04:21:24.796ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "40a0fdb7-67ec-44a6-bd14-0f1d37267c44", "date": "Tue, 14 May 2024 04:21:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
31982024-05-14T04:21:24.796ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
31992024-05-14T04:21:24.797ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32002024-05-14T04:21:24.797ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 277.913202ms iteration = 11
32012024-05-14T04:21:24.807ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
32022024-05-14T04:21:24.807ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32032024-05-14T04:21:24.808ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e48b88aa-0e1e-44ea-9e41-3aeebd3f92f2", "content-length": "26", "date": "Tue, 14 May 2024 04:21:24 GMT"} })
32042024-05-14T04:21:24.808ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32052024-05-14T04:21:24.810ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32062024-05-14T04:21:24.810ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
32072024-05-14T04:21:24.811ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
32082024-05-14T04:21:24.811ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32092024-05-14T04:21:24.812ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9fc127b9-78d0-4c16-b5c4-7f007cf6089a", "content-length": "26", "date": "Tue, 14 May 2024 04:21:24 GMT"} })
32102024-05-14T04:21:24.812ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32112024-05-14T04:21:24.814ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32122024-05-14T04:21:24.945ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
32132024-05-14T04:21:24.946ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7a131ced-19cc-4d06-832f-c73c7f50b5a0", "date": "Tue, 14 May 2024 04:21:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32142024-05-14T04:21:24.946ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32152024-05-14T04:21:24.947ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1d06a1f3-7dea-4a7e-99fe-6fb5569e5dc7", "date": "Tue, 14 May 2024 04:21:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32162024-05-14T04:21:24.947ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32172024-05-14T04:21:24.949ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9aef45dc-76b4-4e8d-90d3-65bd15d11a4a", "date": "Tue, 14 May 2024 04:21:24 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32182024-05-14T04:21:24.949ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32192024-05-14T04:21:24.951ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32202024-05-14T04:21:24.951ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 430.848746ms iteration = 11
32212024-05-14T04:21:25.035ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32222024-05-14T04:21:25.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32232024-05-14T04:21:25.037ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: 384a5237-ec35-4842-bb57-a29c998a8bde, rack_id: b73a9888-3baa-4f16-9152-a3e1ba42c461, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(854ac53f-66c7-4bae-88ab-8ddb0746c0c4) } rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
32242024-05-14T04:21:25.037ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32252024-05-14T04:21:25.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32262024-05-14T04:21:25.045ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
32272024-05-14T04:21:25.045ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
32282024-05-14T04:21:25.046ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5ee3b0f0-9065-4e6a-8f8a-80b54f26ed45", "content-length": "121", "date": "Tue, 14 May 2024 04:21:24 GMT"} })
32292024-05-14T04:21:25.046ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
32302024-05-14T04:21:25.046ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 854ac53f-66c7-4bae-88ab-8ddb0746c0c4
32312024-05-14T04:21:25.046ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7719e9eb-3cf3-4b4f-a0a7-4cb20324a275", "content-length": "2", "date": "Tue, 14 May 2024 04:21:24 GMT"} })
32322024-05-14T04:21:25.046ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
32332024-05-14T04:21:25.046ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
32342024-05-14T04:21:25.046ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
32352024-05-14T04:21:25.046ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
32362024-05-14T04:21:25.046ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
32372024-05-14T04:21:25.046ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
32382024-05-14T04:21:25.046ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
32392024-05-14T04:21:25.046ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32402024-05-14T04:21:25.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32412024-05-14T04:21:25.047ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 527.398033ms iteration = 17
32422024-05-14T04:21:25.048ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
32432024-05-14T04:21:25.048ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
32442024-05-14T04:21:25.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
32452024-05-14T04:21:25.048ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fa48f5cf-1099-4ae6-8c0c-3dc772f52b26", "content-length": "2", "date": "Tue, 14 May 2024 04:21:24 GMT"} })
32462024-05-14T04:21:25.048ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
32472024-05-14T04:21:25.048ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "19db4b5c-054b-450d-b388-9f7a918ed49b", "content-length": "51", "date": "Tue, 14 May 2024 04:21:24 GMT"} })
32482024-05-14T04:21:25.048ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
32492024-05-14T04:21:25.048ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
32502024-05-14T04:21:25.050ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 529.912854ms iteration = 17
32512024-05-14T04:21:25.093ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "46391e41-c473-4eb8-a828-5998c6406139", "date": "Tue, 14 May 2024 04:21:24 GMT"} })
32522024-05-14T04:21:25.093ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch = Switch0
32532024-05-14T04:21:25.093ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
32542024-05-14T04:21:25.094ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "30fcb3d9-9d64-444c-8048-acf510015b20", "content-length": "467", "date": "Tue, 14 May 2024 04:21:24 GMT"} })
32552024-05-14T04:21:25.094ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32562024-05-14T04:21:25.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32572024-05-14T04:21:25.096ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32582024-05-14T04:21:25.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32592024-05-14T04:21:25.099ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32602024-05-14T04:21:25.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32612024-05-14T04:21:25.100ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32622024-05-14T04:21:25.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32632024-05-14T04:21:25.106ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 586.615179ms iteration = 17
32642024-05-14T04:21:25.491ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 247 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 988a8476-1bf1-4b1c-828f-7d30fb32c800 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
32652024-05-14T04:21:34.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
32662024-05-14T04:21:34.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32672024-05-14T04:21:34.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32682024-05-14T04:21:34.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32692024-05-14T04:21:34.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
32702024-05-14T04:21:34.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32712024-05-14T04:21:34.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32722024-05-14T04:21:34.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32732024-05-14T04:21:34.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32742024-05-14T04:21:34.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32752024-05-14T04:21:34.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
32762024-05-14T04:21:34.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32772024-05-14T04:21:34.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32782024-05-14T04:21:34.534ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
32792024-05-14T04:21:34.534ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.778901ms iteration = 50
32802024-05-14T04:21:35.492ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 168 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = d9ae301a-2bbf-4e25-890b-9adc8112fa10 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
32812024-05-14T04:21:44.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
32822024-05-14T04:21:44.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32832024-05-14T04:21:44.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32842024-05-14T04:21:44.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32852024-05-14T04:21:44.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
32862024-05-14T04:21:44.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32872024-05-14T04:21:44.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32882024-05-14T04:21:44.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32892024-05-14T04:21:44.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32902024-05-14T04:21:44.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32912024-05-14T04:21:44.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
32922024-05-14T04:21:44.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32932024-05-14T04:21:44.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32942024-05-14T04:21:44.535ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
32952024-05-14T04:21:44.535ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.474035ms iteration = 51
32962024-05-14T04:21:45.493ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 1158 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = 8eed95d6-36b4-48dc-b8af-16385c5a1bf8 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
32972024-05-14T04:21:54.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
32982024-05-14T04:21:54.520ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
32992024-05-14T04:21:54.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33002024-05-14T04:21:54.520ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33012024-05-14T04:21:54.520ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
33022024-05-14T04:21:54.520ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
33032024-05-14T04:21:54.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
33042024-05-14T04:21:54.520ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
33052024-05-14T04:21:54.520ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33062024-05-14T04:21:54.520ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
33072024-05-14T04:21:54.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
33082024-05-14T04:21:54.520ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
33092024-05-14T04:21:54.520ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
33102024-05-14T04:21:54.520ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
33112024-05-14T04:21:54.520ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
33122024-05-14T04:21:54.520ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
33132024-05-14T04:21:54.520ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 4.9\u{b5}s iteration = 18
33142024-05-14T04:21:54.520ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33152024-05-14T04:21:54.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33162024-05-14T04:21:54.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33172024-05-14T04:21:54.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
33182024-05-14T04:21:54.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
33192024-05-14T04:21:54.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33202024-05-14T04:21:54.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
33212024-05-14T04:21:54.522ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33222024-05-14T04:21:54.522ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33232024-05-14T04:21:54.523ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 7139, tv_nsec: 182584506 } })
33242024-05-14T04:21:54.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33252024-05-14T04:21:54.523ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
33262024-05-14T04:21:54.523ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.526901ms iteration = 18
33272024-05-14T04:21:54.523ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33282024-05-14T04:21:54.524ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
33292024-05-14T04:21:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
33302024-05-14T04:21:54.525ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33312024-05-14T04:21:54.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
33322024-05-14T04:21:54.525ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 7139, tv_nsec: 185290512 } })
33332024-05-14T04:21:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
33342024-05-14T04:21:54.526ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33352024-05-14T04:21:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33362024-05-14T04:21:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
33372024-05-14T04:21:54.526ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33382024-05-14T04:21:54.526ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
33392024-05-14T04:21:54.526ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
33402024-05-14T04:21:54.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33412024-05-14T04:21:54.526ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
33422024-05-14T04:21:54.527ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
33432024-05-14T04:21:54.527ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33442024-05-14T04:21:54.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
33452024-05-14T04:21:54.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
33462024-05-14T04:21:54.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33472024-05-14T04:21:54.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33482024-05-14T04:21:54.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33492024-05-14T04:21:54.527ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33502024-05-14T04:21:54.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33512024-05-14T04:21:54.528ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33522024-05-14T04:21:54.528ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33532024-05-14T04:21:54.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33542024-05-14T04:21:54.528ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-5a4ac577-a6fd-4dea-aa82-fc399ad2b82d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 7139, tv_nsec: 187917025 } })
33552024-05-14T04:21:54.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33562024-05-14T04:21:54.529ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
33572024-05-14T04:21:54.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33582024-05-14T04:21:54.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
33592024-05-14T04:21:54.531ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
33602024-05-14T04:21:54.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
33612024-05-14T04:21:54.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
33622024-05-14T04:21:54.531ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33632024-05-14T04:21:54.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
33642024-05-14T04:21:54.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bce37bb-4ca0-4afa-9eac-b87eae7491ce, lookup_type: ById(1bce37bb-4ca0-4afa-9eac-b87eae7491ce) } result = Ok(())
33652024-05-14T04:21:54.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33662024-05-14T04:21:54.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33672024-05-14T04:21:54.535ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
33682024-05-14T04:21:54.535ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 14.913534ms iteration = 19
33692024-05-14T04:21:54.537ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
33702024-05-14T04:21:54.537ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 10.242319ms iteration = 18
33712024-05-14T04:21:54.538ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce original_time_created = 2024-05-14 04:13:15.171099 UTC target_id = 1bce37bb-4ca0-4afa-9eac-b87eae7491ce time_created = 2024-05-14 04:13:15.171099 UTC
33722024-05-14T04:21:54.538ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.508406ms iteration = 52
33732024-05-14T04:21:54.543ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33742024-05-14T04:21:54.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33752024-05-14T04:21:54.545ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33762024-05-14T04:21:54.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33772024-05-14T04:21:54.546ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33782024-05-14T04:21:54.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 5a4ac577-a6fd-4dea-aa82-fc399ad2b82d, lookup_type: ById(5a4ac577-a6fd-4dea-aa82-fc399ad2b82d) } result = Ok(())
33792024-05-14T04:21:54.549ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33802024-05-14T04:21:54.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33812024-05-14T04:21:54.551ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 30.357249ms iteration = 18
33822024-05-14T04:21:54.568ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
33832024-05-14T04:21:54.568ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33842024-05-14T04:21:54.568ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d0309a91-95a2-47e6-bc69-32b329c4f454", "content-length": "26", "date": "Tue, 14 May 2024 04:21:54 GMT"} })
33852024-05-14T04:21:54.569ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33862024-05-14T04:21:54.571ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
33872024-05-14T04:21:54.575ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
33882024-05-14T04:21:54.575ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33892024-05-14T04:21:54.575ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f378acf6-3327-449e-a7f2-b625828fb190", "content-length": "26", "date": "Tue, 14 May 2024 04:21:54 GMT"} })
33902024-05-14T04:21:54.575ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33912024-05-14T04:21:54.577ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 zone_address = fd00:1122:3344:101::2
33922024-05-14T04:21:54.577ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33932024-05-14T04:21:54.577ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "73860bb0-2571-487c-a364-ebba96193ad3", "content-length": "26", "date": "Tue, 14 May 2024 04:21:54 GMT"} })
33942024-05-14T04:21:54.577ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33952024-05-14T04:21:54.578ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
33962024-05-14T04:21:54.578ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
33972024-05-14T04:21:54.678ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
33982024-05-14T04:21:54.680ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e2192866-b175-4f4f-ae3a-949b6fcfe490", "content-length": "2", "date": "Tue, 14 May 2024 04:21:53 GMT"} })
33992024-05-14T04:21:54.681ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
34002024-05-14T04:21:54.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
34012024-05-14T04:21:54.684ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 163.944179ms iteration = 18
34022024-05-14T04:21:54.763ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 243.070842ms iteration = 18
34032024-05-14T04:21:54.824ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34042024-05-14T04:21:54.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34052024-05-14T04:21:54.826ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: 384a5237-ec35-4842-bb57-a29c998a8bde, rack_id: b73a9888-3baa-4f16-9152-a3e1ba42c461, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(854ac53f-66c7-4bae-88ab-8ddb0746c0c4) } rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
34062024-05-14T04:21:54.826ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34072024-05-14T04:21:54.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34082024-05-14T04:21:54.838ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
34092024-05-14T04:21:54.841ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9c439c79-c28f-4185-b3da-863e8e976269", "content-length": "121", "date": "Tue, 14 May 2024 04:21:53 GMT"} })
34102024-05-14T04:21:54.841ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
34112024-05-14T04:21:54.841ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 854ac53f-66c7-4bae-88ab-8ddb0746c0c4
34122024-05-14T04:21:54.841ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
34132024-05-14T04:21:54.841ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
34142024-05-14T04:21:54.841ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
34152024-05-14T04:21:54.841ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
34162024-05-14T04:21:54.841ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
34172024-05-14T04:21:54.841ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 routes = {}
34182024-05-14T04:21:54.841ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461
34192024-05-14T04:21:54.841ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34202024-05-14T04:21:54.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34212024-05-14T04:21:54.843ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
34222024-05-14T04:21:54.844ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3a8ef5f6-4871-4253-9ac3-346505add6ed", "content-length": "2", "date": "Tue, 14 May 2024 04:21:53 GMT"} })
34232024-05-14T04:21:54.844ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
34242024-05-14T04:21:54.844ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "eb5e24f2-f585-45d8-bfeb-0c4bf9e1aa72", "content-length": "51", "date": "Tue, 14 May 2024 04:21:53 GMT"} })
34252024-05-14T04:21:54.844ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch_location = Switch0
34262024-05-14T04:21:54.844ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
34272024-05-14T04:21:54.891ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "88b20959-01d7-409d-aee7-0b664e1f6c24", "date": "Tue, 14 May 2024 04:21:54 GMT"} })
34282024-05-14T04:21:54.891ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 switch = Switch0
34292024-05-14T04:21:54.891ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
34302024-05-14T04:21:54.891ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b73a9888-3baa-4f16-9152-a3e1ba42c461 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e0f5f121-86f8-421e-9568-3754f828e382", "content-length": "467", "date": "Tue, 14 May 2024 04:21:54 GMT"} })
34312024-05-14T04:21:54.891ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34322024-05-14T04:21:54.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34332024-05-14T04:21:54.894ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34342024-05-14T04:21:54.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34352024-05-14T04:21:54.897ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34362024-05-14T04:21:54.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34372024-05-14T04:21:54.898ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34382024-05-14T04:21:54.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34392024-05-14T04:21:54.904ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 383.827549ms iteration = 18
34402024-05-14T04:21:55.492ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 148 local_addr = [fd00:1122:3344:101::c]:55243 method = GET remote_addr = [fd00:1122:3344:101::d]:65013 req_id = c56ed2d5-728e-41e8-9745-12e52e2d5c99 response_code = 200 uri = //ef1c9f55-f835-41e6-8d3a-81e26da276c7
34412024-05-14T04:21:55.495ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
34422024-05-14T04:21:55.495ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
34432024-05-14T04:21:55.525ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
34442024-05-14T04:21:55.525ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:34153
34452024-05-14T04:21:55.525ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34153 req_id = 88b63118-b269-48ac-b61c-9fed7336f6a2 roles = RoleSet { roles: {} } uri = /metrics/producers
34462024-05-14T04:21:55.526ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34153 req_id = 88b63118-b269-48ac-b61c-9fed7336f6a2 resource = Database result = Ok(()) uri = /metrics/producers
34472024-05-14T04:21:55.557ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 90873ca0-52f3-4dd5-a08c-9897fa8723c5
34482024-05-14T04:21:55.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34153 req_id = 88b63118-b269-48ac-b61c-9fed7336f6a2 roles = RoleSet { roles: {} } uri = /metrics/producers
34492024-05-14T04:21:55.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34153 req_id = 88b63118-b269-48ac-b61c-9fed7336f6a2 resource = Database result = Ok(()) uri = /metrics/producers
34502024-05-14T04:21:55.594ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 90873ca0-52f3-4dd5-a08c-9897fa8723c5 uri = http://[fd00:1122:3344:101::d]:12223/producers
34512024-05-14T04:21:55.594ZDEBGnexus: client response oximeter-collector = 90873ca0-52f3-4dd5-a08c-9897fa8723c5 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ec7f5176-45a9-4ed3-8f83-13c0a164fcd7", "date": "Tue, 14 May 2024 04:21:55 GMT"} })
34522024-05-14T04:21:55.594ZINFOnexus: assigned collector to new producer collector_id = 90873ca0-52f3-4dd5-a08c-9897fa8723c5 file = nexus/src/app/oximeter.rs:125 producer_id = ef1c9f55-f835-41e6-8d3a-81e26da276c7
34532024-05-14T04:21:55.596ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/283d897/dropshot/src/server.rs:878 latency_us = 70819 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34153 req_id = 88b63118-b269-48ac-b61c-9fed7336f6a2 response_code = 201 uri = /metrics/producers
34542024-05-14T04:21:55.596ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::c)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "88b63118-b269-48ac-b61c-9fed7336f6a2", "content-length": "41", "date": "Tue, 14 May 2024 04:21:55 GMT"} })
34552024-05-14T04:21:55.598ZDEBGnexus (producer-registration-task): registered with nexus successfully
34562024-05-14T04:21:55.598ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s