Max level shown:
12024-02-03T08:15:01.649ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:47767
22024-02-03T08:15:01.650ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 683 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = fcb8cdde-0c0b-41a1-b643-516a0982227e response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
32024-02-03T08:15:09.389ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 6 reason = Timeout
42024-02-03T08:15:09.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52024-02-03T08:15:09.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62024-02-03T08:15:09.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72024-02-03T08:15:09.393ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
82024-02-03T08:15:09.393ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92024-02-03T08:15:09.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
102024-02-03T08:15:09.395ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.259366ms iteration = 6
112024-02-03T08:15:11.619ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 843 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 8bc1020e-ee9f-45e5-bb26-93ceb0027dc1 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
122024-02-03T08:15:19.362ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 7 reason = Timeout
132024-02-03T08:15:19.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
142024-02-03T08:15:19.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
152024-02-03T08:15:19.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162024-02-03T08:15:19.366ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
172024-02-03T08:15:19.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
182024-02-03T08:15:19.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
192024-02-03T08:15:19.367ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.378657ms iteration = 7
202024-02-03T08:15:21.591ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 687 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 6c8aa95d-697a-4d0d-8398-489f67be9f27 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
212024-02-03T08:15:29.330ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
222024-02-03T08:15:29.330ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
232024-02-03T08:15:29.330ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
242024-02-03T08:15:29.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
252024-02-03T08:15:29.331ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
262024-02-03T08:15:29.331ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
272024-02-03T08:15:29.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
282024-02-03T08:15:29.331ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
292024-02-03T08:15:29.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
302024-02-03T08:15:29.332ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
312024-02-03T08:15:29.332ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
322024-02-03T08:15:29.333ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
332024-02-03T08:15:29.333ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
342024-02-03T08:15:29.333ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
352024-02-03T08:15:29.333ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
362024-02-03T08:15:29.333ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
372024-02-03T08:15:29.333ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
382024-02-03T08:15:29.333ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
392024-02-03T08:15:29.333ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
402024-02-03T08:15:29.334ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
412024-02-03T08:15:29.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
422024-02-03T08:15:29.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
432024-02-03T08:15:29.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
442024-02-03T08:15:29.334ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
452024-02-03T08:15:29.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
462024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 8 reason = Timeout
472024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 4 reason = Timeout
482024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
492024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 3 reason = Timeout
502024-02-03T08:15:29.335ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
512024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
522024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 5.58\u{b5}s iteration = 3
532024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 4 reason = Timeout
542024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 4 reason = Timeout
552024-02-03T08:15:29.335ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
562024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
572024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 4 reason = Timeout
582024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
592024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
602024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
612024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
622024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
632024-02-03T08:15:29.335ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
642024-02-03T08:15:29.336ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
652024-02-03T08:15:29.336ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
662024-02-03T08:15:29.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
672024-02-03T08:15:29.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
682024-02-03T08:15:29.337ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
692024-02-03T08:15:29.337ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
702024-02-03T08:15:29.337ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
712024-02-03T08:15:29.337ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
722024-02-03T08:15:29.337ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
732024-02-03T08:15:29.338ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
742024-02-03T08:15:29.338ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.986956ms iteration = 4
752024-02-03T08:15:29.338ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
762024-02-03T08:15:29.338ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ca86e6c8-eb93-4b53-bd4d-5f7d9b87ddb5", "content-length": "1", "date": "Sat, 03 Feb 2024 08:15:29 GMT"} })
772024-02-03T08:15:29.338ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
782024-02-03T08:15:29.338ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
792024-02-03T08:15:29.338ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
802024-02-03T08:15:29.338ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
812024-02-03T08:15:29.338ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
822024-02-03T08:15:29.338ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "45cb26e5-4142-4ff1-b059-2a8dfae02926", "content-length": "2", "date": "Sat, 03 Feb 2024 08:15:29 GMT"} })
832024-02-03T08:15:29.338ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 3.712102ms iteration = 4
842024-02-03T08:15:29.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
852024-02-03T08:15:29.338ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
862024-02-03T08:15:29.338ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 7.694356ms iteration = 4
872024-02-03T08:15:29.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
882024-02-03T08:15:29.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
892024-02-03T08:15:29.339ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
902024-02-03T08:15:29.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
912024-02-03T08:15:29.339ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
922024-02-03T08:15:29.340ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 5.118784ms iteration = 4
932024-02-03T08:15:29.340ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
942024-02-03T08:15:29.340ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 6.96548ms iteration = 4
952024-02-03T08:15:29.342ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
962024-02-03T08:15:29.342ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
972024-02-03T08:15:29.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
982024-02-03T08:15:29.342ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
992024-02-03T08:15:29.342ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1002024-02-03T08:15:29.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1012024-02-03T08:15:29.342ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1022024-02-03T08:15:29.343ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1032024-02-03T08:15:29.343ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1042024-02-03T08:15:29.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1052024-02-03T08:15:29.344ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1062024-02-03T08:15:29.344ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1072024-02-03T08:15:29.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1082024-02-03T08:15:29.347ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 12.869591ms iteration = 8
1092024-02-03T08:15:29.348ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1102024-02-03T08:15:29.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1112024-02-03T08:15:29.348ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
1122024-02-03T08:15:29.348ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
1132024-02-03T08:15:29.348ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
1142024-02-03T08:15:29.348ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1152024-02-03T08:15:29.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1162024-02-03T08:15:29.348ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
1172024-02-03T08:15:29.348ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
1182024-02-03T08:15:29.348ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1192024-02-03T08:15:29.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1202024-02-03T08:15:29.349ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1212024-02-03T08:15:29.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1222024-02-03T08:15:29.350ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1232024-02-03T08:15:29.350ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1242024-02-03T08:15:29.350ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1252024-02-03T08:15:29.353ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1262024-02-03T08:15:29.353ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1272024-02-03T08:15:29.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1282024-02-03T08:15:29.355ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
1292024-02-03T08:15:29.355ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 0 version = 2
1302024-02-03T08:15:29.355ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1312024-02-03T08:15:29.355ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1322024-02-03T08:15:29.355ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1332024-02-03T08:15:29.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1342024-02-03T08:15:29.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1352024-02-03T08:15:29.355ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
1362024-02-03T08:15:29.355ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1372024-02-03T08:15:29.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1382024-02-03T08:15:29.356ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1392024-02-03T08:15:29.357ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1402024-02-03T08:15:29.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1412024-02-03T08:15:29.357ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
1422024-02-03T08:15:29.357ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
1432024-02-03T08:15:29.357ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-03 08:14:50.307019 UTC dns_group = internal generation = 1
1442024-02-03T08:15:29.357ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 27.070773ms iteration = 4
1452024-02-03T08:15:29.358ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1462024-02-03T08:15:29.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1472024-02-03T08:15:29.360ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1482024-02-03T08:15:29.360ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1492024-02-03T08:15:29.360ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1502024-02-03T08:15:29.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1512024-02-03T08:15:29.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1522024-02-03T08:15:29.361ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 1 version = 2
1532024-02-03T08:15:29.361ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
1542024-02-03T08:15:29.361ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-03 08:14:50.600977 UTC dns_group = external generation = 2
1552024-02-03T08:15:29.361ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 27.583918ms iteration = 4
1562024-02-03T08:15:29.364ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1572024-02-03T08:15:29.365ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-03T08:14:29.510171Z, time_modified: 2024-02-03T08:14:29.510171Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(2)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(2)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
1582024-02-03T08:15:29.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1592024-02-03T08:15:29.365ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 31.37795ms iteration = 4
1602024-02-03T08:15:29.366ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1612024-02-03T08:15:29.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1622024-02-03T08:15:29.367ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1632024-02-03T08:15:29.369ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
1642024-02-03T08:15:29.369ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1652024-02-03T08:15:29.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1662024-02-03T08:15:29.370ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 35.94366ms iteration = 4
1672024-02-03T08:15:29.396ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
1682024-02-03T08:15:29.401ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
1692024-02-03T08:15:29.455ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
1702024-02-03T08:15:29.456ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b17bedea-c5a3-4ebb-a152-8000b0a51f1b", "date": "Sat, 03 Feb 2024 08:15:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
1712024-02-03T08:15:29.456ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
1722024-02-03T08:15:29.457ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3388ca12-007b-4fd9-a18a-d2ac3bc32bc0", "date": "Sat, 03 Feb 2024 08:15:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
1732024-02-03T08:15:29.457ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
1742024-02-03T08:15:29.458ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
1752024-02-03T08:15:29.458ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 125.064439ms iteration = 6
1762024-02-03T08:15:29.460ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
1772024-02-03T08:15:29.493ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
1782024-02-03T08:15:29.494ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2efaab30-5ae9-4404-8b11-a7e91ece5335", "date": "Sat, 03 Feb 2024 08:15:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1792024-02-03T08:15:29.494ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1802024-02-03T08:15:29.495ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a884fd83-16be-4875-b48d-7f620a06aeeb", "date": "Sat, 03 Feb 2024 08:15:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1812024-02-03T08:15:29.495ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1822024-02-03T08:15:29.496ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a77b99a6-be74-4506-84c0-756c272c75c8", "date": "Sat, 03 Feb 2024 08:15:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1832024-02-03T08:15:29.496ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1842024-02-03T08:15:29.497ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1852024-02-03T08:15:29.497ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 163.993504ms iteration = 6
1862024-02-03T08:15:31.563ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 503 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 62b6eb7c-8f9e-412d-954f-2a34e63ee7d6 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
1872024-02-03T08:15:39.306ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 9 reason = Timeout
1882024-02-03T08:15:39.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1892024-02-03T08:15:39.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1902024-02-03T08:15:39.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1912024-02-03T08:15:39.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1922024-02-03T08:15:39.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1932024-02-03T08:15:39.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1942024-02-03T08:15:39.311ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 4.708331ms iteration = 9
1952024-02-03T08:15:41.535ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 494 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 259da3d5-2760-4080-b45b-ecae5d0a9ab0 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
1962024-02-03T08:15:49.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 10 reason = Timeout
1972024-02-03T08:15:49.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1982024-02-03T08:15:49.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1992024-02-03T08:15:49.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2002024-02-03T08:15:49.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2012024-02-03T08:15:49.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2022024-02-03T08:15:49.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2032024-02-03T08:15:49.283ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.233965ms iteration = 10
2042024-02-03T08:15:51.508ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 492 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 42e06c3c-3b18-458c-94d8-346e0e82c1fe response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
2052024-02-03T08:15:59.250ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 11 reason = Timeout
2062024-02-03T08:15:59.250ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
2072024-02-03T08:15:59.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2082024-02-03T08:15:59.250ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2092024-02-03T08:15:59.250ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 5 reason = Timeout
2102024-02-03T08:15:59.250ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
2112024-02-03T08:15:59.250ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2122024-02-03T08:15:59.250ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2132024-02-03T08:15:59.250ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
2142024-02-03T08:15:59.250ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
2152024-02-03T08:15:59.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2162024-02-03T08:15:59.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
2172024-02-03T08:15:59.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2182024-02-03T08:15:59.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2192024-02-03T08:15:59.251ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2202024-02-03T08:15:59.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2212024-02-03T08:15:59.252ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
2222024-02-03T08:15:59.252ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 1.921727ms iteration = 5
2232024-02-03T08:15:59.252ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2242024-02-03T08:15:59.252ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3a02c6d7-ddb7-483d-b60c-616979859827", "content-length": "1", "date": "Sat, 03 Feb 2024 08:15:58 GMT"} })
2252024-02-03T08:15:59.252ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2262024-02-03T08:15:59.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2272024-02-03T08:15:59.253ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
2282024-02-03T08:15:59.254ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.46349ms iteration = 5
2292024-02-03T08:15:59.254ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bebfcdaa-908b-4f07-bf73-04b41e506d2b", "content-length": "2", "date": "Sat, 03 Feb 2024 08:15:59 GMT"} })
2302024-02-03T08:15:59.254ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 3.574041ms iteration = 5
2312024-02-03T08:15:59.255ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2322024-02-03T08:15:59.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2332024-02-03T08:15:59.255ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2342024-02-03T08:15:59.255ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2352024-02-03T08:15:59.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2362024-02-03T08:15:59.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2372024-02-03T08:15:59.256ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.229814ms iteration = 11
2382024-02-03T08:15:59.257ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2392024-02-03T08:15:59.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2402024-02-03T08:15:59.270ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2412024-02-03T08:15:59.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2422024-02-03T08:15:59.272ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2432024-02-03T08:15:59.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2442024-02-03T08:15:59.273ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2452024-02-03T08:15:59.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
2462024-02-03T08:15:59.276ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2472024-02-03T08:15:59.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2482024-02-03T08:15:59.277ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.719931ms iteration = 5
2492024-02-03T08:16:01.480ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 509 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 87d1bb38-8f2a-47c9-aca3-01886a59bb82 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
2502024-02-03T08:16:09.223ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 12 reason = Timeout
2512024-02-03T08:16:09.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2522024-02-03T08:16:09.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2532024-02-03T08:16:09.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2542024-02-03T08:16:09.227ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2552024-02-03T08:16:09.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2562024-02-03T08:16:09.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2572024-02-03T08:16:09.228ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 4.878862ms iteration = 12
2582024-02-03T08:16:11.453ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 592 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = ff723182-0aaf-4825-a2fd-bdbd320c473b response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
2592024-02-03T08:16:19.196ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
2602024-02-03T08:16:19.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2612024-02-03T08:16:19.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2622024-02-03T08:16:19.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2632024-02-03T08:16:19.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2642024-02-03T08:16:19.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2652024-02-03T08:16:19.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2662024-02-03T08:16:19.201ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 4.729511ms iteration = 13
2672024-02-03T08:16:21.425ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 589 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = d8974412-4eb1-4567-8e8c-51f16f00d84c response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
2682024-02-03T08:16:29.165ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
2692024-02-03T08:16:29.165ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
2702024-02-03T08:16:29.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2712024-02-03T08:16:29.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2722024-02-03T08:16:29.166ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
2732024-02-03T08:16:29.166ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
2742024-02-03T08:16:29.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
2752024-02-03T08:16:29.166ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2762024-02-03T08:16:29.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2772024-02-03T08:16:29.167ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2782024-02-03T08:16:29.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2792024-02-03T08:16:29.168ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
2802024-02-03T08:16:29.168ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
2812024-02-03T08:16:29.168ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
2822024-02-03T08:16:29.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2832024-02-03T08:16:29.168ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
2842024-02-03T08:16:29.168ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2852024-02-03T08:16:29.168ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
2862024-02-03T08:16:29.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
2872024-02-03T08:16:29.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2882024-02-03T08:16:29.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
2892024-02-03T08:16:29.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2902024-02-03T08:16:29.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2912024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
2922024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 6 reason = Timeout
2932024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2942024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2952024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
2962024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
2972024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2982024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2992024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 4 reason = Timeout
3002024-02-03T08:16:29.169ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
3012024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.41\u{b5}s iteration = 4
3022024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
3032024-02-03T08:16:29.169ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
3042024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3052024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3062024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3072024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3082024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3092024-02-03T08:16:29.169ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3102024-02-03T08:16:29.170ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3112024-02-03T08:16:29.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
3122024-02-03T08:16:29.170ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
3132024-02-03T08:16:29.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
3142024-02-03T08:16:29.170ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3152024-02-03T08:16:29.171ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3162024-02-03T08:16:29.171ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
3172024-02-03T08:16:29.171ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8bf17fc7-9523-4a91-afe7-2e40bef46425", "content-length": "1", "date": "Sat, 03 Feb 2024 08:16:28 GMT"} })
3182024-02-03T08:16:29.171ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3192024-02-03T08:16:29.171ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8182c99f-caa7-4590-9764-c07a3d87de6f", "content-length": "2", "date": "Sat, 03 Feb 2024 08:16:29 GMT"} })
3202024-02-03T08:16:29.171ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.214819ms iteration = 6
3212024-02-03T08:16:29.171ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3222024-02-03T08:16:29.171ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
3232024-02-03T08:16:29.171ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 5.489297ms iteration = 5
3242024-02-03T08:16:29.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3252024-02-03T08:16:29.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3262024-02-03T08:16:29.173ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3272024-02-03T08:16:29.173ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3282024-02-03T08:16:29.173ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 4.111366ms iteration = 6
3292024-02-03T08:16:29.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3302024-02-03T08:16:29.173ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
3312024-02-03T08:16:29.173ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.346958ms iteration = 6
3322024-02-03T08:16:29.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3332024-02-03T08:16:29.174ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
3342024-02-03T08:16:29.174ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3352024-02-03T08:16:29.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
3362024-02-03T08:16:29.174ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
3372024-02-03T08:16:29.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3382024-02-03T08:16:29.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
3392024-02-03T08:16:29.175ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
3402024-02-03T08:16:29.175ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
3412024-02-03T08:16:29.175ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
3422024-02-03T08:16:29.175ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3432024-02-03T08:16:29.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3442024-02-03T08:16:29.176ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3452024-02-03T08:16:29.176ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3462024-02-03T08:16:29.176ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
3472024-02-03T08:16:29.176ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 8.765065ms iteration = 5
3482024-02-03T08:16:29.177ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3492024-02-03T08:16:29.177ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
3502024-02-03T08:16:29.177ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
3512024-02-03T08:16:29.177ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
3522024-02-03T08:16:29.177ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3532024-02-03T08:16:29.177ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
3542024-02-03T08:16:29.177ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3552024-02-03T08:16:29.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3562024-02-03T08:16:29.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3572024-02-03T08:16:29.178ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3582024-02-03T08:16:29.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3592024-02-03T08:16:29.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3602024-02-03T08:16:29.178ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3612024-02-03T08:16:29.178ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3622024-02-03T08:16:29.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3632024-02-03T08:16:29.179ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 10.41877ms iteration = 14
3642024-02-03T08:16:29.180ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
3652024-02-03T08:16:29.180ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3662024-02-03T08:16:29.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3672024-02-03T08:16:29.181ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3682024-02-03T08:16:29.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3692024-02-03T08:16:29.181ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3702024-02-03T08:16:29.181ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3712024-02-03T08:16:29.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3722024-02-03T08:16:29.182ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3732024-02-03T08:16:29.183ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
3742024-02-03T08:16:29.183ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
3752024-02-03T08:16:29.183ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3762024-02-03T08:16:29.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3772024-02-03T08:16:29.184ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3782024-02-03T08:16:29.184ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3792024-02-03T08:16:29.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3802024-02-03T08:16:29.184ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3812024-02-03T08:16:29.184ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3822024-02-03T08:16:29.184ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3832024-02-03T08:16:29.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3842024-02-03T08:16:29.185ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
3852024-02-03T08:16:29.185ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 0 version = 2
3862024-02-03T08:16:29.185ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3872024-02-03T08:16:29.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3882024-02-03T08:16:29.186ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3892024-02-03T08:16:29.188ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3902024-02-03T08:16:29.188ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3912024-02-03T08:16:29.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3922024-02-03T08:16:29.189ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3932024-02-03T08:16:29.189ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3942024-02-03T08:16:29.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3952024-02-03T08:16:29.190ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
3962024-02-03T08:16:29.190ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3972024-02-03T08:16:29.190ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-03 08:14:50.307019 UTC dns_group = internal generation = 1
3982024-02-03T08:16:29.190ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 24.753064ms iteration = 5
3992024-02-03T08:16:29.191ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-03T08:14:29.510171Z, time_modified: 2024-02-03T08:14:29.510171Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
4002024-02-03T08:16:29.191ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 23.081099ms iteration = 5
4012024-02-03T08:16:29.193ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 1 version = 2
4022024-02-03T08:16:29.193ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
4032024-02-03T08:16:29.193ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-03 08:14:50.600977 UTC dns_group = external generation = 2
4042024-02-03T08:16:29.193ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 25.46522ms iteration = 5
4052024-02-03T08:16:29.202ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4062024-02-03T08:16:29.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4072024-02-03T08:16:29.204ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4082024-02-03T08:16:29.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4092024-02-03T08:16:29.205ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4102024-02-03T08:16:29.207ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
4112024-02-03T08:16:29.207ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4122024-02-03T08:16:29.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4132024-02-03T08:16:29.208ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 39.356769ms iteration = 6
4142024-02-03T08:16:29.301ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
4152024-02-03T08:16:29.303ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
4162024-02-03T08:16:29.428ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
4172024-02-03T08:16:29.446ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4182024-02-03T08:16:29.447ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "cbc23f80-970d-4521-863b-ec8b64fb9cf6", "date": "Sat, 03 Feb 2024 08:16:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4192024-02-03T08:16:29.447ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4202024-02-03T08:16:29.448ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6c85771d-be92-4ad9-b81f-d4df92f84722", "date": "Sat, 03 Feb 2024 08:16:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4212024-02-03T08:16:29.448ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4222024-02-03T08:16:29.449ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4232024-02-03T08:16:29.449ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 282.365205ms iteration = 7
4242024-02-03T08:16:29.472ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
4252024-02-03T08:16:29.473ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7549f7b3-b340-4d18-a576-b7ca965a66a0", "date": "Sat, 03 Feb 2024 08:16:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4262024-02-03T08:16:29.473ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4272024-02-03T08:16:29.474ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "92828c2a-cad7-43f5-a2cf-3c2d4b519fcf", "date": "Sat, 03 Feb 2024 08:16:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4282024-02-03T08:16:29.474ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4292024-02-03T08:16:29.476ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b467e05d-2de1-44c1-82f8-9d62d97651b0", "date": "Sat, 03 Feb 2024 08:16:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4302024-02-03T08:16:29.476ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4312024-02-03T08:16:29.478ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4322024-02-03T08:16:29.478ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 304.266734ms iteration = 7
4332024-02-03T08:16:31.398ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 498 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 6fef66db-ef13-4229-a9eb-c0fe4ab8807a response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
4342024-02-03T08:16:39.142ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
4352024-02-03T08:16:39.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4362024-02-03T08:16:39.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4372024-02-03T08:16:39.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4382024-02-03T08:16:39.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4392024-02-03T08:16:39.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4402024-02-03T08:16:39.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4412024-02-03T08:16:39.147ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.029103ms iteration = 15
4422024-02-03T08:16:41.371ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 505 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 992b7730-97b6-4f7a-9426-00b45c047688 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
4432024-02-03T08:16:49.115ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
4442024-02-03T08:16:49.115ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4452024-02-03T08:16:49.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4462024-02-03T08:16:49.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4472024-02-03T08:16:49.119ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4482024-02-03T08:16:49.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4492024-02-03T08:16:49.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4502024-02-03T08:16:49.120ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 4.969563ms iteration = 16
4512024-02-03T08:16:51.344ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 489 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = cb0754de-901c-42cd-b64d-8fa582ce1e72 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
4522024-02-03T08:16:59.088ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
4532024-02-03T08:16:59.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4542024-02-03T08:16:59.088ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 7 reason = Timeout
4552024-02-03T08:16:59.088ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4562024-02-03T08:16:59.088ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
4572024-02-03T08:16:59.088ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4582024-02-03T08:16:59.088ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
4592024-02-03T08:16:59.088ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
4602024-02-03T08:16:59.088ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
4612024-02-03T08:16:59.088ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4622024-02-03T08:16:59.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4632024-02-03T08:16:59.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4642024-02-03T08:16:59.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4652024-02-03T08:16:59.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4662024-02-03T08:16:59.090ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
4672024-02-03T08:16:59.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4682024-02-03T08:16:59.090ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2aff2c92-10f3-47bb-bc46-2ae689ee9937", "content-length": "2", "date": "Sat, 03 Feb 2024 08:16:59 GMT"} })
4692024-02-03T08:16:59.090ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4702024-02-03T08:16:59.090ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 1.804616ms iteration = 7
4712024-02-03T08:16:59.090ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e853da2f-50a9-4473-a7e1-6b1ecd299d45", "content-length": "1", "date": "Sat, 03 Feb 2024 08:16:58 GMT"} })
4722024-02-03T08:16:59.090ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4732024-02-03T08:16:59.090ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4742024-02-03T08:16:59.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4752024-02-03T08:16:59.091ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
4762024-02-03T08:16:59.091ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.095836ms iteration = 7
4772024-02-03T08:16:59.091ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.242438ms iteration = 7
4782024-02-03T08:16:59.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4792024-02-03T08:16:59.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4802024-02-03T08:16:59.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4812024-02-03T08:16:59.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4822024-02-03T08:16:59.094ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4832024-02-03T08:16:59.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4842024-02-03T08:16:59.094ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.175853ms iteration = 17
4852024-02-03T08:16:59.095ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4862024-02-03T08:16:59.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4872024-02-03T08:16:59.105ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4882024-02-03T08:16:59.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4892024-02-03T08:16:59.107ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4902024-02-03T08:16:59.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4912024-02-03T08:16:59.108ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4922024-02-03T08:16:59.110ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
4932024-02-03T08:16:59.110ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4942024-02-03T08:16:59.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4952024-02-03T08:16:59.111ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 23.050548ms iteration = 7
4962024-02-03T08:17:01.321ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1264 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = d25d40ff-c9f7-49e9-97d2-6daf7f950dc5 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
4972024-02-03T08:17:09.085ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
4982024-02-03T08:17:09.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4992024-02-03T08:17:09.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5002024-02-03T08:17:09.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5012024-02-03T08:17:09.089ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5022024-02-03T08:17:09.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5032024-02-03T08:17:09.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5042024-02-03T08:17:09.091ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.094714ms iteration = 18
5052024-02-03T08:17:11.321ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 843 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 91da21d5-724a-43b2-8997-bd865c840848 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
5062024-02-03T08:17:19.086ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
5072024-02-03T08:17:19.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5082024-02-03T08:17:19.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5092024-02-03T08:17:19.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5102024-02-03T08:17:19.090ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5112024-02-03T08:17:19.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5122024-02-03T08:17:19.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5132024-02-03T08:17:19.091ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 4.749211ms iteration = 19
5142024-02-03T08:17:21.321ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 497 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = a9dcb4c2-165a-47fd-9186-c720aea34e6b response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
5152024-02-03T08:17:29.083ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
5162024-02-03T08:17:29.083ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
5172024-02-03T08:17:29.083ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5182024-02-03T08:17:29.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5192024-02-03T08:17:29.084ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
5202024-02-03T08:17:29.084ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
5212024-02-03T08:17:29.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
5222024-02-03T08:17:29.084ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5232024-02-03T08:17:29.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5242024-02-03T08:17:29.085ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5252024-02-03T08:17:29.085ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5262024-02-03T08:17:29.086ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
5272024-02-03T08:17:29.086ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
5282024-02-03T08:17:29.086ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5292024-02-03T08:17:29.086ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
5302024-02-03T08:17:29.086ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
5312024-02-03T08:17:29.086ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
5322024-02-03T08:17:29.086ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5332024-02-03T08:17:29.086ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
5342024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
5352024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5362024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
5372024-02-03T08:17:29.087ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
5382024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
5392024-02-03T08:17:29.087ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
5402024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.65\u{b5}s iteration = 5
5412024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
5422024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 8 reason = Timeout
5432024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5442024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5452024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
5462024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5472024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5482024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5492024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5502024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5512024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5522024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5532024-02-03T08:17:29.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
5542024-02-03T08:17:29.088ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5552024-02-03T08:17:29.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5562024-02-03T08:17:29.088ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
5572024-02-03T08:17:29.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
5582024-02-03T08:17:29.089ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5592024-02-03T08:17:29.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
5602024-02-03T08:17:29.089ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5612024-02-03T08:17:29.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5622024-02-03T08:17:29.089ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
5632024-02-03T08:17:29.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5642024-02-03T08:17:29.089ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fff04fcb-2d58-4844-a897-1e3da089cce0", "content-length": "2", "date": "Sat, 03 Feb 2024 08:17:29 GMT"} })
5652024-02-03T08:17:29.089ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.691543ms iteration = 8
5662024-02-03T08:17:29.090ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
5672024-02-03T08:17:29.090ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5682024-02-03T08:17:29.090ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 6.223054ms iteration = 6
5692024-02-03T08:17:29.090ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5702024-02-03T08:17:29.090ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
5712024-02-03T08:17:29.090ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.659071ms iteration = 8
5722024-02-03T08:17:29.091ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5d3c9ef6-761b-4cf8-a44f-662cfd2c53a4", "content-length": "1", "date": "Sat, 03 Feb 2024 08:17:28 GMT"} })
5732024-02-03T08:17:29.091ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5742024-02-03T08:17:29.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5752024-02-03T08:17:29.092ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5762024-02-03T08:17:29.093ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
5772024-02-03T08:17:29.093ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
5782024-02-03T08:17:29.093ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
5792024-02-03T08:17:29.093ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5802024-02-03T08:17:29.093ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 6.289284ms iteration = 8
5812024-02-03T08:17:29.093ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
5822024-02-03T08:17:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5832024-02-03T08:17:29.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
5842024-02-03T08:17:29.094ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
5852024-02-03T08:17:29.094ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5862024-02-03T08:17:29.094ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5872024-02-03T08:17:29.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
5882024-02-03T08:17:29.094ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5892024-02-03T08:17:29.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5902024-02-03T08:17:29.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5912024-02-03T08:17:29.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5922024-02-03T08:17:29.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5932024-02-03T08:17:29.095ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5942024-02-03T08:17:29.095ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5952024-02-03T08:17:29.095ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5962024-02-03T08:17:29.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5972024-02-03T08:17:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5982024-02-03T08:17:29.098ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5992024-02-03T08:17:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6002024-02-03T08:17:29.102ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
6012024-02-03T08:17:29.102ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 15.421533ms iteration = 6
6022024-02-03T08:17:29.102ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
6032024-02-03T08:17:29.102ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
6042024-02-03T08:17:29.102ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
6052024-02-03T08:17:29.102ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6062024-02-03T08:17:29.102ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.647765ms iteration = 20
6072024-02-03T08:17:29.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6082024-02-03T08:17:29.102ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6092024-02-03T08:17:29.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6102024-02-03T08:17:29.103ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
6112024-02-03T08:17:29.103ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
6122024-02-03T08:17:29.103ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6132024-02-03T08:17:29.103ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6142024-02-03T08:17:29.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6152024-02-03T08:17:29.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6162024-02-03T08:17:29.103ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6172024-02-03T08:17:29.104ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6182024-02-03T08:17:29.104ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6192024-02-03T08:17:29.106ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6202024-02-03T08:17:29.106ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6212024-02-03T08:17:29.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6222024-02-03T08:17:29.107ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
6232024-02-03T08:17:29.107ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6242024-02-03T08:17:29.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6252024-02-03T08:17:29.108ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6262024-02-03T08:17:29.108ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6272024-02-03T08:17:29.108ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
6282024-02-03T08:17:29.108ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 0 version = 2
6292024-02-03T08:17:29.108ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6302024-02-03T08:17:29.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6312024-02-03T08:17:29.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6322024-02-03T08:17:29.109ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6332024-02-03T08:17:29.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6342024-02-03T08:17:29.109ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6352024-02-03T08:17:29.110ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6362024-02-03T08:17:29.110ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
6372024-02-03T08:17:29.110ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
6382024-02-03T08:17:29.110ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-03 08:14:50.307019 UTC dns_group = internal generation = 1
6392024-02-03T08:17:29.110ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 27.037343ms iteration = 6
6402024-02-03T08:17:29.112ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6412024-02-03T08:17:29.112ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6422024-02-03T08:17:29.112ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6432024-02-03T08:17:29.112ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6442024-02-03T08:17:29.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6452024-02-03T08:17:29.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6462024-02-03T08:17:29.114ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 1 version = 2
6472024-02-03T08:17:29.114ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
6482024-02-03T08:17:29.114ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-03 08:14:50.600977 UTC dns_group = external generation = 2
6492024-02-03T08:17:29.114ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 27.304125ms iteration = 6
6502024-02-03T08:17:29.117ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6512024-02-03T08:17:29.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6522024-02-03T08:17:29.118ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-03T08:14:29.510171Z, time_modified: 2024-02-03T08:14:29.510171Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }) }
6532024-02-03T08:17:29.118ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 31.608753ms iteration = 6
6542024-02-03T08:17:29.119ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6552024-02-03T08:17:29.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6562024-02-03T08:17:29.119ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6572024-02-03T08:17:29.122ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
6582024-02-03T08:17:29.122ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6592024-02-03T08:17:29.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6602024-02-03T08:17:29.123ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 36.254183ms iteration = 8
6612024-02-03T08:17:29.151ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
6622024-02-03T08:17:29.159ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
6632024-02-03T08:17:29.223ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
6642024-02-03T08:17:29.224ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "73649cf9-97d8-4345-8f45-769c9316bfad", "date": "Sat, 03 Feb 2024 08:17:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
6652024-02-03T08:17:29.224ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
6662024-02-03T08:17:29.225ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "02914119-790c-4b68-b3ba-d1b3017db2a3", "date": "Sat, 03 Feb 2024 08:17:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
6672024-02-03T08:17:29.225ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
6682024-02-03T08:17:29.225ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
6692024-02-03T08:17:29.226ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
6702024-02-03T08:17:29.226ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 139.928957ms iteration = 8
6712024-02-03T08:17:29.258ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
6722024-02-03T08:17:29.258ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9d5905cb-fcae-4747-9183-f949f8a6c678", "date": "Sat, 03 Feb 2024 08:17:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
6732024-02-03T08:17:29.258ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
6742024-02-03T08:17:29.261ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ae45eb6d-280a-4bde-8ac1-fc20bb3073f1", "date": "Sat, 03 Feb 2024 08:17:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
6752024-02-03T08:17:29.261ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
6762024-02-03T08:17:29.262ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2894fbe2-3be3-4352-9217-712de2f27f26", "date": "Sat, 03 Feb 2024 08:17:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
6772024-02-03T08:17:29.262ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
6782024-02-03T08:17:29.264ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
6792024-02-03T08:17:29.264ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 171.232827ms iteration = 8
6802024-02-03T08:17:31.323ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 591 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 22ad4db0-d1c1-4e87-9d0a-679aca90d411 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
6812024-02-03T08:17:39.087ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
6822024-02-03T08:17:39.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6832024-02-03T08:17:39.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6842024-02-03T08:17:39.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6852024-02-03T08:17:39.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6862024-02-03T08:17:39.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6872024-02-03T08:17:39.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6882024-02-03T08:17:39.120ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 32.44446ms iteration = 21
6892024-02-03T08:17:41.323ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 487 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = cf1d52af-bd20-4c16-a2ef-e293cfe85c99 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
6902024-02-03T08:17:49.088ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
6912024-02-03T08:17:49.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6922024-02-03T08:17:49.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6932024-02-03T08:17:49.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6942024-02-03T08:17:49.092ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6952024-02-03T08:17:49.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6962024-02-03T08:17:49.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6972024-02-03T08:17:49.093ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.462377ms iteration = 22
6982024-02-03T08:17:51.323ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 493 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = ce491923-858c-4b70-9022-a5d98664fc2c response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
6992024-02-03T08:17:59.088ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
7002024-02-03T08:17:59.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7012024-02-03T08:17:59.088ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
7022024-02-03T08:17:59.088ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
7032024-02-03T08:17:59.088ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
7042024-02-03T08:17:59.088ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
7052024-02-03T08:17:59.089ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7062024-02-03T08:17:59.089ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7072024-02-03T08:17:59.089ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
7082024-02-03T08:17:59.089ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7092024-02-03T08:17:59.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7102024-02-03T08:17:59.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7112024-02-03T08:17:59.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7122024-02-03T08:17:59.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7132024-02-03T08:17:59.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7142024-02-03T08:17:59.090ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7152024-02-03T08:17:59.091ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7162024-02-03T08:17:59.091ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
7172024-02-03T08:17:59.091ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
7182024-02-03T08:17:59.091ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.477152ms iteration = 9
7192024-02-03T08:17:59.091ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a4b4cd6f-fcb9-4c57-8481-dac7c48a25ff", "content-length": "2", "date": "Sat, 03 Feb 2024 08:17:59 GMT"} })
7202024-02-03T08:17:59.091ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.528742ms iteration = 9
7212024-02-03T08:17:59.091ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "862c8b73-e777-4e6d-96c2-e276ef54e2bc", "content-length": "1", "date": "Sat, 03 Feb 2024 08:17:58 GMT"} })
7222024-02-03T08:17:59.091ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7232024-02-03T08:17:59.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7242024-02-03T08:17:59.092ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.923414ms iteration = 9
7252024-02-03T08:17:59.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7262024-02-03T08:17:59.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7272024-02-03T08:17:59.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7282024-02-03T08:17:59.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
7292024-02-03T08:17:59.094ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7302024-02-03T08:17:59.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7312024-02-03T08:17:59.095ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7322024-02-03T08:17:59.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7332024-02-03T08:17:59.095ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.291314ms iteration = 23
7342024-02-03T08:17:59.109ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7352024-02-03T08:17:59.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7362024-02-03T08:17:59.110ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7372024-02-03T08:17:59.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7382024-02-03T08:17:59.111ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7392024-02-03T08:17:59.114ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
7402024-02-03T08:17:59.114ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7412024-02-03T08:17:59.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7422024-02-03T08:17:59.115ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.386918ms iteration = 9
7432024-02-03T08:18:01.324ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 833 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 07ef39bf-c74d-4679-a105-9241802e9917 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
7442024-02-03T08:18:09.089ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
7452024-02-03T08:18:09.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7462024-02-03T08:18:09.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7472024-02-03T08:18:09.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7482024-02-03T08:18:09.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7492024-02-03T08:18:09.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7502024-02-03T08:18:09.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7512024-02-03T08:18:09.094ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.688199ms iteration = 24
7522024-02-03T08:18:11.325ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1051 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = df9e16c2-ad1f-4bdf-83ce-2deedb8108b6 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
7532024-02-03T08:18:19.088ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
7542024-02-03T08:18:19.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7552024-02-03T08:18:19.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7562024-02-03T08:18:19.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7572024-02-03T08:18:19.092ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7582024-02-03T08:18:19.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7592024-02-03T08:18:19.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7602024-02-03T08:18:19.094ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.878201ms iteration = 25
7612024-02-03T08:18:21.323ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 524 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 11b8648f-c8e3-4c5d-bf8c-ad085bd8b44b response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
7622024-02-03T08:18:29.085ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
7632024-02-03T08:18:29.085ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
7642024-02-03T08:18:29.085ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7652024-02-03T08:18:29.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7662024-02-03T08:18:29.085ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
7672024-02-03T08:18:29.085ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
7682024-02-03T08:18:29.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
7692024-02-03T08:18:29.085ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7702024-02-03T08:18:29.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7712024-02-03T08:18:29.087ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7722024-02-03T08:18:29.087ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7732024-02-03T08:18:29.088ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
7742024-02-03T08:18:29.088ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
7752024-02-03T08:18:29.088ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
7762024-02-03T08:18:29.088ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
7772024-02-03T08:18:29.088ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
7782024-02-03T08:18:29.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7792024-02-03T08:18:29.088ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7802024-02-03T08:18:29.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
7812024-02-03T08:18:29.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
7822024-02-03T08:18:29.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7832024-02-03T08:18:29.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7842024-02-03T08:18:29.089ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
7852024-02-03T08:18:29.089ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
7862024-02-03T08:18:29.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7872024-02-03T08:18:29.089ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7882024-02-03T08:18:29.089ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
7892024-02-03T08:18:29.089ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
7902024-02-03T08:18:29.089ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
7912024-02-03T08:18:29.089ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
7922024-02-03T08:18:29.089ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.29\u{b5}s iteration = 6
7932024-02-03T08:18:29.089ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
7942024-02-03T08:18:29.089ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
7952024-02-03T08:18:29.089ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7962024-02-03T08:18:29.089ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7972024-02-03T08:18:29.089ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7982024-02-03T08:18:29.089ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7992024-02-03T08:18:29.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8002024-02-03T08:18:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
8012024-02-03T08:18:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8022024-02-03T08:18:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8032024-02-03T08:18:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8042024-02-03T08:18:29.090ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8052024-02-03T08:18:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
8062024-02-03T08:18:29.090ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
8072024-02-03T08:18:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
8082024-02-03T08:18:29.091ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8092024-02-03T08:18:29.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8102024-02-03T08:18:29.092ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
8112024-02-03T08:18:29.092ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 6.517736ms iteration = 7
8122024-02-03T08:18:29.092ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8132024-02-03T08:18:29.092ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
8142024-02-03T08:18:29.092ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.914975ms iteration = 10
8152024-02-03T08:18:29.092ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
8162024-02-03T08:18:29.092ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
8172024-02-03T08:18:29.092ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8182024-02-03T08:18:29.092ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9b196ff8-2f39-4a22-b0a1-cacde6a0d45d", "content-length": "2", "date": "Sat, 03 Feb 2024 08:18:29 GMT"} })
8192024-02-03T08:18:29.092ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 3.239248ms iteration = 10
8202024-02-03T08:18:29.092ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e535c06a-415b-4e34-bee8-a33afbab9775", "content-length": "1", "date": "Sat, 03 Feb 2024 08:18:28 GMT"} })
8212024-02-03T08:18:29.092ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8222024-02-03T08:18:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8232024-02-03T08:18:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
8242024-02-03T08:18:29.093ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
8252024-02-03T08:18:29.093ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8262024-02-03T08:18:29.093ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8272024-02-03T08:18:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
8282024-02-03T08:18:29.093ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
8292024-02-03T08:18:29.093ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
8302024-02-03T08:18:29.093ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
8312024-02-03T08:18:29.093ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8322024-02-03T08:18:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8332024-02-03T08:18:29.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8342024-02-03T08:18:29.094ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 4.61037ms iteration = 10
8352024-02-03T08:18:29.094ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
8362024-02-03T08:18:29.095ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8372024-02-03T08:18:29.095ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8382024-02-03T08:18:29.095ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
8392024-02-03T08:18:29.095ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 6.693717ms iteration = 7
8402024-02-03T08:18:29.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8412024-02-03T08:18:29.096ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8422024-02-03T08:18:29.097ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8432024-02-03T08:18:29.097ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8442024-02-03T08:18:29.097ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8452024-02-03T08:18:29.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8462024-02-03T08:18:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8472024-02-03T08:18:29.098ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8482024-02-03T08:18:29.098ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
8492024-02-03T08:18:29.098ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
8502024-02-03T08:18:29.098ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
8512024-02-03T08:18:29.098ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8522024-02-03T08:18:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8532024-02-03T08:18:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8542024-02-03T08:18:29.099ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8552024-02-03T08:18:29.099ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8562024-02-03T08:18:29.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8572024-02-03T08:18:29.099ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8582024-02-03T08:18:29.099ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 10.39116ms iteration = 26
8592024-02-03T08:18:29.099ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8602024-02-03T08:18:29.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8612024-02-03T08:18:29.100ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
8622024-02-03T08:18:29.100ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
8632024-02-03T08:18:29.100ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8642024-02-03T08:18:29.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8652024-02-03T08:18:29.101ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
8662024-02-03T08:18:29.101ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8672024-02-03T08:18:29.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8682024-02-03T08:18:29.101ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8692024-02-03T08:18:29.102ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8702024-02-03T08:18:29.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8712024-02-03T08:18:29.103ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8722024-02-03T08:18:29.103ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8732024-02-03T08:18:29.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8742024-02-03T08:18:29.103ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8752024-02-03T08:18:29.104ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
8762024-02-03T08:18:29.104ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 0 version = 2
8772024-02-03T08:18:29.104ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8782024-02-03T08:18:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8792024-02-03T08:18:29.105ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8802024-02-03T08:18:29.105ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8812024-02-03T08:18:29.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8822024-02-03T08:18:29.105ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8832024-02-03T08:18:29.106ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8842024-02-03T08:18:29.106ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8852024-02-03T08:18:29.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8862024-02-03T08:18:29.107ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
8872024-02-03T08:18:29.107ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
8882024-02-03T08:18:29.107ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-03 08:14:50.307019 UTC dns_group = internal generation = 1
8892024-02-03T08:18:29.107ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 21.876368ms iteration = 7
8902024-02-03T08:18:29.108ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8912024-02-03T08:18:29.108ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8922024-02-03T08:18:29.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8932024-02-03T08:18:29.110ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 1 version = 2
8942024-02-03T08:18:29.110ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
8952024-02-03T08:18:29.110ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-03 08:14:50.600977 UTC dns_group = external generation = 2
8962024-02-03T08:18:29.110ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 21.861539ms iteration = 7
8972024-02-03T08:18:29.110ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-03T08:14:29.510171Z, time_modified: 2024-02-03T08:14:29.510171Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }) }
8982024-02-03T08:18:29.110ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 22.509734ms iteration = 7
8992024-02-03T08:18:29.115ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9002024-02-03T08:18:29.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9012024-02-03T08:18:29.116ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9022024-02-03T08:18:29.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9032024-02-03T08:18:29.117ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9042024-02-03T08:18:29.119ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
9052024-02-03T08:18:29.119ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9062024-02-03T08:18:29.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9072024-02-03T08:18:29.121ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 31.706623ms iteration = 10
9082024-02-03T08:18:29.223ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
9092024-02-03T08:18:29.225ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
9102024-02-03T08:18:29.351ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
9112024-02-03T08:18:29.366ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
9122024-02-03T08:18:29.367ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f516beca-e151-48b3-9ec9-d1b93da56ba2", "date": "Sat, 03 Feb 2024 08:18:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
9132024-02-03T08:18:29.367ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
9142024-02-03T08:18:29.369ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0b1f92c0-bb6a-4094-be58-d92c64339a04", "date": "Sat, 03 Feb 2024 08:18:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
9152024-02-03T08:18:29.369ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
9162024-02-03T08:18:29.370ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
9172024-02-03T08:18:29.370ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 282.244904ms iteration = 9
9182024-02-03T08:18:29.393ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
9192024-02-03T08:18:29.394ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6807c80a-3859-4a13-ad7b-6e6ac6cc2d35", "date": "Sat, 03 Feb 2024 08:18:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
9202024-02-03T08:18:29.394ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
9212024-02-03T08:18:29.395ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2dace059-3762-43c1-ade3-d7254863c0fa", "date": "Sat, 03 Feb 2024 08:18:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
9222024-02-03T08:18:29.395ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
9232024-02-03T08:18:29.396ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "eeb62de4-6693-44c9-963a-5fc4b0150fc8", "date": "Sat, 03 Feb 2024 08:18:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
9242024-02-03T08:18:29.396ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
9252024-02-03T08:18:29.398ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
9262024-02-03T08:18:29.398ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 303.82682ms iteration = 9
9272024-02-03T08:18:31.324ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 501 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 7ddd42cc-4470-446b-8123-0ffbcf98e4fe response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
9282024-02-03T08:18:39.089ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
9292024-02-03T08:18:39.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9302024-02-03T08:18:39.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9312024-02-03T08:18:39.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9322024-02-03T08:18:39.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9332024-02-03T08:18:39.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9342024-02-03T08:18:39.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9352024-02-03T08:18:39.094ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.568788ms iteration = 27
9362024-02-03T08:18:41.325ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1124 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 91eafe20-c95f-459f-b377-20e97d212087 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
9372024-02-03T08:18:49.089ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
9382024-02-03T08:18:49.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9392024-02-03T08:18:49.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9402024-02-03T08:18:49.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9412024-02-03T08:18:49.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9422024-02-03T08:18:49.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9432024-02-03T08:18:49.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9442024-02-03T08:18:49.095ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.979521ms iteration = 28
9452024-02-03T08:18:51.324ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 934 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 52aab228-927d-4dcd-b0d1-755624187f7b response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
9462024-02-03T08:18:59.089ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
9472024-02-03T08:18:59.089ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
9482024-02-03T08:18:59.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9492024-02-03T08:18:59.089ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
9502024-02-03T08:18:59.089ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9512024-02-03T08:18:59.089ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
9522024-02-03T08:18:59.089ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9532024-02-03T08:18:59.089ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9542024-02-03T08:18:59.089ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
9552024-02-03T08:18:59.089ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
9562024-02-03T08:18:59.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9572024-02-03T08:18:59.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9582024-02-03T08:18:59.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
9592024-02-03T08:18:59.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9602024-02-03T08:18:59.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9612024-02-03T08:18:59.091ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
9622024-02-03T08:18:59.091ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9632024-02-03T08:18:59.091ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c7b27ea4-a63a-4844-a7be-1711e8702d78", "content-length": "1", "date": "Sat, 03 Feb 2024 08:18:58 GMT"} })
9642024-02-03T08:18:59.091ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9652024-02-03T08:18:59.091ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
9662024-02-03T08:18:59.091ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4c7bd6d1-7f46-4c2e-b5d6-b9cd57d8708f", "content-length": "2", "date": "Sat, 03 Feb 2024 08:18:59 GMT"} })
9672024-02-03T08:18:59.091ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.486421ms iteration = 11
9682024-02-03T08:18:59.092ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
9692024-02-03T08:18:59.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9702024-02-03T08:18:59.092ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.579632ms iteration = 11
9712024-02-03T08:18:59.093ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 4.023265ms iteration = 11
9722024-02-03T08:18:59.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9732024-02-03T08:18:59.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9742024-02-03T08:18:59.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9752024-02-03T08:18:59.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
9762024-02-03T08:18:59.094ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9772024-02-03T08:18:59.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9782024-02-03T08:18:59.096ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9792024-02-03T08:18:59.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9802024-02-03T08:18:59.100ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 10.739963ms iteration = 29
9812024-02-03T08:18:59.106ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9822024-02-03T08:18:59.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9832024-02-03T08:18:59.108ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9842024-02-03T08:18:59.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9852024-02-03T08:18:59.109ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9862024-02-03T08:18:59.111ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
9872024-02-03T08:18:59.111ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9882024-02-03T08:18:59.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9892024-02-03T08:18:59.112ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 23.122339ms iteration = 11
9902024-02-03T08:19:01.324ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 514 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 76c6d06d-e728-45a7-a12f-6ae79f88af30 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
9912024-02-03T08:19:09.089ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
9922024-02-03T08:19:09.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9932024-02-03T08:19:09.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9942024-02-03T08:19:09.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9952024-02-03T08:19:09.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9962024-02-03T08:19:09.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9972024-02-03T08:19:09.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9982024-02-03T08:19:09.095ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.387116ms iteration = 30
9992024-02-03T08:19:11.325ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 596 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = da8f587d-2fdb-4064-87c7-0b2f9f3ab74d response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
10002024-02-03T08:19:19.090ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
10012024-02-03T08:19:19.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10022024-02-03T08:19:19.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10032024-02-03T08:19:19.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10042024-02-03T08:19:19.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10052024-02-03T08:19:19.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10062024-02-03T08:19:19.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10072024-02-03T08:19:19.095ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.377516ms iteration = 31
10082024-02-03T08:19:21.326ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1104 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 6e7649f4-8c69-4fb8-8460-1572803d529a response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
10092024-02-03T08:19:29.086ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
10102024-02-03T08:19:29.086ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
10112024-02-03T08:19:29.086ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10122024-02-03T08:19:29.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10132024-02-03T08:19:29.086ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
10142024-02-03T08:19:29.086ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
10152024-02-03T08:19:29.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
10162024-02-03T08:19:29.087ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10172024-02-03T08:19:29.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10182024-02-03T08:19:29.087ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10192024-02-03T08:19:29.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10202024-02-03T08:19:29.089ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
10212024-02-03T08:19:29.089ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
10222024-02-03T08:19:29.089ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
10232024-02-03T08:19:29.089ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
10242024-02-03T08:19:29.089ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10252024-02-03T08:19:29.089ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
10262024-02-03T08:19:29.089ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10272024-02-03T08:19:29.089ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
10282024-02-03T08:19:29.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
10292024-02-03T08:19:29.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10302024-02-03T08:19:29.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10312024-02-03T08:19:29.090ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
10322024-02-03T08:19:29.090ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
10332024-02-03T08:19:29.090ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
10342024-02-03T08:19:29.090ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
10352024-02-03T08:19:29.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10362024-02-03T08:19:29.090ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
10372024-02-03T08:19:29.090ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10382024-02-03T08:19:29.090ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
10392024-02-03T08:19:29.090ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
10402024-02-03T08:19:29.090ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
10412024-02-03T08:19:29.090ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 7.96\u{b5}s iteration = 7
10422024-02-03T08:19:29.090ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
10432024-02-03T08:19:29.090ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10442024-02-03T08:19:29.090ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10452024-02-03T08:19:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
10462024-02-03T08:19:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10472024-02-03T08:19:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10482024-02-03T08:19:29.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10492024-02-03T08:19:29.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10502024-02-03T08:19:29.091ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10512024-02-03T08:19:29.091ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10522024-02-03T08:19:29.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
10532024-02-03T08:19:29.091ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
10542024-02-03T08:19:29.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
10552024-02-03T08:19:29.092ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10562024-02-03T08:19:29.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10572024-02-03T08:19:29.092ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
10582024-02-03T08:19:29.092ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9f04768e-fc9c-44fc-a9d6-d1f2b0139867", "content-length": "1", "date": "Sat, 03 Feb 2024 08:19:28 GMT"} })
10592024-02-03T08:19:29.092ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10602024-02-03T08:19:29.093ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
10612024-02-03T08:19:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10622024-02-03T08:19:29.093ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
10632024-02-03T08:19:29.093ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 6.518817ms iteration = 8
10642024-02-03T08:19:29.093ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10652024-02-03T08:19:29.093ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "81f0e6d6-1706-484b-8999-3ad99b7b612c", "content-length": "2", "date": "Sat, 03 Feb 2024 08:19:29 GMT"} })
10662024-02-03T08:19:29.093ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.969676ms iteration = 12
10672024-02-03T08:19:29.093ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
10682024-02-03T08:19:29.093ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.137677ms iteration = 12
10692024-02-03T08:19:29.094ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.730792ms iteration = 12
10702024-02-03T08:19:29.095ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
10712024-02-03T08:19:29.095ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
10722024-02-03T08:19:29.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10732024-02-03T08:19:29.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
10742024-02-03T08:19:29.096ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
10752024-02-03T08:19:29.096ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10762024-02-03T08:19:29.096ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
10772024-02-03T08:19:29.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10782024-02-03T08:19:29.096ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
10792024-02-03T08:19:29.096ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
10802024-02-03T08:19:29.096ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
10812024-02-03T08:19:29.096ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10822024-02-03T08:19:29.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10832024-02-03T08:19:29.097ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10842024-02-03T08:19:29.097ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
10852024-02-03T08:19:29.097ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 8.418462ms iteration = 8
10862024-02-03T08:19:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10872024-02-03T08:19:29.098ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10882024-02-03T08:19:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
10892024-02-03T08:19:29.098ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10902024-02-03T08:19:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10912024-02-03T08:19:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10922024-02-03T08:19:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10932024-02-03T08:19:29.098ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10942024-02-03T08:19:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10952024-02-03T08:19:29.099ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10962024-02-03T08:19:29.099ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
10972024-02-03T08:19:29.099ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
10982024-02-03T08:19:29.099ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
10992024-02-03T08:19:29.099ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11002024-02-03T08:19:29.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11012024-02-03T08:19:29.100ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11022024-02-03T08:19:29.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11032024-02-03T08:19:29.100ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 9.923155ms iteration = 32
11042024-02-03T08:19:29.100ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11052024-02-03T08:19:29.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11062024-02-03T08:19:29.102ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11072024-02-03T08:19:29.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11082024-02-03T08:19:29.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
11092024-02-03T08:19:29.102ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11102024-02-03T08:19:29.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11112024-02-03T08:19:29.103ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
11122024-02-03T08:19:29.103ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
11132024-02-03T08:19:29.103ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11142024-02-03T08:19:29.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11152024-02-03T08:19:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11162024-02-03T08:19:29.104ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11172024-02-03T08:19:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11182024-02-03T08:19:29.110ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11192024-02-03T08:19:29.110ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11202024-02-03T08:19:29.110ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
11212024-02-03T08:19:29.110ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 0 version = 2
11222024-02-03T08:19:29.110ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11232024-02-03T08:19:29.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11242024-02-03T08:19:29.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11252024-02-03T08:19:29.113ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11262024-02-03T08:19:29.113ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11272024-02-03T08:19:29.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11282024-02-03T08:19:29.114ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11292024-02-03T08:19:29.115ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11302024-02-03T08:19:29.116ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
11312024-02-03T08:19:29.116ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
11322024-02-03T08:19:29.116ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-03 08:14:50.307019 UTC dns_group = internal generation = 1
11332024-02-03T08:19:29.116ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 29.803107ms iteration = 8
11342024-02-03T08:19:29.117ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
11352024-02-03T08:19:29.117ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11362024-02-03T08:19:29.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11372024-02-03T08:19:29.118ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11382024-02-03T08:19:29.118ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11392024-02-03T08:19:29.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11402024-02-03T08:19:29.119ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 1 version = 2
11412024-02-03T08:19:29.120ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
11422024-02-03T08:19:29.120ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-03 08:14:50.600977 UTC dns_group = external generation = 2
11432024-02-03T08:19:29.120ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 30.640884ms iteration = 8
11442024-02-03T08:19:29.120ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11452024-02-03T08:19:29.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11462024-02-03T08:19:29.122ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11472024-02-03T08:19:29.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11482024-02-03T08:19:29.123ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11492024-02-03T08:19:29.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
11502024-02-03T08:19:29.125ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11512024-02-03T08:19:29.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11522024-02-03T08:19:29.126ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 36.177912ms iteration = 12
11532024-02-03T08:19:29.127ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-03T08:14:29.510171Z, time_modified: 2024-02-03T08:14:29.510171Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }) }
11542024-02-03T08:19:29.127ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 38.017928ms iteration = 8
11552024-02-03T08:19:29.142ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
11562024-02-03T08:19:29.145ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
11572024-02-03T08:19:29.180ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
11582024-02-03T08:19:29.180ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "bea48b93-3e01-4583-a485-56d3b2be4a18", "date": "Sat, 03 Feb 2024 08:19:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11592024-02-03T08:19:29.180ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11602024-02-03T08:19:29.182ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "aaa4c9d2-0c43-4530-8a6c-870a08d7a431", "date": "Sat, 03 Feb 2024 08:19:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11612024-02-03T08:19:29.182ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11622024-02-03T08:19:29.183ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
11632024-02-03T08:19:29.183ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11642024-02-03T08:19:29.184ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 94.661326ms iteration = 10
11652024-02-03T08:19:29.214ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
11662024-02-03T08:19:29.215ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "00562cde-a55a-4e4b-b897-b8cd3b8dab37", "date": "Sat, 03 Feb 2024 08:19:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
11672024-02-03T08:19:29.215ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
11682024-02-03T08:19:29.217ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "460b7218-838a-4f38-a432-b23d39a644be", "date": "Sat, 03 Feb 2024 08:19:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
11692024-02-03T08:19:29.217ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
11702024-02-03T08:19:29.218ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9c0d3f42-d7a6-46be-81e1-03fc3948d3fe", "date": "Sat, 03 Feb 2024 08:19:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
11712024-02-03T08:19:29.218ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
11722024-02-03T08:19:29.219ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
11732024-02-03T08:19:29.219ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 122.736619ms iteration = 10
11742024-02-03T08:19:31.326ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1093 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 9d05716b-82a3-4c6e-a206-2c86e04a54f0 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
11752024-02-03T08:19:39.089ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
11762024-02-03T08:19:39.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11772024-02-03T08:19:39.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11782024-02-03T08:19:39.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11792024-02-03T08:19:39.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11802024-02-03T08:19:39.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11812024-02-03T08:19:39.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11822024-02-03T08:19:39.098ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 8.08274ms iteration = 33
11832024-02-03T08:19:41.325ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 559 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 8f0ae931-092e-4e67-9254-b4b429608b2b response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
11842024-02-03T08:19:49.091ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
11852024-02-03T08:19:49.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11862024-02-03T08:19:49.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11872024-02-03T08:19:49.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11882024-02-03T08:19:49.095ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11892024-02-03T08:19:49.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11902024-02-03T08:19:49.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11912024-02-03T08:19:49.096ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.310506ms iteration = 34
11922024-02-03T08:19:51.326ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 490 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 66f657f7-b5b9-4d4c-b2ab-5032f93d732a response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
11932024-02-03T08:19:59.090ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
11942024-02-03T08:19:59.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11952024-02-03T08:19:59.090ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
11962024-02-03T08:19:59.090ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
11972024-02-03T08:19:59.090ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
11982024-02-03T08:19:59.090ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
11992024-02-03T08:19:59.090ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
12002024-02-03T08:19:59.090ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12012024-02-03T08:19:59.090ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
12022024-02-03T08:19:59.090ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12032024-02-03T08:19:59.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12042024-02-03T08:19:59.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12052024-02-03T08:19:59.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
12062024-02-03T08:19:59.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12072024-02-03T08:19:59.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12082024-02-03T08:19:59.092ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12092024-02-03T08:19:59.092ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
12102024-02-03T08:19:59.092ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
12112024-02-03T08:19:59.092ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.259179ms iteration = 13
12122024-02-03T08:19:59.093ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
12132024-02-03T08:19:59.093ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "361917b6-1746-4cde-ac7b-3d7a3764a7d6", "content-length": "1", "date": "Sat, 03 Feb 2024 08:19:58 GMT"} })
12142024-02-03T08:19:59.093ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12152024-02-03T08:19:59.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12162024-02-03T08:19:59.094ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b597eb7a-8cb2-470c-a7a6-e7b5421f485c", "content-length": "2", "date": "Sat, 03 Feb 2024 08:19:59 GMT"} })
12172024-02-03T08:19:59.094ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 3.844353ms iteration = 13
12182024-02-03T08:19:59.095ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 4.60333ms iteration = 13
12192024-02-03T08:19:59.095ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
12202024-02-03T08:19:59.095ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12212024-02-03T08:19:59.095ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12222024-02-03T08:19:59.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12232024-02-03T08:19:59.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12242024-02-03T08:19:59.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12252024-02-03T08:19:59.096ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12262024-02-03T08:19:59.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12272024-02-03T08:19:59.097ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.570436ms iteration = 35
12282024-02-03T08:19:59.106ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12292024-02-03T08:19:59.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12302024-02-03T08:19:59.107ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12312024-02-03T08:19:59.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12322024-02-03T08:19:59.108ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12332024-02-03T08:19:59.111ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
12342024-02-03T08:19:59.111ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12352024-02-03T08:19:59.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12362024-02-03T08:19:59.112ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 21.947119ms iteration = 13
12372024-02-03T08:20:01.326ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 792 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = feb176ee-2203-4be9-af38-9e8190f2f023 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
12382024-02-03T08:20:09.090ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
12392024-02-03T08:20:09.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12402024-02-03T08:20:09.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12412024-02-03T08:20:09.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12422024-02-03T08:20:09.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12432024-02-03T08:20:09.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12442024-02-03T08:20:09.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12452024-02-03T08:20:09.096ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.303075ms iteration = 36
12462024-02-03T08:20:11.326ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 500 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = ec1771ec-2d53-41d0-a015-13ffd31a4701 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
12472024-02-03T08:20:19.091ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
12482024-02-03T08:20:19.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12492024-02-03T08:20:19.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12502024-02-03T08:20:19.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12512024-02-03T08:20:19.095ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12522024-02-03T08:20:19.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12532024-02-03T08:20:19.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12542024-02-03T08:20:19.097ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.590378ms iteration = 37
12552024-02-03T08:20:21.326ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 521 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 9d6cde7c-7abf-4898-9e81-dbde04f579a6 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
12562024-02-03T08:20:29.087ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
12572024-02-03T08:20:29.087ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
12582024-02-03T08:20:29.087ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12592024-02-03T08:20:29.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12602024-02-03T08:20:29.088ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
12612024-02-03T08:20:29.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
12622024-02-03T08:20:29.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
12632024-02-03T08:20:29.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12642024-02-03T08:20:29.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12652024-02-03T08:20:29.089ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12662024-02-03T08:20:29.089ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12672024-02-03T08:20:29.089ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
12682024-02-03T08:20:29.089ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
12692024-02-03T08:20:29.089ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
12702024-02-03T08:20:29.089ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12712024-02-03T08:20:29.089ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
12722024-02-03T08:20:29.089ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
12732024-02-03T08:20:29.089ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12742024-02-03T08:20:29.089ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
12752024-02-03T08:20:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12762024-02-03T08:20:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12772024-02-03T08:20:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
12782024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
12792024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
12802024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12812024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
12822024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
12832024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12842024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
12852024-02-03T08:20:29.091ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
12862024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.17\u{b5}s iteration = 8
12872024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
12882024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
12892024-02-03T08:20:29.091ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
12902024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12912024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12922024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12932024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12942024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12952024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
12962024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12972024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12982024-02-03T08:20:29.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12992024-02-03T08:20:29.092ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13002024-02-03T08:20:29.092ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
13012024-02-03T08:20:29.093ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "91b50776-e68d-4d14-ab7d-225cb257bfaa", "content-length": "2", "date": "Sat, 03 Feb 2024 08:20:29 GMT"} })
13022024-02-03T08:20:29.093ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.123738ms iteration = 14
13032024-02-03T08:20:29.093ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13042024-02-03T08:20:29.093ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
13052024-02-03T08:20:29.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13062024-02-03T08:20:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
13072024-02-03T08:20:29.093ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
13082024-02-03T08:20:29.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
13092024-02-03T08:20:29.094ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
13102024-02-03T08:20:29.094ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.50321ms iteration = 14
13112024-02-03T08:20:29.095ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5180ab2e-c147-49fa-8756-810a1a83194f", "content-length": "1", "date": "Sat, 03 Feb 2024 08:20:28 GMT"} })
13122024-02-03T08:20:29.095ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13132024-02-03T08:20:29.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13142024-02-03T08:20:29.095ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
13152024-02-03T08:20:29.095ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13162024-02-03T08:20:29.096ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
13172024-02-03T08:20:29.096ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 7.850087ms iteration = 9
13182024-02-03T08:20:29.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13192024-02-03T08:20:29.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
13202024-02-03T08:20:29.096ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 5.555448ms iteration = 14
13212024-02-03T08:20:29.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
13222024-02-03T08:20:29.096ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
13232024-02-03T08:20:29.097ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
13242024-02-03T08:20:29.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
13252024-02-03T08:20:29.097ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
13262024-02-03T08:20:29.097ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
13272024-02-03T08:20:29.097ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
13282024-02-03T08:20:29.097ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13292024-02-03T08:20:29.097ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13302024-02-03T08:20:29.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13312024-02-03T08:20:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13322024-02-03T08:20:29.098ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
13332024-02-03T08:20:29.098ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 8.302551ms iteration = 9
13342024-02-03T08:20:29.098ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13352024-02-03T08:20:29.099ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13362024-02-03T08:20:29.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13372024-02-03T08:20:29.099ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13382024-02-03T08:20:29.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13392024-02-03T08:20:29.099ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
13402024-02-03T08:20:29.099ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13412024-02-03T08:20:29.099ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
13422024-02-03T08:20:29.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13432024-02-03T08:20:29.100ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 9.560262ms iteration = 38
13442024-02-03T08:20:29.100ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
13452024-02-03T08:20:29.100ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
13462024-02-03T08:20:29.100ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
13472024-02-03T08:20:29.100ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13482024-02-03T08:20:29.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13492024-02-03T08:20:29.101ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13502024-02-03T08:20:29.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13512024-02-03T08:20:29.101ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13522024-02-03T08:20:29.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
13532024-02-03T08:20:29.102ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13542024-02-03T08:20:29.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13552024-02-03T08:20:29.103ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
13562024-02-03T08:20:29.103ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13572024-02-03T08:20:29.104ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
13582024-02-03T08:20:29.104ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
13592024-02-03T08:20:29.104ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13602024-02-03T08:20:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13612024-02-03T08:20:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13622024-02-03T08:20:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
13632024-02-03T08:20:29.104ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13642024-02-03T08:20:29.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13652024-02-03T08:20:29.105ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13662024-02-03T08:20:29.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13672024-02-03T08:20:29.105ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13682024-02-03T08:20:29.105ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
13692024-02-03T08:20:29.105ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 0 version = 2
13702024-02-03T08:20:29.106ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13712024-02-03T08:20:29.106ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13722024-02-03T08:20:29.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13732024-02-03T08:20:29.107ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13742024-02-03T08:20:29.108ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
13752024-02-03T08:20:29.108ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13762024-02-03T08:20:29.108ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
13772024-02-03T08:20:29.109ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13782024-02-03T08:20:29.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13792024-02-03T08:20:29.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13802024-02-03T08:20:29.110ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
13812024-02-03T08:20:29.110ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
13822024-02-03T08:20:29.110ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-03 08:14:50.307019 UTC dns_group = internal generation = 1
13832024-02-03T08:20:29.110ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 22.999458ms iteration = 9
13842024-02-03T08:20:29.111ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
13852024-02-03T08:20:29.111ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13862024-02-03T08:20:29.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13872024-02-03T08:20:29.112ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 1 version = 2
13882024-02-03T08:20:29.112ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
13892024-02-03T08:20:29.112ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-03 08:14:50.600977 UTC dns_group = external generation = 2
13902024-02-03T08:20:29.112ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 22.965538ms iteration = 9
13912024-02-03T08:20:29.113ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13922024-02-03T08:20:29.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13932024-02-03T08:20:29.114ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13942024-02-03T08:20:29.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13952024-02-03T08:20:29.115ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13962024-02-03T08:20:29.118ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-03T08:14:29.510171Z, time_modified: 2024-02-03T08:14:29.510171Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }) }
13972024-02-03T08:20:29.118ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 28.071982ms iteration = 9
13982024-02-03T08:20:29.118ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
13992024-02-03T08:20:29.118ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14002024-02-03T08:20:29.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14012024-02-03T08:20:29.119ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.381995ms iteration = 14
14022024-02-03T08:20:29.157ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
14032024-02-03T08:20:29.161ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
14042024-02-03T08:20:29.216ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
14052024-02-03T08:20:29.220ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
14062024-02-03T08:20:29.221ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7c477c93-31ae-485c-8a2f-7d65e6e9dd26", "date": "Sat, 03 Feb 2024 08:20:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14072024-02-03T08:20:29.221ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14082024-02-03T08:20:29.222ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8b9b340a-84ed-41ab-95ed-33eca96651a2", "date": "Sat, 03 Feb 2024 08:20:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14092024-02-03T08:20:29.222ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14102024-02-03T08:20:29.224ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14112024-02-03T08:20:29.224ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 134.983254ms iteration = 11
14122024-02-03T08:20:29.249ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
14132024-02-03T08:20:29.249ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1a8db47c-400a-4cbc-9fb3-9348404dd789", "date": "Sat, 03 Feb 2024 08:20:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
14142024-02-03T08:20:29.249ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
14152024-02-03T08:20:29.250ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b2e6deb0-5eea-4a89-807c-b1b33f073ab5", "date": "Sat, 03 Feb 2024 08:20:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
14162024-02-03T08:20:29.250ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
14172024-02-03T08:20:29.252ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e99de4f4-1c02-4915-884f-ebc70d80a4af", "date": "Sat, 03 Feb 2024 08:20:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
14182024-02-03T08:20:29.252ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
14192024-02-03T08:20:29.254ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
14202024-02-03T08:20:29.254ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 157.405517ms iteration = 11
14212024-02-03T08:20:31.326ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 493 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 5aa1c1ba-5b1f-4a47-a620-11dc4d916cb3 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
14222024-02-03T08:20:39.091ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
14232024-02-03T08:20:39.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14242024-02-03T08:20:39.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14252024-02-03T08:20:39.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14262024-02-03T08:20:39.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14272024-02-03T08:20:39.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14282024-02-03T08:20:39.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14292024-02-03T08:20:39.096ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.489158ms iteration = 39
14302024-02-03T08:20:41.326ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 552 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 71ce78d9-e3a6-4f26-91d9-182b5f91d168 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
14312024-02-03T08:20:49.092ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
14322024-02-03T08:20:49.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14332024-02-03T08:20:49.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14342024-02-03T08:20:49.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14352024-02-03T08:20:49.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14362024-02-03T08:20:49.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14372024-02-03T08:20:49.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14382024-02-03T08:20:49.097ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.450167ms iteration = 40
14392024-02-03T08:20:51.327ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 499 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 5b5d2539-8e2b-4949-8471-b752ab7709cd response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
14402024-02-03T08:20:59.092ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
14412024-02-03T08:20:59.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14422024-02-03T08:20:59.092ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
14432024-02-03T08:20:59.092ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
14442024-02-03T08:20:59.092ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
14452024-02-03T08:20:59.092ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
14462024-02-03T08:20:59.092ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
14472024-02-03T08:20:59.092ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
14482024-02-03T08:20:59.092ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14492024-02-03T08:20:59.092ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14502024-02-03T08:20:59.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14512024-02-03T08:20:59.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14522024-02-03T08:20:59.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14532024-02-03T08:20:59.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
14542024-02-03T08:20:59.095ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14552024-02-03T08:20:59.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14562024-02-03T08:20:59.096ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
14572024-02-03T08:20:59.096ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
14582024-02-03T08:20:59.096ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2ead9e3c-d8b2-41ac-956b-63567cb1b668", "content-length": "1", "date": "Sat, 03 Feb 2024 08:20:58 GMT"} })
14592024-02-03T08:20:59.096ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14602024-02-03T08:20:59.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14612024-02-03T08:20:59.098ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9f9608a4-d9e9-47fc-b531-5ee8e8044ca0", "content-length": "2", "date": "Sat, 03 Feb 2024 08:20:59 GMT"} })
14622024-02-03T08:20:59.098ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 5.963172ms iteration = 15
14632024-02-03T08:20:59.098ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
14642024-02-03T08:20:59.098ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14652024-02-03T08:20:59.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14662024-02-03T08:20:59.099ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14672024-02-03T08:20:59.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14682024-02-03T08:20:59.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14692024-02-03T08:20:59.100ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 8.15293ms iteration = 15
14702024-02-03T08:20:59.100ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
14712024-02-03T08:20:59.100ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 8.565054ms iteration = 15
14722024-02-03T08:20:59.101ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14732024-02-03T08:20:59.101ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 9.692073ms iteration = 41
14742024-02-03T08:20:59.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14752024-02-03T08:20:59.112ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14762024-02-03T08:20:59.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14772024-02-03T08:20:59.114ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14782024-02-03T08:20:59.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14792024-02-03T08:20:59.114ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14802024-02-03T08:20:59.117ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
14812024-02-03T08:20:59.117ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14822024-02-03T08:20:59.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14832024-02-03T08:20:59.118ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.220116ms iteration = 15
14842024-02-03T08:21:01.327ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 726 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = c6281711-ecba-467f-860b-7e8dc104a92b response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
14852024-02-03T08:21:09.092ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
14862024-02-03T08:21:09.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14872024-02-03T08:21:09.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14882024-02-03T08:21:09.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14892024-02-03T08:21:09.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14902024-02-03T08:21:09.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14912024-02-03T08:21:09.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14922024-02-03T08:21:09.097ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.254155ms iteration = 42
14932024-02-03T08:21:11.328ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 566 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = dd7fd2c7-7cd0-43c4-bdb4-4f891f929c95 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
14942024-02-03T08:21:19.092ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
14952024-02-03T08:21:19.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14962024-02-03T08:21:19.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14972024-02-03T08:21:19.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14982024-02-03T08:21:19.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14992024-02-03T08:21:19.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15002024-02-03T08:21:19.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15012024-02-03T08:21:19.097ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.410277ms iteration = 43
15022024-02-03T08:21:21.327ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 508 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 020c60df-01a1-43f1-97f5-b39d0205336c response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
15032024-02-03T08:21:29.088ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
15042024-02-03T08:21:29.088ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
15052024-02-03T08:21:29.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15062024-02-03T08:21:29.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15072024-02-03T08:21:29.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15082024-02-03T08:21:29.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15092024-02-03T08:21:29.089ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
15102024-02-03T08:21:29.089ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
15112024-02-03T08:21:29.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
15122024-02-03T08:21:29.090ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15132024-02-03T08:21:29.090ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15142024-02-03T08:21:29.091ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
15152024-02-03T08:21:29.091ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
15162024-02-03T08:21:29.091ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
15172024-02-03T08:21:29.091ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15182024-02-03T08:21:29.091ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
15192024-02-03T08:21:29.091ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
15202024-02-03T08:21:29.091ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
15212024-02-03T08:21:29.091ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15222024-02-03T08:21:29.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15232024-02-03T08:21:29.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15242024-02-03T08:21:29.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
15252024-02-03T08:21:29.092ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15262024-02-03T08:21:29.092ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
15272024-02-03T08:21:29.092ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
15282024-02-03T08:21:29.092ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15292024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15302024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
15312024-02-03T08:21:29.093ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
15322024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
15332024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
15342024-02-03T08:21:29.093ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
15352024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15362024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.55\u{b5}s iteration = 9
15372024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
15382024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
15392024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15402024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15412024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
15422024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
15432024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15442024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15452024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15462024-02-03T08:21:29.093ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15472024-02-03T08:21:29.094ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15482024-02-03T08:21:29.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
15492024-02-03T08:21:29.094ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
15502024-02-03T08:21:29.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15512024-02-03T08:21:29.094ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
15522024-02-03T08:21:29.094ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
15532024-02-03T08:21:29.094ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
15542024-02-03T08:21:29.094ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15552024-02-03T08:21:29.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
15562024-02-03T08:21:29.094ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
15572024-02-03T08:21:29.095ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15582024-02-03T08:21:29.095ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d7c01320-6d0c-4a92-9ce9-a4d00e8e2483", "content-length": "2", "date": "Sat, 03 Feb 2024 08:21:29 GMT"} })
15592024-02-03T08:21:29.095ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.069528ms iteration = 16
15602024-02-03T08:21:29.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15612024-02-03T08:21:29.095ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
15622024-02-03T08:21:29.096ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
15632024-02-03T08:21:29.096ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 6.721638ms iteration = 10
15642024-02-03T08:21:29.096ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15652024-02-03T08:21:29.097ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
15662024-02-03T08:21:29.097ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.147425ms iteration = 16
15672024-02-03T08:21:29.097ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "080aba59-99d0-458d-aee6-f001322d8456", "content-length": "1", "date": "Sat, 03 Feb 2024 08:21:28 GMT"} })
15682024-02-03T08:21:29.097ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15692024-02-03T08:21:29.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
15702024-02-03T08:21:29.098ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
15712024-02-03T08:21:29.098ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
15722024-02-03T08:21:29.098ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15732024-02-03T08:21:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15742024-02-03T08:21:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
15752024-02-03T08:21:29.098ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
15762024-02-03T08:21:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
15772024-02-03T08:21:29.099ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 6.090622ms iteration = 16
15782024-02-03T08:21:29.099ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15792024-02-03T08:21:29.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15802024-02-03T08:21:29.100ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
15812024-02-03T08:21:29.100ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 8.511503ms iteration = 10
15822024-02-03T08:21:29.100ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
15832024-02-03T08:21:29.101ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15842024-02-03T08:21:29.101ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
15852024-02-03T08:21:29.101ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15862024-02-03T08:21:29.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15872024-02-03T08:21:29.101ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
15882024-02-03T08:21:29.101ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
15892024-02-03T08:21:29.101ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
15902024-02-03T08:21:29.101ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15912024-02-03T08:21:29.101ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15922024-02-03T08:21:29.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15932024-02-03T08:21:29.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15942024-02-03T08:21:29.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15952024-02-03T08:21:29.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15962024-02-03T08:21:29.102ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15972024-02-03T08:21:29.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15982024-02-03T08:21:29.103ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15992024-02-03T08:21:29.103ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16002024-02-03T08:21:29.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16012024-02-03T08:21:29.103ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 10.684963ms iteration = 44
16022024-02-03T08:21:29.103ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
16032024-02-03T08:21:29.103ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
16042024-02-03T08:21:29.103ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16052024-02-03T08:21:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16062024-02-03T08:21:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
16072024-02-03T08:21:29.104ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16082024-02-03T08:21:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16092024-02-03T08:21:29.105ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16102024-02-03T08:21:29.105ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16112024-02-03T08:21:29.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16122024-02-03T08:21:29.106ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16132024-02-03T08:21:29.107ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
16142024-02-03T08:21:29.107ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16152024-02-03T08:21:29.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16162024-02-03T08:21:29.108ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
16172024-02-03T08:21:29.108ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 0 version = 2
16182024-02-03T08:21:29.108ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16192024-02-03T08:21:29.109ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
16202024-02-03T08:21:29.109ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16212024-02-03T08:21:29.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16222024-02-03T08:21:29.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16232024-02-03T08:21:29.109ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
16242024-02-03T08:21:29.109ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16252024-02-03T08:21:29.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16262024-02-03T08:21:29.110ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16272024-02-03T08:21:29.111ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
16282024-02-03T08:21:29.111ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
16292024-02-03T08:21:29.111ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-03 08:14:50.307019 UTC dns_group = internal generation = 1
16302024-02-03T08:21:29.111ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 23.115ms iteration = 10
16312024-02-03T08:21:29.113ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
16322024-02-03T08:21:29.113ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16332024-02-03T08:21:29.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16342024-02-03T08:21:29.114ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16352024-02-03T08:21:29.114ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 1 version = 2
16362024-02-03T08:21:29.114ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
16372024-02-03T08:21:29.114ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-03 08:14:50.600977 UTC dns_group = external generation = 2
16382024-02-03T08:21:29.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16392024-02-03T08:21:29.114ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 22.931168ms iteration = 10
16402024-02-03T08:21:29.116ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16412024-02-03T08:21:29.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16422024-02-03T08:21:29.117ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16432024-02-03T08:21:29.120ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
16442024-02-03T08:21:29.120ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16452024-02-03T08:21:29.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16462024-02-03T08:21:29.120ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-03T08:14:29.510171Z, time_modified: 2024-02-03T08:14:29.510171Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
16472024-02-03T08:21:29.120ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 28.495605ms iteration = 10
16482024-02-03T08:21:29.121ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.433855ms iteration = 16
16492024-02-03T08:21:29.160ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
16502024-02-03T08:21:29.169ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
16512024-02-03T08:21:29.220ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
16522024-02-03T08:21:29.221ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "933c6e14-639a-4b6e-832a-44b131cde5f4", "date": "Sat, 03 Feb 2024 08:21:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16532024-02-03T08:21:29.221ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16542024-02-03T08:21:29.222ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d5e88c15-7aa9-4d14-876f-e22f53dff223", "date": "Sat, 03 Feb 2024 08:21:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16552024-02-03T08:21:29.222ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16562024-02-03T08:21:29.223ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16572024-02-03T08:21:29.223ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 131.302032ms iteration = 12
16582024-02-03T08:21:29.227ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
16592024-02-03T08:21:29.258ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
16602024-02-03T08:21:29.258ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "12c2d162-0256-4ebb-b9f6-9afe7256876f", "date": "Sat, 03 Feb 2024 08:21:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16612024-02-03T08:21:29.258ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16622024-02-03T08:21:29.260ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "18e4559c-ae01-4281-94ab-c776b533677e", "date": "Sat, 03 Feb 2024 08:21:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16632024-02-03T08:21:29.260ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16642024-02-03T08:21:29.262ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6b0359b8-fe10-455d-8c2c-ed8ad39b6f8c", "date": "Sat, 03 Feb 2024 08:21:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16652024-02-03T08:21:29.262ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16662024-02-03T08:21:29.264ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16672024-02-03T08:21:29.264ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 165.951771ms iteration = 12
16682024-02-03T08:21:31.328ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1029 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 2fe7ac7a-99e4-4334-8552-ab7aaa899d44 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
16692024-02-03T08:21:37.647ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:43188
16702024-02-03T08:21:37.657ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309 instance_state = InstanceRuntimeState { propolis_id: Some(46071dbd-3ecb-4ffc-a312-f5439f87a713), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-03T08:21:29.080966435Z } propolis_id = 46071dbd-3ecb-4ffc-a312-f5439f87a713 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-02-03T08:21:37.644097843Z }
16712024-02-03T08:21:37.659ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 9d718883-c27c-4891-a763-a11a44a3ece7 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16722024-02-03T08:21:37.660ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 9d718883-c27c-4891-a763-a11a44a3ece7 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16732024-02-03T08:21:37.663ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 9d718883-c27c-4891-a763-a11a44a3ece7 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16742024-02-03T08:21:37.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 9d718883-c27c-4891-a763-a11a44a3ece7 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16752024-02-03T08:21:37.665ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 9d718883-c27c-4891-a763-a11a44a3ece7 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16762024-02-03T08:21:37.665ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 9d718883-c27c-4891-a763-a11a44a3ece7 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16772024-02-03T08:21:37.666ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 9d718883-c27c-4891-a763-a11a44a3ece7 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16782024-02-03T08:21:37.666ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 9d718883-c27c-4891-a763-a11a44a3ece7 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16792024-02-03T08:21:37.667ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 9d718883-c27c-4891-a763-a11a44a3ece7 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16802024-02-03T08:21:37.667ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 9d718883-c27c-4891-a763-a11a44a3ece7 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16812024-02-03T08:21:37.668ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 9d718883-c27c-4891-a763-a11a44a3ece7 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16822024-02-03T08:21:37.668ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 9d718883-c27c-4891-a763-a11a44a3ece7 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16832024-02-03T08:21:37.669ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 9d718883-c27c-4891-a763-a11a44a3ece7 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16842024-02-03T08:21:37.672ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 9d718883-c27c-4891-a763-a11a44a3ece7 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: aef853c2-ff00-447f-ab12-e3d54ae65e46, lookup_type: ById(aef853c2-ff00-447f-ab12-e3d54ae65e46) }, key: f1285c0f-1d24-4d7a-bfec-890131a2f6c1, lookup_type: ById(f1285c0f-1d24-4d7a-bfec-890131a2f6c1) }, key: 987e0de3-b3ef-402a-b81f-d555ebc29309, lookup_type: ById(987e0de3-b3ef-402a-b81f-d555ebc29309) } result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16852024-02-03T08:21:37.672ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309
16862024-02-03T08:21:37.689ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309 instance_updated = false propolis_id = 46071dbd-3ecb-4ffc-a312-f5439f87a713 vmm_updated = true
16872024-02-03T08:21:37.689ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 42240 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 9d718883-c27c-4891-a763-a11a44a3ece7 response_code = 204 uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16882024-02-03T08:21:37.690ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309 instance_state = InstanceRuntimeState { propolis_id: Some(46071dbd-3ecb-4ffc-a312-f5439f87a713), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-03T08:21:29.080966435Z } propolis_id = 46071dbd-3ecb-4ffc-a312-f5439f87a713 vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-02-03T08:21:37.689958359Z }
16892024-02-03T08:21:37.690ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = e5b43066-c9b3-4f7c-a600-6038651684b5 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16902024-02-03T08:21:37.690ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = e5b43066-c9b3-4f7c-a600-6038651684b5 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16912024-02-03T08:21:37.692ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = e5b43066-c9b3-4f7c-a600-6038651684b5 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16922024-02-03T08:21:37.692ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = e5b43066-c9b3-4f7c-a600-6038651684b5 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16932024-02-03T08:21:37.693ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = e5b43066-c9b3-4f7c-a600-6038651684b5 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16942024-02-03T08:21:37.693ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = e5b43066-c9b3-4f7c-a600-6038651684b5 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16952024-02-03T08:21:37.694ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = e5b43066-c9b3-4f7c-a600-6038651684b5 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16962024-02-03T08:21:37.695ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = e5b43066-c9b3-4f7c-a600-6038651684b5 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16972024-02-03T08:21:37.695ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = e5b43066-c9b3-4f7c-a600-6038651684b5 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16982024-02-03T08:21:37.696ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = e5b43066-c9b3-4f7c-a600-6038651684b5 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
16992024-02-03T08:21:37.696ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = e5b43066-c9b3-4f7c-a600-6038651684b5 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17002024-02-03T08:21:37.696ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = e5b43066-c9b3-4f7c-a600-6038651684b5 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17012024-02-03T08:21:37.697ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = e5b43066-c9b3-4f7c-a600-6038651684b5 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17022024-02-03T08:21:37.700ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = e5b43066-c9b3-4f7c-a600-6038651684b5 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: aef853c2-ff00-447f-ab12-e3d54ae65e46, lookup_type: ById(aef853c2-ff00-447f-ab12-e3d54ae65e46) }, key: f1285c0f-1d24-4d7a-bfec-890131a2f6c1, lookup_type: ById(f1285c0f-1d24-4d7a-bfec-890131a2f6c1) }, key: 987e0de3-b3ef-402a-b81f-d555ebc29309, lookup_type: ById(987e0de3-b3ef-402a-b81f-d555ebc29309) } result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17032024-02-03T08:21:37.700ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309
17042024-02-03T08:21:37.711ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309 instance_updated = false propolis_id = 46071dbd-3ecb-4ffc-a312-f5439f87a713 vmm_updated = true
17052024-02-03T08:21:37.711ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 21003 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = e5b43066-c9b3-4f7c-a600-6038651684b5 response_code = 204 uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17062024-02-03T08:21:39.092ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
17072024-02-03T08:21:39.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17082024-02-03T08:21:39.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17092024-02-03T08:21:39.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17102024-02-03T08:21:39.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17112024-02-03T08:21:39.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17122024-02-03T08:21:39.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17132024-02-03T08:21:39.097ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.394376ms iteration = 45
17142024-02-03T08:21:41.328ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 637 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 445beff3-baa6-4e8d-8a9f-33f92967ea0c response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
17152024-02-03T08:21:49.093ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
17162024-02-03T08:21:49.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17172024-02-03T08:21:49.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17182024-02-03T08:21:49.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17192024-02-03T08:21:49.097ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17202024-02-03T08:21:49.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17212024-02-03T08:21:49.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17222024-02-03T08:21:49.099ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.037922ms iteration = 46
17232024-02-03T08:21:51.329ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1509 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 39a52120-877e-4fcb-9c5d-26adb560cd0b response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
17242024-02-03T08:21:54.751ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309 instance_state = InstanceRuntimeState { propolis_id: Some(46071dbd-3ecb-4ffc-a312-f5439f87a713), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-03T08:21:29.080966435Z } propolis_id = 46071dbd-3ecb-4ffc-a312-f5439f87a713 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-02-03T08:21:54.751177205Z }
17252024-02-03T08:21:54.751ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60598897-3996-440c-bcff-913493654907 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17262024-02-03T08:21:54.751ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60598897-3996-440c-bcff-913493654907 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17272024-02-03T08:21:54.757ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60598897-3996-440c-bcff-913493654907 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17282024-02-03T08:21:54.758ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60598897-3996-440c-bcff-913493654907 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17292024-02-03T08:21:54.760ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60598897-3996-440c-bcff-913493654907 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17302024-02-03T08:21:54.760ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60598897-3996-440c-bcff-913493654907 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17312024-02-03T08:21:54.761ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60598897-3996-440c-bcff-913493654907 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17322024-02-03T08:21:54.761ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60598897-3996-440c-bcff-913493654907 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17332024-02-03T08:21:54.762ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60598897-3996-440c-bcff-913493654907 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17342024-02-03T08:21:54.762ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60598897-3996-440c-bcff-913493654907 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17352024-02-03T08:21:54.763ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60598897-3996-440c-bcff-913493654907 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17362024-02-03T08:21:54.763ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60598897-3996-440c-bcff-913493654907 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17372024-02-03T08:21:54.764ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60598897-3996-440c-bcff-913493654907 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17382024-02-03T08:21:54.767ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60598897-3996-440c-bcff-913493654907 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: aef853c2-ff00-447f-ab12-e3d54ae65e46, lookup_type: ById(aef853c2-ff00-447f-ab12-e3d54ae65e46) }, key: f1285c0f-1d24-4d7a-bfec-890131a2f6c1, lookup_type: ById(f1285c0f-1d24-4d7a-bfec-890131a2f6c1) }, key: 987e0de3-b3ef-402a-b81f-d555ebc29309, lookup_type: ById(987e0de3-b3ef-402a-b81f-d555ebc29309) } result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17392024-02-03T08:21:54.767ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309
17402024-02-03T08:21:54.804ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309 instance_updated = false propolis_id = 46071dbd-3ecb-4ffc-a312-f5439f87a713 vmm_updated = true
17412024-02-03T08:21:54.804ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 53506 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60598897-3996-440c-bcff-913493654907 response_code = 204 uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17422024-02-03T08:21:54.805ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309 instance_state = InstanceRuntimeState { propolis_id: Some(46071dbd-3ecb-4ffc-a312-f5439f87a713), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-03T08:21:29.080966435Z } propolis_id = 46071dbd-3ecb-4ffc-a312-f5439f87a713 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-02-03T08:21:54.805668365Z }
17432024-02-03T08:21:54.805ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 609cc75f-aa52-48fe-8e4e-876c8ce4462f roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17442024-02-03T08:21:54.806ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 609cc75f-aa52-48fe-8e4e-876c8ce4462f resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17452024-02-03T08:21:54.807ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 609cc75f-aa52-48fe-8e4e-876c8ce4462f roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17462024-02-03T08:21:54.807ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 609cc75f-aa52-48fe-8e4e-876c8ce4462f resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17472024-02-03T08:21:54.808ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 609cc75f-aa52-48fe-8e4e-876c8ce4462f roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17482024-02-03T08:21:54.808ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 609cc75f-aa52-48fe-8e4e-876c8ce4462f resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17492024-02-03T08:21:54.809ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 609cc75f-aa52-48fe-8e4e-876c8ce4462f roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17502024-02-03T08:21:54.810ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 609cc75f-aa52-48fe-8e4e-876c8ce4462f resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17512024-02-03T08:21:54.810ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 609cc75f-aa52-48fe-8e4e-876c8ce4462f roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17522024-02-03T08:21:54.810ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 609cc75f-aa52-48fe-8e4e-876c8ce4462f resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17532024-02-03T08:21:54.811ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 609cc75f-aa52-48fe-8e4e-876c8ce4462f roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17542024-02-03T08:21:54.811ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 609cc75f-aa52-48fe-8e4e-876c8ce4462f resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17552024-02-03T08:21:54.812ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 609cc75f-aa52-48fe-8e4e-876c8ce4462f roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17562024-02-03T08:21:54.815ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 609cc75f-aa52-48fe-8e4e-876c8ce4462f resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: aef853c2-ff00-447f-ab12-e3d54ae65e46, lookup_type: ById(aef853c2-ff00-447f-ab12-e3d54ae65e46) }, key: f1285c0f-1d24-4d7a-bfec-890131a2f6c1, lookup_type: ById(f1285c0f-1d24-4d7a-bfec-890131a2f6c1) }, key: 987e0de3-b3ef-402a-b81f-d555ebc29309, lookup_type: ById(987e0de3-b3ef-402a-b81f-d555ebc29309) } result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17572024-02-03T08:21:54.815ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309
17582024-02-03T08:21:54.838ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309 instance_updated = false propolis_id = 46071dbd-3ecb-4ffc-a312-f5439f87a713 vmm_updated = true
17592024-02-03T08:21:54.838ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 33048 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 609cc75f-aa52-48fe-8e4e-876c8ce4462f response_code = 204 uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17602024-02-03T08:21:57.230ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309 instance_state = InstanceRuntimeState { propolis_id: None, dst_propolis_id: None, migration_id: None, gen: Generation(5), time_updated: 2024-02-03T08:21:54.952254669Z } propolis_id = 46071dbd-3ecb-4ffc-a312-f5439f87a713 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-02-03T08:21:54.952254669Z }
17612024-02-03T08:21:57.230ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17622024-02-03T08:21:57.231ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17632024-02-03T08:21:57.233ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17642024-02-03T08:21:57.233ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17652024-02-03T08:21:57.234ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17662024-02-03T08:21:57.234ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17672024-02-03T08:21:57.235ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17682024-02-03T08:21:57.235ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17692024-02-03T08:21:57.236ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17702024-02-03T08:21:57.236ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17712024-02-03T08:21:57.237ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17722024-02-03T08:21:57.237ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17732024-02-03T08:21:57.238ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17742024-02-03T08:21:57.242ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: aef853c2-ff00-447f-ab12-e3d54ae65e46, lookup_type: ById(aef853c2-ff00-447f-ab12-e3d54ae65e46) }, key: f1285c0f-1d24-4d7a-bfec-890131a2f6c1, lookup_type: ById(f1285c0f-1d24-4d7a-bfec-890131a2f6c1) }, key: 987e0de3-b3ef-402a-b81f-d555ebc29309, lookup_type: ById(987e0de3-b3ef-402a-b81f-d555ebc29309) } result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17752024-02-03T08:21:57.242ZINFOnexus: instance cleared its Propolis ID, cleaning network config file = nexus/src/app/instance_network.rs:739 instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309 propolis_id = Some(46071dbd-3ecb-4ffc-a312-f5439f87a713)
17762024-02-03T08:21:57.242ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17772024-02-03T08:21:57.242ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17782024-02-03T08:21:57.243ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17792024-02-03T08:21:57.243ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17802024-02-03T08:21:57.244ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17812024-02-03T08:21:57.244ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17822024-02-03T08:21:57.245ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17832024-02-03T08:21:57.245ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17842024-02-03T08:21:57.246ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17852024-02-03T08:21:57.246ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17862024-02-03T08:21:57.247ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17872024-02-03T08:21:57.247ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17882024-02-03T08:21:57.248ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17892024-02-03T08:21:57.250ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: aef853c2-ff00-447f-ab12-e3d54ae65e46, lookup_type: ById(aef853c2-ff00-447f-ab12-e3d54ae65e46) }, key: f1285c0f-1d24-4d7a-bfec-890131a2f6c1, lookup_type: ById(f1285c0f-1d24-4d7a-bfec-890131a2f6c1) }, key: 987e0de3-b3ef-402a-b81f-d555ebc29309, lookup_type: ById(987e0de3-b3ef-402a-b81f-d555ebc29309) } result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17902024-02-03T08:21:57.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17912024-02-03T08:21:57.251ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17922024-02-03T08:21:57.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17932024-02-03T08:21:57.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17942024-02-03T08:21:57.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17952024-02-03T08:21:57.253ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17962024-02-03T08:21:57.253ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17972024-02-03T08:21:57.256ZDEBGnexus (dropshot_internal): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: aef853c2-ff00-447f-ab12-e3d54ae65e46, lookup_type: ById(aef853c2-ff00-447f-ab12-e3d54ae65e46) }, key: f1285c0f-1d24-4d7a-bfec-890131a2f6c1, lookup_type: ById(f1285c0f-1d24-4d7a-bfec-890131a2f6c1) }, key: 987e0de3-b3ef-402a-b81f-d555ebc29309, lookup_type: ById(987e0de3-b3ef-402a-b81f-d555ebc29309) } result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17982024-02-03T08:21:57.256ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
17992024-02-03T08:21:57.256ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
18002024-02-03T08:21:57.260ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
18012024-02-03T08:21:57.261ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18022024-02-03T08:21:57.261ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
18032024-02-03T08:21:57.262ZDEBGnexus (InstanceAllocator): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
18042024-02-03T08:21:57.262ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
18052024-02-03T08:21:57.262ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18062024-02-03T08:21:57.264ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
18072024-02-03T08:21:57.265ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18082024-02-03T08:21:57.266ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
18092024-02-03T08:21:57.266ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18102024-02-03T08:21:57.267ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
18112024-02-03T08:21:57.268ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
18122024-02-03T08:21:57.299ZDEBGnexus: client request SledAgent = 139fb5ac-06c6-4816-a9fe-3967ecbe5f17 body = Some(Body) method = DELETE uri = http://[fd00:1122:3344:101::1]:12345/v2p/c7febeb3-bb16-4c32-9e83-def07003a689
18132024-02-03T08:21:57.306ZDEBGnexus: client response SledAgent = 139fb5ac-06c6-4816-a9fe-3967ecbe5f17 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p/c7febeb3-bb16-4c32-9e83-def07003a689", query: None, fragment: None }, status: 204, headers: {"x-request-id": "26457b66-3572-4f37-92b4-35fd6529302e", "date": "Sat, 03 Feb 2024 08:21:57 GMT"} })
18142024-02-03T08:21:57.307ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:519 instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309
18152024-02-03T08:21:57.307ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
18162024-02-03T08:21:57.307ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
18172024-02-03T08:21:57.311ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
18182024-02-03T08:21:57.311ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
18192024-02-03T08:21:57.318ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:609
18202024-02-03T08:21:57.318ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
18212024-02-03T08:21:57.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
18222024-02-03T08:21:57.319ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:609
18232024-02-03T08:21:57.319ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
18242024-02-03T08:21:57.319ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18252024-02-03T08:21:57.320ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
18262024-02-03T08:21:57.321ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
18272024-02-03T08:21:57.321ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
18282024-02-03T08:21:57.321ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18292024-02-03T08:21:57.327ZDEBGnexus: notifying dendrite of updates instance_id = Some(987e0de3-b3ef-402a-b81f-d555ebc29309) switch = switch0
18302024-02-03T08:21:57.328ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
18312024-02-03T08:21:57.328ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/trigger", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e48c6618-bb2c-456f-9c90-ef77f78adf19", "content-length": "4", "date": "Sat, 03 Feb 2024 08:21:56 GMT"} })
18322024-02-03T08:21:57.328ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
18332024-02-03T08:21:57.329ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18342024-02-03T08:21:57.329ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
18352024-02-03T08:21:57.330ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
18362024-02-03T08:21:57.330ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
18372024-02-03T08:21:57.331ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18382024-02-03T08:21:57.334ZDEBGnexus: notifying dendrite of updates instance_id = Some(987e0de3-b3ef-402a-b81f-d555ebc29309) switch = switch0
18392024-02-03T08:21:57.334ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
18402024-02-03T08:21:57.334ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/trigger", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ab1d946d-b8c8-408c-b3f2-80b9d59285f5", "content-length": "4", "date": "Sat, 03 Feb 2024 08:21:56 GMT"} })
18412024-02-03T08:21:57.334ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
18422024-02-03T08:21:57.335ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
18432024-02-03T08:21:57.408ZDEBGnexus: deleted metric producer assignment collector_id = 1d9cac27-6d1e-47a5-891e-117fa49f58e6 producer_id = 987e0de3-b3ef-402a-b81f-d555ebc29309
18442024-02-03T08:21:57.441ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:374 id = 987e0de3-b3ef-402a-b81f-d555ebc29309
18452024-02-03T08:21:57.441ZDEBGnexus: client request body = None method = DELETE oximeter-collector = 987e0de3-b3ef-402a-b81f-d555ebc29309 uri = http://[fd00:1122:3344:101::d]:12223/producers/987e0de3-b3ef-402a-b81f-d555ebc29309
18462024-02-03T08:21:57.441ZDEBGnexus: client response oximeter-collector = 987e0de3-b3ef-402a-b81f-d555ebc29309 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers/987e0de3-b3ef-402a-b81f-d555ebc29309", query: None, fragment: None }, status: 204, headers: {"x-request-id": "13103a3a-59d9-4972-a1ad-3117559c2822", "date": "Sat, 03 Feb 2024 08:21:57 GMT"} })
18472024-02-03T08:21:57.441ZDEBGnexus: successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = 1d9cac27-6d1e-47a5-891e-117fa49f58e6 producer_id = 987e0de3-b3ef-402a-b81f-d555ebc29309
18482024-02-03T08:21:57.470ZINFOnexus: vmm is terminated, cleaning up resources file = nexus/src/app/instance.rs:1585 instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309 propolis_id = 46071dbd-3ecb-4ffc-a312-f5439f87a713
18492024-02-03T08:21:57.470ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
18502024-02-03T08:21:57.470ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
18512024-02-03T08:21:57.501ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 roles = RoleSet { roles: {} } uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
18522024-02-03T08:21:57.501ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 resource = Database result = Ok(()) uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
18532024-02-03T08:21:57.508ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = 987e0de3-b3ef-402a-b81f-d555ebc29309 instance_updated = true propolis_id = 46071dbd-3ecb-4ffc-a312-f5439f87a713 vmm_updated = true
18542024-02-03T08:21:57.508ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 277769 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:43188 req_id = 60ed560e-bade-47a1-b2e0-70e3646f7c84 response_code = 204 uri = /instances/987e0de3-b3ef-402a-b81f-d555ebc29309
18552024-02-03T08:21:59.093ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
18562024-02-03T08:21:59.093ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
18572024-02-03T08:21:59.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18582024-02-03T08:21:59.093ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
18592024-02-03T08:21:59.093ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
18602024-02-03T08:21:59.093ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
18612024-02-03T08:21:59.093ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
18622024-02-03T08:21:59.093ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
18632024-02-03T08:21:59.093ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
18642024-02-03T08:21:59.093ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18652024-02-03T08:21:59.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
18662024-02-03T08:21:59.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18672024-02-03T08:21:59.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
18682024-02-03T08:21:59.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18692024-02-03T08:21:59.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18702024-02-03T08:21:59.094ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18712024-02-03T08:21:59.095ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
18722024-02-03T08:21:59.095ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
18732024-02-03T08:21:59.095ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "45a50dd2-c6ee-47ef-8479-3eb6d82f77ee", "content-length": "2", "date": "Sat, 03 Feb 2024 08:21:59 GMT"} })
18742024-02-03T08:21:59.095ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.30218ms iteration = 17
18752024-02-03T08:21:59.095ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cd06f92a-a9c5-418e-b80f-22559d5e10e6", "content-length": "2", "date": "Sat, 03 Feb 2024 08:21:58 GMT"} })
18762024-02-03T08:21:59.095ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
18772024-02-03T08:21:59.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
18782024-02-03T08:21:59.096ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
18792024-02-03T08:21:59.096ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.012846ms iteration = 17
18802024-02-03T08:21:59.096ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.45758ms iteration = 17
18812024-02-03T08:21:59.098ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18822024-02-03T08:21:59.098ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18832024-02-03T08:21:59.098ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
18842024-02-03T08:21:59.098ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18852024-02-03T08:21:59.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18862024-02-03T08:21:59.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18872024-02-03T08:21:59.099ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18882024-02-03T08:21:59.099ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.413516ms iteration = 47
18892024-02-03T08:21:59.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18902024-02-03T08:21:59.115ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18912024-02-03T08:21:59.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18922024-02-03T08:21:59.116ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18932024-02-03T08:21:59.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18942024-02-03T08:21:59.117ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18952024-02-03T08:21:59.120ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
18962024-02-03T08:21:59.120ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18972024-02-03T08:21:59.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18982024-02-03T08:21:59.121ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.245044ms iteration = 17
18992024-02-03T08:22:01.328ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 879 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 7198c05d-0041-44f8-8658-468530889d0c response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
19002024-02-03T08:22:09.093ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
19012024-02-03T08:22:09.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19022024-02-03T08:22:09.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19032024-02-03T08:22:09.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19042024-02-03T08:22:09.097ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19052024-02-03T08:22:09.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19062024-02-03T08:22:09.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19072024-02-03T08:22:09.098ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.670299ms iteration = 48
19082024-02-03T08:22:11.329ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1050 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = d0ef72ac-75b5-4576-a901-0a6656534a40 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
19092024-02-03T08:22:19.093ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
19102024-02-03T08:22:19.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19112024-02-03T08:22:19.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19122024-02-03T08:22:19.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19132024-02-03T08:22:19.097ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19142024-02-03T08:22:19.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19152024-02-03T08:22:19.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19162024-02-03T08:22:19.098ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.347486ms iteration = 49
19172024-02-03T08:22:21.329ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 621 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 67798a15-3cd3-48a2-9820-82e701023957 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
19182024-02-03T08:22:29.089ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
19192024-02-03T08:22:29.089ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
19202024-02-03T08:22:29.089ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19212024-02-03T08:22:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19222024-02-03T08:22:29.090ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
19232024-02-03T08:22:29.090ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
19242024-02-03T08:22:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
19252024-02-03T08:22:29.090ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19262024-02-03T08:22:29.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19272024-02-03T08:22:29.092ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19282024-02-03T08:22:29.092ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
19292024-02-03T08:22:29.092ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19302024-02-03T08:22:29.092ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
19312024-02-03T08:22:29.092ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
19322024-02-03T08:22:29.092ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
19332024-02-03T08:22:29.092ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19342024-02-03T08:22:29.092ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
19352024-02-03T08:22:29.092ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
19362024-02-03T08:22:29.092ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19372024-02-03T08:22:29.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
19382024-02-03T08:22:29.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19392024-02-03T08:22:29.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19402024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
19412024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
19422024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19432024-02-03T08:22:29.093ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
19442024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
19452024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
19462024-02-03T08:22:29.093ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
19472024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
19482024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 7.431\u{b5}s iteration = 10
19492024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
19502024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19512024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
19522024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19532024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19542024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19552024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
19562024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19572024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19582024-02-03T08:22:29.093ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19592024-02-03T08:22:29.094ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19602024-02-03T08:22:29.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19612024-02-03T08:22:29.094ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19622024-02-03T08:22:29.095ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19632024-02-03T08:22:29.095ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
19642024-02-03T08:22:29.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19652024-02-03T08:22:29.095ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
19662024-02-03T08:22:29.095ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
19672024-02-03T08:22:29.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
19682024-02-03T08:22:29.096ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
19692024-02-03T08:22:29.096ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.943446ms iteration = 18
19702024-02-03T08:22:29.096ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
19712024-02-03T08:22:29.097ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7f3d0085-67bb-4fa3-acaa-a504ac48db4c", "content-length": "2", "date": "Sat, 03 Feb 2024 08:22:29 GMT"} })
19722024-02-03T08:22:29.097ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 3.970965ms iteration = 18
19732024-02-03T08:22:29.097ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
19742024-02-03T08:22:29.097ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 7.121552ms iteration = 11
19752024-02-03T08:22:29.097ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "af4272ca-7de1-4225-b650-edaa2c4de78f", "content-length": "2", "date": "Sat, 03 Feb 2024 08:22:28 GMT"} })
19762024-02-03T08:22:29.097ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19772024-02-03T08:22:29.097ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
19782024-02-03T08:22:29.097ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
19792024-02-03T08:22:29.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19802024-02-03T08:22:29.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
19812024-02-03T08:22:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19822024-02-03T08:22:29.098ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
19832024-02-03T08:22:29.098ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19842024-02-03T08:22:29.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19852024-02-03T08:22:29.099ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 5.687209ms iteration = 18
19862024-02-03T08:22:29.099ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
19872024-02-03T08:22:29.099ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
19882024-02-03T08:22:29.099ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
19892024-02-03T08:22:29.099ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19902024-02-03T08:22:29.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19912024-02-03T08:22:29.099ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
19922024-02-03T08:22:29.099ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 7.066311ms iteration = 11
19932024-02-03T08:22:29.099ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19942024-02-03T08:22:29.099ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
19952024-02-03T08:22:29.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19962024-02-03T08:22:29.100ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19972024-02-03T08:22:29.100ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19982024-02-03T08:22:29.101ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19992024-02-03T08:22:29.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20002024-02-03T08:22:29.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20012024-02-03T08:22:29.101ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20022024-02-03T08:22:29.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
20032024-02-03T08:22:29.102ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20042024-02-03T08:22:29.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20052024-02-03T08:22:29.102ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 9.329111ms iteration = 50
20062024-02-03T08:22:29.102ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
20072024-02-03T08:22:29.102ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
20082024-02-03T08:22:29.102ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
20092024-02-03T08:22:29.102ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20102024-02-03T08:22:29.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20112024-02-03T08:22:29.103ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20122024-02-03T08:22:29.103ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20132024-02-03T08:22:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20142024-02-03T08:22:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20152024-02-03T08:22:29.104ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20162024-02-03T08:22:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
20172024-02-03T08:22:29.104ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20182024-02-03T08:22:29.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20192024-02-03T08:22:29.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20202024-02-03T08:22:29.105ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
20212024-02-03T08:22:29.105ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
20222024-02-03T08:22:29.105ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20232024-02-03T08:22:29.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20242024-02-03T08:22:29.106ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20252024-02-03T08:22:29.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20262024-02-03T08:22:29.106ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20272024-02-03T08:22:29.107ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20282024-02-03T08:22:29.109ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20292024-02-03T08:22:29.109ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20302024-02-03T08:22:29.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20312024-02-03T08:22:29.110ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
20322024-02-03T08:22:29.110ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 0 version = 2
20332024-02-03T08:22:29.110ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20342024-02-03T08:22:29.110ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
20352024-02-03T08:22:29.110ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20362024-02-03T08:22:29.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20372024-02-03T08:22:29.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20382024-02-03T08:22:29.111ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20392024-02-03T08:22:29.112ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20402024-02-03T08:22:29.112ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20412024-02-03T08:22:29.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20422024-02-03T08:22:29.114ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = cecb2399-c713-4c1a-9386-b684a5c54f62 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
20432024-02-03T08:22:29.114ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
20442024-02-03T08:22:29.114ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-03 08:14:50.307019 UTC dns_group = internal generation = 1
20452024-02-03T08:22:29.114ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 24.866725ms iteration = 11
20462024-02-03T08:22:29.115ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-03T08:14:29.510171Z, time_modified: 2024-02-03T08:14:29.510171Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: aef853c2-ff00-447f-ab12-e3d54ae65e46, db_silo: Silo { identity: SiloIdentity { id: aef853c2-ff00-447f-ab12-e3d54ae65e46, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T08:14:50.579231Z, time_modified: 2024-02-03T08:14:50.579231Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "86d37c8cf5532d0426858203d1d8079a6cdaf38653525d2217e6e3c196e23752" }] }) }
20472024-02-03T08:22:29.115ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 23.476133ms iteration = 11
20482024-02-03T08:22:29.115ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20492024-02-03T08:22:29.115ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20502024-02-03T08:22:29.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20512024-02-03T08:22:29.117ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = c706d714-32bc-4773-bf31-c4721293a98c dns_zone_name = oxide.test found_so_far = 1 version = 2
20522024-02-03T08:22:29.117ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
20532024-02-03T08:22:29.117ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-03 08:14:50.600977 UTC dns_group = external generation = 2
20542024-02-03T08:22:29.117ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 24.918285ms iteration = 11
20552024-02-03T08:22:29.119ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20562024-02-03T08:22:29.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20572024-02-03T08:22:29.120ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20582024-02-03T08:22:29.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20592024-02-03T08:22:29.121ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20602024-02-03T08:22:29.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
20612024-02-03T08:22:29.125ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20622024-02-03T08:22:29.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20632024-02-03T08:22:29.126ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 32.864053ms iteration = 18
20642024-02-03T08:22:29.133ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
20652024-02-03T08:22:29.141ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
20662024-02-03T08:22:29.171ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
20672024-02-03T08:22:29.171ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3e493a3b-7c34-41b7-af9c-d25667cb74ac", "date": "Sat, 03 Feb 2024 08:22:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
20682024-02-03T08:22:29.171ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
20692024-02-03T08:22:29.173ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0d1ce766-07d4-4afc-b910-04cf3a621f13", "date": "Sat, 03 Feb 2024 08:22:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
20702024-02-03T08:22:29.173ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
20712024-02-03T08:22:29.174ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
20722024-02-03T08:22:29.174ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 82.31377ms iteration = 13
20732024-02-03T08:22:29.178ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
20742024-02-03T08:22:29.209ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
20752024-02-03T08:22:29.209ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b6f4125d-b706-4ba6-8724-3ec79b6d30e5", "date": "Sat, 03 Feb 2024 08:22:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
20762024-02-03T08:22:29.209ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
20772024-02-03T08:22:29.211ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c271f4dc-1d6d-4991-b8ef-c593a987881f", "date": "Sat, 03 Feb 2024 08:22:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
20782024-02-03T08:22:29.211ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
20792024-02-03T08:22:29.212ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9c8ca488-dcb1-4208-b4c0-798e3a3c7e75", "date": "Sat, 03 Feb 2024 08:22:29 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
20802024-02-03T08:22:29.212ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
20812024-02-03T08:22:29.213ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
20822024-02-03T08:22:29.213ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 113.944863ms iteration = 13
20832024-02-03T08:22:31.329ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1331 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = bc871ee5-a22b-4cf2-bd49-d4ac4d045e61 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
20842024-02-03T08:22:39.094ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
20852024-02-03T08:22:39.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20862024-02-03T08:22:39.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20872024-02-03T08:22:39.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20882024-02-03T08:22:39.098ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20892024-02-03T08:22:39.098ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20902024-02-03T08:22:39.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20912024-02-03T08:22:39.099ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.298056ms iteration = 51
20922024-02-03T08:22:41.329ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 619 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = ad6e7319-5e32-4036-ba36-5a9f3ff96354 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
20932024-02-03T08:22:49.093ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
20942024-02-03T08:22:49.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20952024-02-03T08:22:49.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20962024-02-03T08:22:49.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20972024-02-03T08:22:49.097ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20982024-02-03T08:22:49.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20992024-02-03T08:22:49.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21002024-02-03T08:22:49.099ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.164924ms iteration = 52
21012024-02-03T08:22:51.329ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 528 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = c9f620e1-45ae-408e-a391-69f4e7f89f3b response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
21022024-02-03T08:22:59.093ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
21032024-02-03T08:22:59.094ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
21042024-02-03T08:22:59.094ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
21052024-02-03T08:22:59.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21062024-02-03T08:22:59.094ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
21072024-02-03T08:22:59.094ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
21082024-02-03T08:22:59.094ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
21092024-02-03T08:22:59.094ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
21102024-02-03T08:22:59.094ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
21112024-02-03T08:22:59.094ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21122024-02-03T08:22:59.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
21132024-02-03T08:22:59.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21142024-02-03T08:22:59.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
21152024-02-03T08:22:59.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21162024-02-03T08:22:59.095ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21172024-02-03T08:22:59.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21182024-02-03T08:22:59.096ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
21192024-02-03T08:22:59.096ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
21202024-02-03T08:22:59.096ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "795464b7-7824-4189-8aed-0c363bfb04b7", "content-length": "2", "date": "Sat, 03 Feb 2024 08:22:59 GMT"} })
21212024-02-03T08:22:59.096ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.32097ms iteration = 19
21222024-02-03T08:22:59.096ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6efbd793-cf9d-4316-b8bc-5bd7f67fbf64", "content-length": "2", "date": "Sat, 03 Feb 2024 08:22:58 GMT"} })
21232024-02-03T08:22:59.096ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
21242024-02-03T08:22:59.096ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
21252024-02-03T08:22:59.096ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.489042ms iteration = 19
21262024-02-03T08:22:59.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
21272024-02-03T08:22:59.097ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.43166ms iteration = 19
21282024-02-03T08:22:59.099ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
21292024-02-03T08:22:59.099ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21302024-02-03T08:22:59.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21312024-02-03T08:22:59.099ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21322024-02-03T08:22:59.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21332024-02-03T08:22:59.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21342024-02-03T08:22:59.100ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21352024-02-03T08:22:59.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21362024-02-03T08:22:59.100ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.746838ms iteration = 53
21372024-02-03T08:22:59.110ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21382024-02-03T08:22:59.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21392024-02-03T08:22:59.112ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21402024-02-03T08:22:59.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21412024-02-03T08:22:59.113ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21422024-02-03T08:22:59.115ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 139fb5ac-06c6-4816-a9fe-3967ecbe5f17, lookup_type: ById(139fb5ac-06c6-4816-a9fe-3967ecbe5f17) } result = Ok(())
21432024-02-03T08:22:59.115ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21442024-02-03T08:22:59.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21452024-02-03T08:22:59.116ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 22.598364ms iteration = 19
21462024-02-03T08:23:01.329ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 642 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = c1fabb13-746e-414f-8c7e-7740323c711e response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
21472024-02-03T08:23:09.094ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
21482024-02-03T08:23:09.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21492024-02-03T08:23:09.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21502024-02-03T08:23:09.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21512024-02-03T08:23:09.097ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21522024-02-03T08:23:09.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21532024-02-03T08:23:09.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21542024-02-03T08:23:09.099ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.520018ms iteration = 54
21552024-02-03T08:23:11.330ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 619 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = 6c05e413-fc05-4856-961e-545b60f40cd3 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d
21562024-02-03T08:23:19.095ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
21572024-02-03T08:23:19.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21582024-02-03T08:23:19.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21592024-02-03T08:23:19.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21602024-02-03T08:23:19.099ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21612024-02-03T08:23:19.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21622024-02-03T08:23:19.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21632024-02-03T08:23:19.100ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.262955ms iteration = 55
21642024-02-03T08:23:21.330ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 517 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47767 req_id = a27ca294-1edc-4cfa-8af8-6070aa7c3755 response_code = 200 uri = /metrics/collect/450b9459-744c-4952-8d83-aa130e5dcf6d