Max level shown:
12024-02-03T11:30:01.708ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2027 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = f1d30317-0935-46b6-9473-4dfe263933ce response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
22024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
32024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
42024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
62024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
72024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
82024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
92024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
102024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
112024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
122024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
132024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
142024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
152024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
162024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
172024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
182024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
192024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
202024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
212024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
222024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
232024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
242024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
252024-02-03T11:30:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
262024-02-03T11:30:04.437ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
272024-02-03T11:30:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
282024-02-03T11:30:04.437ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
292024-02-03T11:30:04.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
302024-02-03T11:30:04.438ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
312024-02-03T11:30:04.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
322024-02-03T11:30:04.438ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
332024-02-03T11:30:04.439ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
342024-02-03T11:30:04.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
352024-02-03T11:30:04.439ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
362024-02-03T11:30:04.440ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
372024-02-03T11:30:04.440ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "827c6aac-cfdf-4ffb-aec1-d0b8140b11d0", "content-length": "2", "date": "Sat, 03 Feb 2024 11:30:04 GMT"} })
382024-02-03T11:30:04.440ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 4.249325ms iteration = 10
392024-02-03T11:30:04.441ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
402024-02-03T11:30:04.441ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
412024-02-03T11:30:04.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
422024-02-03T11:30:04.441ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
432024-02-03T11:30:04.441ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
442024-02-03T11:30:04.441ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
452024-02-03T11:30:04.441ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.15\u{b5}s iteration = 6
462024-02-03T11:30:04.441ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
472024-02-03T11:30:04.441ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
482024-02-03T11:30:04.441ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
492024-02-03T11:30:04.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
502024-02-03T11:30:04.442ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
512024-02-03T11:30:04.442ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
522024-02-03T11:30:04.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
532024-02-03T11:30:04.442ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
542024-02-03T11:30:04.443ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
552024-02-03T11:30:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
562024-02-03T11:30:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
572024-02-03T11:30:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
582024-02-03T11:30:04.443ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
592024-02-03T11:30:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
602024-02-03T11:30:04.443ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
612024-02-03T11:30:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
622024-02-03T11:30:04.443ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
632024-02-03T11:30:04.443ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
642024-02-03T11:30:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
652024-02-03T11:30:04.444ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ee1e9969-2839-41bb-8a37-f8939e7478cd", "content-length": "1", "date": "Sat, 03 Feb 2024 11:30:04 GMT"} })
662024-02-03T11:30:04.444ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
672024-02-03T11:30:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
682024-02-03T11:30:04.444ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
692024-02-03T11:30:04.444ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
702024-02-03T11:30:04.444ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
712024-02-03T11:30:04.444ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
722024-02-03T11:30:04.444ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
732024-02-03T11:30:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
742024-02-03T11:30:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
752024-02-03T11:30:04.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
762024-02-03T11:30:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
772024-02-03T11:30:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
782024-02-03T11:30:04.445ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
792024-02-03T11:30:04.445ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.5928ms iteration = 10
802024-02-03T11:30:04.445ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
812024-02-03T11:30:04.445ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 8.727175ms iteration = 7
822024-02-03T11:30:04.445ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
832024-02-03T11:30:04.445ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
842024-02-03T11:30:04.445ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
852024-02-03T11:30:04.445ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
862024-02-03T11:30:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
872024-02-03T11:30:04.445ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 4.165503ms iteration = 10
882024-02-03T11:30:04.445ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
892024-02-03T11:30:04.445ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
902024-02-03T11:30:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
912024-02-03T11:30:04.446ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 10.100376ms iteration = 26
922024-02-03T11:30:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
932024-02-03T11:30:04.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
942024-02-03T11:30:04.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
952024-02-03T11:30:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
962024-02-03T11:30:04.448ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
972024-02-03T11:30:04.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 5.465592ms iteration = 7
982024-02-03T11:30:04.448ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
992024-02-03T11:30:04.448ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1002024-02-03T11:30:04.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1012024-02-03T11:30:04.449ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1022024-02-03T11:30:04.449ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1032024-02-03T11:30:04.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1042024-02-03T11:30:04.449ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1052024-02-03T11:30:04.449ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1062024-02-03T11:30:04.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1072024-02-03T11:30:04.450ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1082024-02-03T11:30:04.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1092024-02-03T11:30:04.450ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
1102024-02-03T11:30:04.450ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 9fa37d3d-ae47-4d04-964d-64e63a91f6ab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
1112024-02-03T11:30:04.450ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1122024-02-03T11:30:04.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1132024-02-03T11:30:04.451ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
1142024-02-03T11:30:04.451ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = e9da49a1-d573-4cd3-b35c-dc304a5b2ae5 dns_zone_name = oxide.test found_so_far = 0 version = 2
1152024-02-03T11:30:04.451ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1162024-02-03T11:30:04.451ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1172024-02-03T11:30:04.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1182024-02-03T11:30:04.451ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1192024-02-03T11:30:04.452ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1202024-02-03T11:30:04.453ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1212024-02-03T11:30:04.453ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1222024-02-03T11:30:04.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1232024-02-03T11:30:04.455ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1242024-02-03T11:30:04.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1252024-02-03T11:30:04.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1262024-02-03T11:30:04.455ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1272024-02-03T11:30:04.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1282024-02-03T11:30:04.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1292024-02-03T11:30:04.456ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = e9da49a1-d573-4cd3-b35c-dc304a5b2ae5 dns_zone_name = oxide.test found_so_far = 1 version = 2
1302024-02-03T11:30:04.456ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
1312024-02-03T11:30:04.456ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-03 11:26:18.232692 UTC dns_group = external generation = 2
1322024-02-03T11:30:04.456ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 20.205931ms iteration = 7
1332024-02-03T11:30:04.456ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1342024-02-03T11:30:04.456ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 9fa37d3d-ae47-4d04-964d-64e63a91f6ab dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
1352024-02-03T11:30:04.456ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
1362024-02-03T11:30:04.456ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-03 11:26:17.704603 UTC dns_group = internal generation = 1
1372024-02-03T11:30:04.456ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 20.352715ms iteration = 7
1382024-02-03T11:30:04.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1392024-02-03T11:30:04.457ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1402024-02-03T11:30:04.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1412024-02-03T11:30:04.459ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1422024-02-03T11:30:04.459ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-03T11:26:04.511335Z, time_modified: 2024-02-03T11:26:04.511335Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, db_silo: Silo { identity: SiloIdentity { id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T11:26:18.205947Z, time_modified: 2024-02-03T11:26:18.205947Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bb35bef0102448ed4611ffc4764b455e95a566c988926739dd37a18ca5354574" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, db_silo: Silo { identity: SiloIdentity { id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T11:26:18.205947Z, time_modified: 2024-02-03T11:26:18.205947Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bb35bef0102448ed4611ffc4764b455e95a566c988926739dd37a18ca5354574" }] }) }
1432024-02-03T11:30:04.459ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 22.973544ms iteration = 7
1442024-02-03T11:30:04.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 7b759a21-13c7-4395-b4fb-95d9207b0f57, lookup_type: ById(7b759a21-13c7-4395-b4fb-95d9207b0f57) } result = Ok(())
1452024-02-03T11:30:04.461ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1462024-02-03T11:30:04.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1472024-02-03T11:30:04.463ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.499573ms iteration = 10
1482024-02-03T11:30:04.489ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
1492024-02-03T11:30:04.506ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
1502024-02-03T11:30:04.548ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
1512024-02-03T11:30:04.571ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
1522024-02-03T11:30:04.571ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7ba7a21f-96af-43b2-8297-a6bfaa1906ff", "date": "Sat, 03 Feb 2024 11:30:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
1532024-02-03T11:30:04.571ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
1542024-02-03T11:30:04.573ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2d257bd8-8764-4b64-a436-98398969a01b", "date": "Sat, 03 Feb 2024 11:30:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
1552024-02-03T11:30:04.574ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
1562024-02-03T11:30:04.574ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
1572024-02-03T11:30:04.574ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 130.728693ms iteration = 9
1582024-02-03T11:30:04.591ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
1592024-02-03T11:30:04.591ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d0cd14ef-0303-4b4a-a752-85e3225ed441", "date": "Sat, 03 Feb 2024 11:30:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1602024-02-03T11:30:04.591ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1612024-02-03T11:30:04.592ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3b909145-1240-4853-a0ad-83f17d6c27e4", "date": "Sat, 03 Feb 2024 11:30:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1622024-02-03T11:30:04.592ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1632024-02-03T11:30:04.594ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ef37628a-48af-412f-9d9c-44b5d5f99f4e", "date": "Sat, 03 Feb 2024 11:30:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1642024-02-03T11:30:04.594ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1652024-02-03T11:30:04.596ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
1662024-02-03T11:30:04.596ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 160.057869ms iteration = 9
1672024-02-03T11:30:11.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 888 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = a35bcb15-bd7c-471b-9b04-1001b37bd6d6 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
1682024-02-03T11:30:14.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
1692024-02-03T11:30:14.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1702024-02-03T11:30:14.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1712024-02-03T11:30:14.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1722024-02-03T11:30:14.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1732024-02-03T11:30:14.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1742024-02-03T11:30:14.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1752024-02-03T11:30:14.442ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.007224ms iteration = 27
1762024-02-03T11:30:18.688ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = f7c9078b-6577-4592-95f0-d973baea98b6 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/7?limit=100
1772024-02-03T11:30:18.689ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = f7c9078b-6577-4592-95f0-d973baea98b6 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/7?limit=100
1782024-02-03T11:30:18.692ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 3261 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = f7c9078b-6577-4592-95f0-d973baea98b6 response_code = 200 uri = /nat/ipv4/changeset/7?limit=100
1792024-02-03T11:30:21.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 964 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = b139c467-132c-484e-a332-b6c55a398f60 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
1802024-02-03T11:30:24.437ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
1812024-02-03T11:30:24.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1822024-02-03T11:30:24.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1832024-02-03T11:30:24.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1842024-02-03T11:30:24.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1852024-02-03T11:30:24.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1862024-02-03T11:30:24.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1872024-02-03T11:30:24.443ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.746502ms iteration = 28
1882024-02-03T11:30:31.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 977 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 946cc487-ea6e-4998-b6a4-b92513bb14e2 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
1892024-02-03T11:30:34.437ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
1902024-02-03T11:30:34.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1912024-02-03T11:30:34.437ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
1922024-02-03T11:30:34.437ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
1932024-02-03T11:30:34.437ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
1942024-02-03T11:30:34.437ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1952024-02-03T11:30:34.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1962024-02-03T11:30:34.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1972024-02-03T11:30:34.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1982024-02-03T11:30:34.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1992024-02-03T11:30:34.438ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2002024-02-03T11:30:34.439ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
2012024-02-03T11:30:34.439ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b698a42a-7a28-400b-b2c5-c2b381f7d9f6", "content-length": "2", "date": "Sat, 03 Feb 2024 11:30:34 GMT"} })
2022024-02-03T11:30:34.439ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.736411ms iteration = 11
2032024-02-03T11:30:34.441ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
2042024-02-03T11:30:34.441ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
2052024-02-03T11:30:34.441ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
2062024-02-03T11:30:34.441ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2072024-02-03T11:30:34.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2082024-02-03T11:30:34.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2092024-02-03T11:30:34.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2102024-02-03T11:30:34.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2112024-02-03T11:30:34.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2122024-02-03T11:30:34.442ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2132024-02-03T11:30:34.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2142024-02-03T11:30:34.443ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2152024-02-03T11:30:34.443ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "86c8f04c-83e3-467d-9452-9f80b0023152", "content-length": "1", "date": "Sat, 03 Feb 2024 11:30:34 GMT"} })
2162024-02-03T11:30:34.443ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2172024-02-03T11:30:34.443ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.501426ms iteration = 29
2182024-02-03T11:30:34.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2192024-02-03T11:30:34.444ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
2202024-02-03T11:30:34.444ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.364092ms iteration = 11
2212024-02-03T11:30:34.444ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2222024-02-03T11:30:34.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2232024-02-03T11:30:34.445ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.250333ms iteration = 11
2242024-02-03T11:30:34.458ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2252024-02-03T11:30:34.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2262024-02-03T11:30:34.460ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2272024-02-03T11:30:34.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2282024-02-03T11:30:34.461ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2292024-02-03T11:30:34.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 7b759a21-13c7-4395-b4fb-95d9207b0f57, lookup_type: ById(7b759a21-13c7-4395-b4fb-95d9207b0f57) } result = Ok(())
2302024-02-03T11:30:34.464ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2312024-02-03T11:30:34.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2322024-02-03T11:30:34.465ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.325344ms iteration = 11
2332024-02-03T11:30:41.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1024 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 4559b19f-8773-4591-b383-df18c3f65fd5 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
2342024-02-03T11:30:44.437ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
2352024-02-03T11:30:44.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2362024-02-03T11:30:44.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2372024-02-03T11:30:44.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2382024-02-03T11:30:44.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2392024-02-03T11:30:44.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2402024-02-03T11:30:44.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2412024-02-03T11:30:44.444ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.199681ms iteration = 30
2422024-02-03T11:30:48.721ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = b3173577-dd92-4bff-8b8e-aa3d680f1b4e roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/7?limit=100
2432024-02-03T11:30:48.722ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = b3173577-dd92-4bff-8b8e-aa3d680f1b4e resource = Database result = Ok(()) uri = /nat/ipv4/changeset/7?limit=100
2442024-02-03T11:30:48.724ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2589 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = b3173577-dd92-4bff-8b8e-aa3d680f1b4e response_code = 200 uri = /nat/ipv4/changeset/7?limit=100
2452024-02-03T11:30:51.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 995 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 16ae9dd6-be0a-42f4-94fa-bfc4858bf30d response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
2462024-02-03T11:30:54.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
2472024-02-03T11:30:54.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2482024-02-03T11:30:54.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2492024-02-03T11:30:54.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2502024-02-03T11:30:54.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2512024-02-03T11:30:54.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2522024-02-03T11:30:54.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2532024-02-03T11:30:54.443ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.431684ms iteration = 31
2542024-02-03T11:31:01.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 861 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 867275d7-a732-4ae9-b190-a5cab7f71b2e response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
2552024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
2562024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2572024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
2582024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
2592024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
2602024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2612024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2622024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
2632024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
2642024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
2652024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
2662024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
2672024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2682024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2692024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
2702024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2712024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
2722024-02-03T11:31:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2732024-02-03T11:31:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2742024-02-03T11:31:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2752024-02-03T11:31:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
2762024-02-03T11:31:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2772024-02-03T11:31:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2782024-02-03T11:31:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
2792024-02-03T11:31:04.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2802024-02-03T11:31:04.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2812024-02-03T11:31:04.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2822024-02-03T11:31:04.439ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2832024-02-03T11:31:04.439ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
2842024-02-03T11:31:04.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2852024-02-03T11:31:04.439ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2862024-02-03T11:31:04.439ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d9e6ed63-aa39-4c6e-a2ce-9fee3c09fb3f", "content-length": "2", "date": "Sat, 03 Feb 2024 11:31:04 GMT"} })
2872024-02-03T11:31:04.439ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 3.220993ms iteration = 12
2882024-02-03T11:31:04.439ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2892024-02-03T11:31:04.440ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2902024-02-03T11:31:04.440ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2912024-02-03T11:31:04.440ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2922024-02-03T11:31:04.441ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
2932024-02-03T11:31:04.441ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
2942024-02-03T11:31:04.441ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2952024-02-03T11:31:04.441ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
2962024-02-03T11:31:04.441ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 7.791\u{b5}s iteration = 7
2972024-02-03T11:31:04.441ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
2982024-02-03T11:31:04.441ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
2992024-02-03T11:31:04.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3002024-02-03T11:31:04.442ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
3012024-02-03T11:31:04.442ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
3022024-02-03T11:31:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
3032024-02-03T11:31:04.443ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
3042024-02-03T11:31:04.443ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3052024-02-03T11:31:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3062024-02-03T11:31:04.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3072024-02-03T11:31:04.443ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c8e2bc05-7db3-43eb-9cbf-2d69c0fdda55", "content-length": "1", "date": "Sat, 03 Feb 2024 11:31:04 GMT"} })
3082024-02-03T11:31:04.443ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3092024-02-03T11:31:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3102024-02-03T11:31:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3112024-02-03T11:31:04.444ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3122024-02-03T11:31:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
3132024-02-03T11:31:04.444ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
3142024-02-03T11:31:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
3152024-02-03T11:31:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
3162024-02-03T11:31:04.444ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3172024-02-03T11:31:04.445ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 8.677934ms iteration = 32
3182024-02-03T11:31:04.445ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3192024-02-03T11:31:04.445ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3202024-02-03T11:31:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3212024-02-03T11:31:04.445ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
3222024-02-03T11:31:04.445ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.467737ms iteration = 12
3232024-02-03T11:31:04.445ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.631182ms iteration = 12
3242024-02-03T11:31:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3252024-02-03T11:31:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3262024-02-03T11:31:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3272024-02-03T11:31:04.446ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
3282024-02-03T11:31:04.446ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 9.983713ms iteration = 8
3292024-02-03T11:31:04.446ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3302024-02-03T11:31:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
3312024-02-03T11:31:04.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
3322024-02-03T11:31:04.446ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
3332024-02-03T11:31:04.446ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
3342024-02-03T11:31:04.446ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
3352024-02-03T11:31:04.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3362024-02-03T11:31:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3372024-02-03T11:31:04.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
3382024-02-03T11:31:04.447ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3392024-02-03T11:31:04.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3402024-02-03T11:31:04.447ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
3412024-02-03T11:31:04.447ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
3422024-02-03T11:31:04.447ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
3432024-02-03T11:31:04.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3442024-02-03T11:31:04.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3452024-02-03T11:31:04.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3462024-02-03T11:31:04.448ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3472024-02-03T11:31:04.448ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3482024-02-03T11:31:04.448ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3492024-02-03T11:31:04.449ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
3502024-02-03T11:31:04.449ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 6.811232ms iteration = 8
3512024-02-03T11:31:04.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3522024-02-03T11:31:04.451ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3532024-02-03T11:31:04.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3542024-02-03T11:31:04.453ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3552024-02-03T11:31:04.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3562024-02-03T11:31:04.453ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
3572024-02-03T11:31:04.453ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 9fa37d3d-ae47-4d04-964d-64e63a91f6ab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
3582024-02-03T11:31:04.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3592024-02-03T11:31:04.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3602024-02-03T11:31:04.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3612024-02-03T11:31:04.453ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
3622024-02-03T11:31:04.453ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3632024-02-03T11:31:04.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3642024-02-03T11:31:04.454ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3652024-02-03T11:31:04.454ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
3662024-02-03T11:31:04.454ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = e9da49a1-d573-4cd3-b35c-dc304a5b2ae5 dns_zone_name = oxide.test found_so_far = 0 version = 2
3672024-02-03T11:31:04.454ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3682024-02-03T11:31:04.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3692024-02-03T11:31:04.455ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3702024-02-03T11:31:04.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3712024-02-03T11:31:04.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3722024-02-03T11:31:04.456ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3732024-02-03T11:31:04.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3742024-02-03T11:31:04.457ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3752024-02-03T11:31:04.457ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3762024-02-03T11:31:04.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3772024-02-03T11:31:04.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3782024-02-03T11:31:04.459ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3792024-02-03T11:31:04.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3802024-02-03T11:31:04.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3812024-02-03T11:31:04.459ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3822024-02-03T11:31:04.459ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3832024-02-03T11:31:04.459ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3842024-02-03T11:31:04.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3852024-02-03T11:31:04.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3862024-02-03T11:31:04.459ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 9fa37d3d-ae47-4d04-964d-64e63a91f6ab dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
3872024-02-03T11:31:04.459ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3882024-02-03T11:31:04.459ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-03 11:26:17.704603 UTC dns_group = internal generation = 1
3892024-02-03T11:31:04.459ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 23.477935ms iteration = 8
3902024-02-03T11:31:04.460ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3912024-02-03T11:31:04.460ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = e9da49a1-d573-4cd3-b35c-dc304a5b2ae5 dns_zone_name = oxide.test found_so_far = 1 version = 2
3922024-02-03T11:31:04.460ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
3932024-02-03T11:31:04.460ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-03 11:26:18.232692 UTC dns_group = external generation = 2
3942024-02-03T11:31:04.460ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 24.56043ms iteration = 8
3952024-02-03T11:31:04.462ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-03T11:26:04.511335Z, time_modified: 2024-02-03T11:26:04.511335Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, db_silo: Silo { identity: SiloIdentity { id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T11:26:18.205947Z, time_modified: 2024-02-03T11:26:18.205947Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bb35bef0102448ed4611ffc4764b455e95a566c988926739dd37a18ca5354574" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, db_silo: Silo { identity: SiloIdentity { id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T11:26:18.205947Z, time_modified: 2024-02-03T11:26:18.205947Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bb35bef0102448ed4611ffc4764b455e95a566c988926739dd37a18ca5354574" }] }) }
3962024-02-03T11:31:04.462ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 26.420231ms iteration = 8
3972024-02-03T11:31:04.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 7b759a21-13c7-4395-b4fb-95d9207b0f57, lookup_type: ById(7b759a21-13c7-4395-b4fb-95d9207b0f57) } result = Ok(())
3982024-02-03T11:31:04.463ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3992024-02-03T11:31:04.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4002024-02-03T11:31:04.464ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.415705ms iteration = 12
4012024-02-03T11:31:04.475ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
4022024-02-03T11:31:04.485ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
4032024-02-03T11:31:04.513ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
4042024-02-03T11:31:04.524ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4052024-02-03T11:31:04.525ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8e25db80-937d-40cb-a046-8745e3f5cebc", "date": "Sat, 03 Feb 2024 11:31:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4062024-02-03T11:31:04.525ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4072024-02-03T11:31:04.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1a98778f-e7a8-4153-a97e-bbebe9778b99", "date": "Sat, 03 Feb 2024 11:31:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4082024-02-03T11:31:04.526ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4092024-02-03T11:31:04.527ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4102024-02-03T11:31:04.527ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 84.007409ms iteration = 10
4112024-02-03T11:31:04.545ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
4122024-02-03T11:31:04.546ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "608d3262-bfd0-42b0-a20b-635dc0120a80", "date": "Sat, 03 Feb 2024 11:31:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4132024-02-03T11:31:04.546ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4142024-02-03T11:31:04.547ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8d9e957e-0885-46fb-a7e8-97e5101d136d", "date": "Sat, 03 Feb 2024 11:31:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4152024-02-03T11:31:04.547ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4162024-02-03T11:31:04.548ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2bfb1880-e362-4c7c-aec5-8b093b1e287b", "date": "Sat, 03 Feb 2024 11:31:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4172024-02-03T11:31:04.548ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4182024-02-03T11:31:04.549ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
4192024-02-03T11:31:04.549ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 113.264074ms iteration = 10
4202024-02-03T11:31:11.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 860 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 9e1df262-cc48-48ce-a197-1ac8d6aa043c response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
4212024-02-03T11:31:14.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
4222024-02-03T11:31:14.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4232024-02-03T11:31:14.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4242024-02-03T11:31:14.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4252024-02-03T11:31:14.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4262024-02-03T11:31:14.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4272024-02-03T11:31:14.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4282024-02-03T11:31:14.442ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.283911ms iteration = 33
4292024-02-03T11:31:18.758ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 9c25c8b3-2be9-4fa2-8577-07d60fbeba3e roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/7?limit=100
4302024-02-03T11:31:18.758ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 9c25c8b3-2be9-4fa2-8577-07d60fbeba3e resource = Database result = Ok(()) uri = /nat/ipv4/changeset/7?limit=100
4312024-02-03T11:31:18.760ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2637 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 9c25c8b3-2be9-4fa2-8577-07d60fbeba3e response_code = 200 uri = /nat/ipv4/changeset/7?limit=100
4322024-02-03T11:31:21.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 859 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 7cfb028a-dc0b-4937-a2a2-69c183baaefe response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
4332024-02-03T11:31:24.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
4342024-02-03T11:31:24.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4352024-02-03T11:31:24.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4362024-02-03T11:31:24.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4372024-02-03T11:31:24.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4382024-02-03T11:31:24.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4392024-02-03T11:31:24.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4402024-02-03T11:31:24.443ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.716321ms iteration = 34
4412024-02-03T11:31:31.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1699 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = bde0e9ea-2531-48d4-af67-eda85d5fcc01 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
4422024-02-03T11:31:34.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
4432024-02-03T11:31:34.436ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
4442024-02-03T11:31:34.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4452024-02-03T11:31:34.436ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4462024-02-03T11:31:34.436ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
4472024-02-03T11:31:34.436ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4482024-02-03T11:31:34.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4492024-02-03T11:31:34.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4502024-02-03T11:31:34.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4512024-02-03T11:31:34.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4522024-02-03T11:31:34.439ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
4532024-02-03T11:31:34.439ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4542024-02-03T11:31:34.440ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5b10bdb8-656a-4fc7-9db0-5e6f2a3e7525", "content-length": "2", "date": "Sat, 03 Feb 2024 11:31:34 GMT"} })
4552024-02-03T11:31:34.440ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 3.748484ms iteration = 13
4562024-02-03T11:31:34.441ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
4572024-02-03T11:31:34.441ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4582024-02-03T11:31:34.441ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
4592024-02-03T11:31:34.441ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
4602024-02-03T11:31:34.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4612024-02-03T11:31:34.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4622024-02-03T11:31:34.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4632024-02-03T11:31:34.443ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4642024-02-03T11:31:34.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4652024-02-03T11:31:34.443ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4662024-02-03T11:31:34.443ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4672024-02-03T11:31:34.443ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e2fccaa3-e586-4bc6-8d73-1f3cf72b7f10", "content-length": "1", "date": "Sat, 03 Feb 2024 11:31:34 GMT"} })
4682024-02-03T11:31:34.443ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4692024-02-03T11:31:34.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4702024-02-03T11:31:34.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4712024-02-03T11:31:34.444ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.797664ms iteration = 35
4722024-02-03T11:31:34.444ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
4732024-02-03T11:31:34.444ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.200032ms iteration = 13
4742024-02-03T11:31:34.444ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4752024-02-03T11:31:34.444ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.443767ms iteration = 13
4762024-02-03T11:31:34.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4772024-02-03T11:31:34.456ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4782024-02-03T11:31:34.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4792024-02-03T11:31:34.457ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4802024-02-03T11:31:34.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4812024-02-03T11:31:34.459ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4822024-02-03T11:31:34.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 7b759a21-13c7-4395-b4fb-95d9207b0f57, lookup_type: ById(7b759a21-13c7-4395-b4fb-95d9207b0f57) } result = Ok(())
4832024-02-03T11:31:34.461ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4842024-02-03T11:31:34.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4852024-02-03T11:31:34.463ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.339999ms iteration = 13
4862024-02-03T11:31:41.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 877 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = c72409aa-cc59-4910-b3bf-9fd7b2362699 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
4872024-02-03T11:31:44.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
4882024-02-03T11:31:44.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4892024-02-03T11:31:44.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4902024-02-03T11:31:44.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4912024-02-03T11:31:44.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4922024-02-03T11:31:44.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4932024-02-03T11:31:44.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4942024-02-03T11:31:44.442ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.007715ms iteration = 36
4952024-02-03T11:31:46.791ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 0c5917a9-464b-4017-b421-a05e9d1e3e25 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/7?limit=100
4962024-02-03T11:31:46.791ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 0c5917a9-464b-4017-b421-a05e9d1e3e25 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/7?limit=100
4972024-02-03T11:31:46.794ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 3391 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 0c5917a9-464b-4017-b421-a05e9d1e3e25 response_code = 200 uri = /nat/ipv4/changeset/7?limit=100
4982024-02-03T11:31:46.794ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 8b35845a-82bc-4c5d-8cf8-fc0fb7f2bca4 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/9?limit=100
4992024-02-03T11:31:46.795ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 8b35845a-82bc-4c5d-8cf8-fc0fb7f2bca4 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/9?limit=100
5002024-02-03T11:31:46.797ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2624 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 8b35845a-82bc-4c5d-8cf8-fc0fb7f2bca4 response_code = 200 uri = /nat/ipv4/changeset/9?limit=100
5012024-02-03T11:31:51.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1017 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 06109e97-9617-4f45-9d9b-21daaa61e31a response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
5022024-02-03T11:31:54.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
5032024-02-03T11:31:54.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5042024-02-03T11:31:54.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5052024-02-03T11:31:54.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5062024-02-03T11:31:54.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5072024-02-03T11:31:54.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5082024-02-03T11:31:54.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5092024-02-03T11:31:54.443ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.975996ms iteration = 37
5102024-02-03T11:31:54.505ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:40678
5112024-02-03T11:31:54.514ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08 instance_state = InstanceRuntimeState { propolis_id: Some(063869e9-7624-468c-b5e0-2fc1b068cd96), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-03T11:31:45.944778176Z } propolis_id = 063869e9-7624-468c-b5e0-2fc1b068cd96 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-02-03T11:31:54.502067302Z }
5122024-02-03T11:31:54.520ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 03069933-6e1d-4c93-8313-3c6bc66d55ed roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5132024-02-03T11:31:54.520ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 03069933-6e1d-4c93-8313-3c6bc66d55ed resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5142024-02-03T11:31:54.524ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 03069933-6e1d-4c93-8313-3c6bc66d55ed roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5152024-02-03T11:31:54.525ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 03069933-6e1d-4c93-8313-3c6bc66d55ed resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5162024-02-03T11:31:54.526ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 03069933-6e1d-4c93-8313-3c6bc66d55ed roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5172024-02-03T11:31:54.526ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 03069933-6e1d-4c93-8313-3c6bc66d55ed resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5182024-02-03T11:31:54.528ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 03069933-6e1d-4c93-8313-3c6bc66d55ed roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5192024-02-03T11:31:54.528ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 03069933-6e1d-4c93-8313-3c6bc66d55ed resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5202024-02-03T11:31:54.529ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 03069933-6e1d-4c93-8313-3c6bc66d55ed roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5212024-02-03T11:31:54.529ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 03069933-6e1d-4c93-8313-3c6bc66d55ed resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5222024-02-03T11:31:54.530ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 03069933-6e1d-4c93-8313-3c6bc66d55ed roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5232024-02-03T11:31:54.530ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 03069933-6e1d-4c93-8313-3c6bc66d55ed resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5242024-02-03T11:31:54.531ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 03069933-6e1d-4c93-8313-3c6bc66d55ed roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5252024-02-03T11:31:54.546ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 03069933-6e1d-4c93-8313-3c6bc66d55ed resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, lookup_type: ById(80079a6f-d5da-49cb-a5dc-11e61f8e10f3) }, key: 9561f876-1ccc-42b2-877e-502bced7c0df, lookup_type: ById(9561f876-1ccc-42b2-877e-502bced7c0df) }, key: d5bca60e-22a1-43a4-b98c-cf5ecfa99d08, lookup_type: ById(d5bca60e-22a1-43a4-b98c-cf5ecfa99d08) } result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5262024-02-03T11:31:54.546ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5272024-02-03T11:31:54.579ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08 instance_updated = false propolis_id = 063869e9-7624-468c-b5e0-2fc1b068cd96 vmm_updated = true
5282024-02-03T11:31:54.579ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 74206 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 03069933-6e1d-4c93-8313-3c6bc66d55ed response_code = 204 uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5292024-02-03T11:31:54.579ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08 instance_state = InstanceRuntimeState { propolis_id: Some(063869e9-7624-468c-b5e0-2fc1b068cd96), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-03T11:31:45.944778176Z } propolis_id = 063869e9-7624-468c-b5e0-2fc1b068cd96 vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-02-03T11:31:54.579585995Z }
5302024-02-03T11:31:54.579ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = c3424b84-b7c7-45ae-b2e1-2853829209d4 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5312024-02-03T11:31:54.580ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = c3424b84-b7c7-45ae-b2e1-2853829209d4 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5322024-02-03T11:31:54.581ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = c3424b84-b7c7-45ae-b2e1-2853829209d4 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5332024-02-03T11:31:54.581ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = c3424b84-b7c7-45ae-b2e1-2853829209d4 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5342024-02-03T11:31:54.583ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = c3424b84-b7c7-45ae-b2e1-2853829209d4 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5352024-02-03T11:31:54.583ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = c3424b84-b7c7-45ae-b2e1-2853829209d4 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5362024-02-03T11:31:54.584ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = c3424b84-b7c7-45ae-b2e1-2853829209d4 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5372024-02-03T11:31:54.584ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = c3424b84-b7c7-45ae-b2e1-2853829209d4 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5382024-02-03T11:31:54.585ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = c3424b84-b7c7-45ae-b2e1-2853829209d4 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5392024-02-03T11:31:54.586ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = c3424b84-b7c7-45ae-b2e1-2853829209d4 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5402024-02-03T11:31:54.586ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = c3424b84-b7c7-45ae-b2e1-2853829209d4 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5412024-02-03T11:31:54.587ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = c3424b84-b7c7-45ae-b2e1-2853829209d4 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5422024-02-03T11:31:54.587ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = c3424b84-b7c7-45ae-b2e1-2853829209d4 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5432024-02-03T11:31:54.591ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = c3424b84-b7c7-45ae-b2e1-2853829209d4 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, lookup_type: ById(80079a6f-d5da-49cb-a5dc-11e61f8e10f3) }, key: 9561f876-1ccc-42b2-877e-502bced7c0df, lookup_type: ById(9561f876-1ccc-42b2-877e-502bced7c0df) }, key: d5bca60e-22a1-43a4-b98c-cf5ecfa99d08, lookup_type: ById(d5bca60e-22a1-43a4-b98c-cf5ecfa99d08) } result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5442024-02-03T11:31:54.591ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5452024-02-03T11:31:54.606ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08 instance_updated = false propolis_id = 063869e9-7624-468c-b5e0-2fc1b068cd96 vmm_updated = true
5462024-02-03T11:31:54.606ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 26907 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = c3424b84-b7c7-45ae-b2e1-2853829209d4 response_code = 204 uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
5472024-02-03T11:32:01.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1199 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 75adccc2-cbdd-4942-95eb-6e41844cc26e response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
5482024-02-03T11:32:04.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
5492024-02-03T11:32:04.436ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
5502024-02-03T11:32:04.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5512024-02-03T11:32:04.436ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
5522024-02-03T11:32:04.436ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5532024-02-03T11:32:04.436ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5542024-02-03T11:32:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
5552024-02-03T11:32:04.436ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
5562024-02-03T11:32:04.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5572024-02-03T11:32:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
5582024-02-03T11:32:04.436ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
5592024-02-03T11:32:04.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5602024-02-03T11:32:04.436ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
5612024-02-03T11:32:04.436ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5622024-02-03T11:32:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
5632024-02-03T11:32:04.437ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
5642024-02-03T11:32:04.437ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
5652024-02-03T11:32:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5662024-02-03T11:32:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
5672024-02-03T11:32:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5682024-02-03T11:32:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5692024-02-03T11:32:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5702024-02-03T11:32:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5712024-02-03T11:32:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5722024-02-03T11:32:04.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5732024-02-03T11:32:04.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5742024-02-03T11:32:04.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5752024-02-03T11:32:04.439ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
5762024-02-03T11:32:04.439ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5772024-02-03T11:32:04.439ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5782024-02-03T11:32:04.439ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "61c268db-cc31-45f0-9089-e8605d34984b", "content-length": "2", "date": "Sat, 03 Feb 2024 11:32:04 GMT"} })
5792024-02-03T11:32:04.439ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 3.072508ms iteration = 14
5802024-02-03T11:32:04.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5812024-02-03T11:32:04.440ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5822024-02-03T11:32:04.440ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5832024-02-03T11:32:04.440ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5842024-02-03T11:32:04.441ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
5852024-02-03T11:32:04.441ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5862024-02-03T11:32:04.441ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5872024-02-03T11:32:04.441ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
5882024-02-03T11:32:04.441ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
5892024-02-03T11:32:04.441ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 7.15\u{b5}s iteration = 8
5902024-02-03T11:32:04.441ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
5912024-02-03T11:32:04.441ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
5922024-02-03T11:32:04.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5932024-02-03T11:32:04.442ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
5942024-02-03T11:32:04.442ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
5952024-02-03T11:32:04.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
5962024-02-03T11:32:04.442ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5972024-02-03T11:32:04.442ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "89fd4057-f9fc-46d6-abdd-12458cef4a2b", "content-length": "1", "date": "Sat, 03 Feb 2024 11:32:04 GMT"} })
5982024-02-03T11:32:04.442ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5992024-02-03T11:32:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6002024-02-03T11:32:04.443ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
6012024-02-03T11:32:04.443ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
6022024-02-03T11:32:04.443ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.556317ms iteration = 14
6032024-02-03T11:32:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
6042024-02-03T11:32:04.443ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
6052024-02-03T11:32:04.443ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6062024-02-03T11:32:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
6072024-02-03T11:32:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6082024-02-03T11:32:04.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6092024-02-03T11:32:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6102024-02-03T11:32:04.444ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.807845ms iteration = 14
6112024-02-03T11:32:04.444ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6122024-02-03T11:32:04.444ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6132024-02-03T11:32:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6142024-02-03T11:32:04.445ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
6152024-02-03T11:32:04.445ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 8.684854ms iteration = 9
6162024-02-03T11:32:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6172024-02-03T11:32:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6182024-02-03T11:32:04.445ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6192024-02-03T11:32:04.445ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 9.041393ms iteration = 38
6202024-02-03T11:32:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6212024-02-03T11:32:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6222024-02-03T11:32:04.446ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
6232024-02-03T11:32:04.446ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
6242024-02-03T11:32:04.446ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
6252024-02-03T11:32:04.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6262024-02-03T11:32:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6272024-02-03T11:32:04.446ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6282024-02-03T11:32:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
6292024-02-03T11:32:04.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
6302024-02-03T11:32:04.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6312024-02-03T11:32:04.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
6322024-02-03T11:32:04.447ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6332024-02-03T11:32:04.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6342024-02-03T11:32:04.447ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
6352024-02-03T11:32:04.447ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
6362024-02-03T11:32:04.447ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
6372024-02-03T11:32:04.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6382024-02-03T11:32:04.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6392024-02-03T11:32:04.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6402024-02-03T11:32:04.448ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6412024-02-03T11:32:04.448ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6422024-02-03T11:32:04.448ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
6432024-02-03T11:32:04.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 6.849164ms iteration = 9
6442024-02-03T11:32:04.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6452024-02-03T11:32:04.451ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6462024-02-03T11:32:04.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6472024-02-03T11:32:04.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
6482024-02-03T11:32:04.452ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6492024-02-03T11:32:04.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6502024-02-03T11:32:04.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6512024-02-03T11:32:04.452ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6522024-02-03T11:32:04.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6532024-02-03T11:32:04.452ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
6542024-02-03T11:32:04.452ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = e9da49a1-d573-4cd3-b35c-dc304a5b2ae5 dns_zone_name = oxide.test found_so_far = 0 version = 2
6552024-02-03T11:32:04.452ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6562024-02-03T11:32:04.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6572024-02-03T11:32:04.453ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6582024-02-03T11:32:04.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6592024-02-03T11:32:04.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6602024-02-03T11:32:04.453ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
6612024-02-03T11:32:04.453ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 9fa37d3d-ae47-4d04-964d-64e63a91f6ab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
6622024-02-03T11:32:04.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6632024-02-03T11:32:04.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6642024-02-03T11:32:04.454ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6652024-02-03T11:32:04.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6662024-02-03T11:32:04.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6672024-02-03T11:32:04.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6682024-02-03T11:32:04.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6692024-02-03T11:32:04.457ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6702024-02-03T11:32:04.457ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6712024-02-03T11:32:04.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6722024-02-03T11:32:04.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6732024-02-03T11:32:04.458ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6742024-02-03T11:32:04.458ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = e9da49a1-d573-4cd3-b35c-dc304a5b2ae5 dns_zone_name = oxide.test found_so_far = 1 version = 2
6752024-02-03T11:32:04.458ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
6762024-02-03T11:32:04.458ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-03 11:26:18.232692 UTC dns_group = external generation = 2
6772024-02-03T11:32:04.458ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 21.421929ms iteration = 9
6782024-02-03T11:32:04.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6792024-02-03T11:32:04.460ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 9fa37d3d-ae47-4d04-964d-64e63a91f6ab dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
6802024-02-03T11:32:04.460ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
6812024-02-03T11:32:04.460ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-03 11:26:17.704603 UTC dns_group = internal generation = 1
6822024-02-03T11:32:04.460ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 23.466895ms iteration = 9
6832024-02-03T11:32:04.461ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-03T11:26:04.511335Z, time_modified: 2024-02-03T11:26:04.511335Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, db_silo: Silo { identity: SiloIdentity { id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T11:26:18.205947Z, time_modified: 2024-02-03T11:26:18.205947Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bb35bef0102448ed4611ffc4764b455e95a566c988926739dd37a18ca5354574" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, db_silo: Silo { identity: SiloIdentity { id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T11:26:18.205947Z, time_modified: 2024-02-03T11:26:18.205947Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bb35bef0102448ed4611ffc4764b455e95a566c988926739dd37a18ca5354574" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
6842024-02-03T11:32:04.461ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 24.747004ms iteration = 9
6852024-02-03T11:32:04.462ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6862024-02-03T11:32:04.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6872024-02-03T11:32:04.463ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6882024-02-03T11:32:04.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6892024-02-03T11:32:04.464ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6902024-02-03T11:32:04.466ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 7b759a21-13c7-4395-b4fb-95d9207b0f57, lookup_type: ById(7b759a21-13c7-4395-b4fb-95d9207b0f57) } result = Ok(())
6912024-02-03T11:32:04.466ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6922024-02-03T11:32:04.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6932024-02-03T11:32:04.468ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 31.655988ms iteration = 14
6942024-02-03T11:32:04.497ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
6952024-02-03T11:32:04.502ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
6962024-02-03T11:32:04.556ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
6972024-02-03T11:32:04.561ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
6982024-02-03T11:32:04.561ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3beeba62-a80c-4ed5-97cc-4d99297c0af6", "date": "Sat, 03 Feb 2024 11:32:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
6992024-02-03T11:32:04.561ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7002024-02-03T11:32:04.562ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "13c82753-85da-4592-8bb7-c45721b6da75", "date": "Sat, 03 Feb 2024 11:32:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7012024-02-03T11:32:04.562ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7022024-02-03T11:32:04.564ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7032024-02-03T11:32:04.564ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 121.172019ms iteration = 11
7042024-02-03T11:32:04.588ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
7052024-02-03T11:32:04.589ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0bd78ee2-fda1-4f4e-821d-96e889e32f36", "date": "Sat, 03 Feb 2024 11:32:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7062024-02-03T11:32:04.589ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7072024-02-03T11:32:04.590ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3ec044f6-c14c-4a98-8966-50369138cc21", "date": "Sat, 03 Feb 2024 11:32:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7082024-02-03T11:32:04.590ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7092024-02-03T11:32:04.591ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fd29bf26-5750-4dd9-a9aa-ae81b77e5607", "date": "Sat, 03 Feb 2024 11:32:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7102024-02-03T11:32:04.591ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7112024-02-03T11:32:04.592ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7122024-02-03T11:32:04.592ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 155.300303ms iteration = 11
7132024-02-03T11:32:11.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 932 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 8220ebb2-bdaf-4558-97b9-2ab12235e673 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
7142024-02-03T11:32:11.885ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08 instance_state = InstanceRuntimeState { propolis_id: Some(063869e9-7624-468c-b5e0-2fc1b068cd96), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-03T11:31:45.944778176Z } propolis_id = 063869e9-7624-468c-b5e0-2fc1b068cd96 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-02-03T11:32:11.884855442Z }
7152024-02-03T11:32:11.885ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = b36a187e-dc22-489e-ae5b-bc9edc7a4fda roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7162024-02-03T11:32:11.885ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = b36a187e-dc22-489e-ae5b-bc9edc7a4fda resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7172024-02-03T11:32:11.887ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = b36a187e-dc22-489e-ae5b-bc9edc7a4fda roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7182024-02-03T11:32:11.887ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = b36a187e-dc22-489e-ae5b-bc9edc7a4fda resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7192024-02-03T11:32:11.889ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = b36a187e-dc22-489e-ae5b-bc9edc7a4fda roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7202024-02-03T11:32:11.889ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = b36a187e-dc22-489e-ae5b-bc9edc7a4fda resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7212024-02-03T11:32:11.890ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = b36a187e-dc22-489e-ae5b-bc9edc7a4fda roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7222024-02-03T11:32:11.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = b36a187e-dc22-489e-ae5b-bc9edc7a4fda resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7232024-02-03T11:32:11.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = b36a187e-dc22-489e-ae5b-bc9edc7a4fda roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7242024-02-03T11:32:11.891ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = b36a187e-dc22-489e-ae5b-bc9edc7a4fda resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7252024-02-03T11:32:11.892ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = b36a187e-dc22-489e-ae5b-bc9edc7a4fda roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7262024-02-03T11:32:11.892ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = b36a187e-dc22-489e-ae5b-bc9edc7a4fda resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7272024-02-03T11:32:11.893ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = b36a187e-dc22-489e-ae5b-bc9edc7a4fda roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7282024-02-03T11:32:11.896ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = b36a187e-dc22-489e-ae5b-bc9edc7a4fda resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, lookup_type: ById(80079a6f-d5da-49cb-a5dc-11e61f8e10f3) }, key: 9561f876-1ccc-42b2-877e-502bced7c0df, lookup_type: ById(9561f876-1ccc-42b2-877e-502bced7c0df) }, key: d5bca60e-22a1-43a4-b98c-cf5ecfa99d08, lookup_type: ById(d5bca60e-22a1-43a4-b98c-cf5ecfa99d08) } result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7292024-02-03T11:32:11.897ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7302024-02-03T11:32:11.944ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08 instance_updated = false propolis_id = 063869e9-7624-468c-b5e0-2fc1b068cd96 vmm_updated = true
7312024-02-03T11:32:11.944ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 59964 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = b36a187e-dc22-489e-ae5b-bc9edc7a4fda response_code = 204 uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7322024-02-03T11:32:11.946ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08 instance_state = InstanceRuntimeState { propolis_id: Some(063869e9-7624-468c-b5e0-2fc1b068cd96), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-03T11:31:45.944778176Z } propolis_id = 063869e9-7624-468c-b5e0-2fc1b068cd96 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-02-03T11:32:11.946032240Z }
7332024-02-03T11:32:11.946ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = fad174b9-6e76-4969-8b89-826c77ea9289 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7342024-02-03T11:32:11.946ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = fad174b9-6e76-4969-8b89-826c77ea9289 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7352024-02-03T11:32:11.948ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = fad174b9-6e76-4969-8b89-826c77ea9289 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7362024-02-03T11:32:11.948ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = fad174b9-6e76-4969-8b89-826c77ea9289 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7372024-02-03T11:32:11.949ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = fad174b9-6e76-4969-8b89-826c77ea9289 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7382024-02-03T11:32:11.950ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = fad174b9-6e76-4969-8b89-826c77ea9289 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7392024-02-03T11:32:11.951ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = fad174b9-6e76-4969-8b89-826c77ea9289 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7402024-02-03T11:32:11.951ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = fad174b9-6e76-4969-8b89-826c77ea9289 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7412024-02-03T11:32:11.952ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = fad174b9-6e76-4969-8b89-826c77ea9289 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7422024-02-03T11:32:11.952ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = fad174b9-6e76-4969-8b89-826c77ea9289 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7432024-02-03T11:32:11.953ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = fad174b9-6e76-4969-8b89-826c77ea9289 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7442024-02-03T11:32:11.954ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = fad174b9-6e76-4969-8b89-826c77ea9289 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7452024-02-03T11:32:11.954ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = fad174b9-6e76-4969-8b89-826c77ea9289 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7462024-02-03T11:32:11.958ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = fad174b9-6e76-4969-8b89-826c77ea9289 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, lookup_type: ById(80079a6f-d5da-49cb-a5dc-11e61f8e10f3) }, key: 9561f876-1ccc-42b2-877e-502bced7c0df, lookup_type: ById(9561f876-1ccc-42b2-877e-502bced7c0df) }, key: d5bca60e-22a1-43a4-b98c-cf5ecfa99d08, lookup_type: ById(d5bca60e-22a1-43a4-b98c-cf5ecfa99d08) } result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7472024-02-03T11:32:11.958ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7482024-02-03T11:32:11.974ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08 instance_updated = false propolis_id = 063869e9-7624-468c-b5e0-2fc1b068cd96 vmm_updated = true
7492024-02-03T11:32:11.974ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 28058 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = fad174b9-6e76-4969-8b89-826c77ea9289 response_code = 204 uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7502024-02-03T11:32:12.823ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 97cc39fc-a84b-4394-bd99-831710adafaf roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/9?limit=100
7512024-02-03T11:32:12.823ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 97cc39fc-a84b-4394-bd99-831710adafaf resource = Database result = Ok(()) uri = /nat/ipv4/changeset/9?limit=100
7522024-02-03T11:32:12.826ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 3568 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 97cc39fc-a84b-4394-bd99-831710adafaf response_code = 200 uri = /nat/ipv4/changeset/9?limit=100
7532024-02-03T11:32:12.828ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 6e4c8331-8a04-49c3-ae14-5b4101a8881e roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
7542024-02-03T11:32:12.828ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 6e4c8331-8a04-49c3-ae14-5b4101a8881e resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
7552024-02-03T11:32:12.830ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2460 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 6e4c8331-8a04-49c3-ae14-5b4101a8881e response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
7562024-02-03T11:32:13.832ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 3c49e02c-5759-4275-953d-1de42e4fb4d4 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
7572024-02-03T11:32:13.832ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 3c49e02c-5759-4275-953d-1de42e4fb4d4 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
7582024-02-03T11:32:13.834ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2645 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 3c49e02c-5759-4275-953d-1de42e4fb4d4 response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
7592024-02-03T11:32:14.310ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08 instance_state = InstanceRuntimeState { propolis_id: None, dst_propolis_id: None, migration_id: None, gen: Generation(5), time_updated: 2024-02-03T11:32:12.080274252Z } propolis_id = 063869e9-7624-468c-b5e0-2fc1b068cd96 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-02-03T11:32:12.080274252Z }
7602024-02-03T11:32:14.310ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7612024-02-03T11:32:14.311ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7622024-02-03T11:32:14.313ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7632024-02-03T11:32:14.313ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7642024-02-03T11:32:14.314ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7652024-02-03T11:32:14.315ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7662024-02-03T11:32:14.316ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7672024-02-03T11:32:14.316ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7682024-02-03T11:32:14.318ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7692024-02-03T11:32:14.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7702024-02-03T11:32:14.319ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7712024-02-03T11:32:14.319ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7722024-02-03T11:32:14.320ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7732024-02-03T11:32:14.323ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, lookup_type: ById(80079a6f-d5da-49cb-a5dc-11e61f8e10f3) }, key: 9561f876-1ccc-42b2-877e-502bced7c0df, lookup_type: ById(9561f876-1ccc-42b2-877e-502bced7c0df) }, key: d5bca60e-22a1-43a4-b98c-cf5ecfa99d08, lookup_type: ById(d5bca60e-22a1-43a4-b98c-cf5ecfa99d08) } result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7742024-02-03T11:32:14.323ZINFOnexus: instance cleared its Propolis ID, cleaning network config file = nexus/src/app/instance_network.rs:739 instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08 propolis_id = Some(063869e9-7624-468c-b5e0-2fc1b068cd96)
7752024-02-03T11:32:14.323ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7762024-02-03T11:32:14.324ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7772024-02-03T11:32:14.325ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7782024-02-03T11:32:14.325ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7792024-02-03T11:32:14.326ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7802024-02-03T11:32:14.326ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7812024-02-03T11:32:14.328ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7822024-02-03T11:32:14.328ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7832024-02-03T11:32:14.329ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7842024-02-03T11:32:14.329ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7852024-02-03T11:32:14.330ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7862024-02-03T11:32:14.330ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7872024-02-03T11:32:14.331ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7882024-02-03T11:32:14.334ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, lookup_type: ById(80079a6f-d5da-49cb-a5dc-11e61f8e10f3) }, key: 9561f876-1ccc-42b2-877e-502bced7c0df, lookup_type: ById(9561f876-1ccc-42b2-877e-502bced7c0df) }, key: d5bca60e-22a1-43a4-b98c-cf5ecfa99d08, lookup_type: ById(d5bca60e-22a1-43a4-b98c-cf5ecfa99d08) } result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7892024-02-03T11:32:14.334ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7902024-02-03T11:32:14.334ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7912024-02-03T11:32:14.336ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7922024-02-03T11:32:14.336ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7932024-02-03T11:32:14.337ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7942024-02-03T11:32:14.337ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7952024-02-03T11:32:14.338ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7962024-02-03T11:32:14.341ZDEBGnexus (dropshot_internal): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, lookup_type: ById(80079a6f-d5da-49cb-a5dc-11e61f8e10f3) }, key: 9561f876-1ccc-42b2-877e-502bced7c0df, lookup_type: ById(9561f876-1ccc-42b2-877e-502bced7c0df) }, key: d5bca60e-22a1-43a4-b98c-cf5ecfa99d08, lookup_type: ById(d5bca60e-22a1-43a4-b98c-cf5ecfa99d08) } result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7972024-02-03T11:32:14.341ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7982024-02-03T11:32:14.341ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
7992024-02-03T11:32:14.345ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
8002024-02-03T11:32:14.346ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
8012024-02-03T11:32:14.347ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
8022024-02-03T11:32:14.348ZDEBGnexus (InstanceAllocator): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
8032024-02-03T11:32:14.348ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
8042024-02-03T11:32:14.348ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
8052024-02-03T11:32:14.359ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
8062024-02-03T11:32:14.360ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
8072024-02-03T11:32:14.361ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
8082024-02-03T11:32:14.361ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
8092024-02-03T11:32:14.362ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
8102024-02-03T11:32:14.364ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: 7b759a21-13c7-4395-b4fb-95d9207b0f57, lookup_type: ById(7b759a21-13c7-4395-b4fb-95d9207b0f57) } result = Ok(())
8112024-02-03T11:32:14.404ZDEBGnexus: client request SledAgent = 7b759a21-13c7-4395-b4fb-95d9207b0f57 body = Some(Body) method = DELETE uri = http://[fd00:1122:3344:101::1]:12345/v2p/109bc45e-9b1a-4754-9530-b636b76530f2
8122024-02-03T11:32:14.405ZDEBGnexus: client response SledAgent = 7b759a21-13c7-4395-b4fb-95d9207b0f57 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p/109bc45e-9b1a-4754-9530-b636b76530f2", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9eea88d3-2a51-44c4-88d6-375c6a6ad3b9", "date": "Sat, 03 Feb 2024 11:32:14 GMT"} })
8132024-02-03T11:32:14.407ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:519 instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8142024-02-03T11:32:14.407ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8152024-02-03T11:32:14.407ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8162024-02-03T11:32:14.415ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8172024-02-03T11:32:14.415ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8182024-02-03T11:32:14.421ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:609
8192024-02-03T11:32:14.421ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8202024-02-03T11:32:14.422ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8212024-02-03T11:32:14.423ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:609
8222024-02-03T11:32:14.423ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
8232024-02-03T11:32:14.423ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
8242024-02-03T11:32:14.424ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
8252024-02-03T11:32:14.426ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
8262024-02-03T11:32:14.426ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
8272024-02-03T11:32:14.426ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
8282024-02-03T11:32:14.436ZDEBGnexus: notifying dendrite of updates instance_id = Some(d5bca60e-22a1-43a4-b98c-cf5ecfa99d08) switch = switch0
8292024-02-03T11:32:14.436ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
8302024-02-03T11:32:14.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
8312024-02-03T11:32:14.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8322024-02-03T11:32:14.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8332024-02-03T11:32:14.437ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/trigger", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "998c8989-00d5-44a1-ac7d-837c4ba0a6ca", "content-length": "4", "date": "Sat, 03 Feb 2024 11:32:14 GMT"} })
8342024-02-03T11:32:14.437ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
8352024-02-03T11:32:14.437ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
8362024-02-03T11:32:14.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8372024-02-03T11:32:14.438ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
8382024-02-03T11:32:14.440ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
8392024-02-03T11:32:14.440ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
8402024-02-03T11:32:14.440ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
8412024-02-03T11:32:14.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8422024-02-03T11:32:14.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8432024-02-03T11:32:14.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8442024-02-03T11:32:14.442ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.703997ms iteration = 39
8452024-02-03T11:32:14.443ZDEBGnexus: notifying dendrite of updates instance_id = Some(d5bca60e-22a1-43a4-b98c-cf5ecfa99d08) switch = switch0
8462024-02-03T11:32:14.443ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
8472024-02-03T11:32:14.444ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/trigger", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6f246b62-c6cf-4f62-bbd2-f1434e1735b6", "content-length": "4", "date": "Sat, 03 Feb 2024 11:32:14 GMT"} })
8482024-02-03T11:32:14.444ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8492024-02-03T11:32:14.444ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8502024-02-03T11:32:14.479ZDEBGnexus: deleted metric producer assignment collector_id = 1a03d1eb-bc7d-46ef-83a9-40ee65e701ce producer_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8512024-02-03T11:32:14.513ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:374 id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8522024-02-03T11:32:14.513ZDEBGnexus: client request body = None method = DELETE oximeter-collector = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08 uri = http://[fd00:1122:3344:101::d]:12223/producers/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8532024-02-03T11:32:14.514ZDEBGnexus: client response oximeter-collector = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4400a854-e057-4b46-b484-ba67f5d62dde", "date": "Sat, 03 Feb 2024 11:32:14 GMT"} })
8542024-02-03T11:32:14.514ZDEBGnexus: successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = 1a03d1eb-bc7d-46ef-83a9-40ee65e701ce producer_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8552024-02-03T11:32:14.542ZINFOnexus: vmm is terminated, cleaning up resources file = nexus/src/app/instance.rs:1585 instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08 propolis_id = 063869e9-7624-468c-b5e0-2fc1b068cd96
8562024-02-03T11:32:14.542ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8572024-02-03T11:32:14.542ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8582024-02-03T11:32:14.556ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 roles = RoleSet { roles: {} } uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8592024-02-03T11:32:14.556ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 resource = Database result = Ok(()) uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8602024-02-03T11:32:14.569ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = d5bca60e-22a1-43a4-b98c-cf5ecfa99d08 instance_updated = true propolis_id = 063869e9-7624-468c-b5e0-2fc1b068cd96 vmm_updated = true
8612024-02-03T11:32:14.569ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 258867 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:40678 req_id = 2611f345-65f3-445d-b2ac-3ea4d4255c04 response_code = 204 uri = /instances/d5bca60e-22a1-43a4-b98c-cf5ecfa99d08
8622024-02-03T11:32:14.836ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = d59b0d5f-395e-4333-8d42-751bafce13ad roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
8632024-02-03T11:32:14.836ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = d59b0d5f-395e-4333-8d42-751bafce13ad resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
8642024-02-03T11:32:14.838ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2411 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = d59b0d5f-395e-4333-8d42-751bafce13ad response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
8652024-02-03T11:32:21.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 962 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 6b9389fc-2828-4a42-8f82-3d284daf8c59 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
8662024-02-03T11:32:24.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
8672024-02-03T11:32:24.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8682024-02-03T11:32:24.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8692024-02-03T11:32:24.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8702024-02-03T11:32:24.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8712024-02-03T11:32:24.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8722024-02-03T11:32:24.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8732024-02-03T11:32:24.443ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.197631ms iteration = 40
8742024-02-03T11:32:31.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 982 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = f4e1630e-dddc-40b1-945c-9c470a173843 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
8752024-02-03T11:32:34.435ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
8762024-02-03T11:32:34.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8772024-02-03T11:32:34.436ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
8782024-02-03T11:32:34.436ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
8792024-02-03T11:32:34.436ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
8802024-02-03T11:32:34.436ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8812024-02-03T11:32:34.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8822024-02-03T11:32:34.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8832024-02-03T11:32:34.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
8842024-02-03T11:32:34.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8852024-02-03T11:32:34.438ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
8862024-02-03T11:32:34.438ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "280cda4b-11e1-4b36-b65f-1dac2d39cc9e", "content-length": "2", "date": "Sat, 03 Feb 2024 11:32:34 GMT"} })
8872024-02-03T11:32:34.438ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.678ms iteration = 15
8882024-02-03T11:32:34.438ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8892024-02-03T11:32:34.440ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
8902024-02-03T11:32:34.440ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8912024-02-03T11:32:34.440ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
8922024-02-03T11:32:34.440ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
8932024-02-03T11:32:34.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8942024-02-03T11:32:34.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8952024-02-03T11:32:34.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8962024-02-03T11:32:34.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8972024-02-03T11:32:34.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8982024-02-03T11:32:34.441ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8992024-02-03T11:32:34.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9002024-02-03T11:32:34.442ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
9012024-02-03T11:32:34.442ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "365ca2d0-f4c9-4b90-bb3d-bfa3748f053c", "content-length": "2", "date": "Sat, 03 Feb 2024 11:32:34 GMT"} })
9022024-02-03T11:32:34.442ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9032024-02-03T11:32:34.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9042024-02-03T11:32:34.443ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.249402ms iteration = 41
9052024-02-03T11:32:34.443ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9062024-02-03T11:32:34.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9072024-02-03T11:32:34.443ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
9082024-02-03T11:32:34.443ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.845914ms iteration = 15
9092024-02-03T11:32:34.444ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.233473ms iteration = 15
9102024-02-03T11:32:34.458ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9112024-02-03T11:32:34.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9122024-02-03T11:32:34.459ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9132024-02-03T11:32:34.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9142024-02-03T11:32:34.460ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9152024-02-03T11:32:34.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 7b759a21-13c7-4395-b4fb-95d9207b0f57, lookup_type: ById(7b759a21-13c7-4395-b4fb-95d9207b0f57) } result = Ok(())
9162024-02-03T11:32:34.463ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9172024-02-03T11:32:34.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9182024-02-03T11:32:34.464ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.734753ms iteration = 15
9192024-02-03T11:32:41.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1014 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = f60e773e-4fb3-4857-b116-2e824d543339 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
9202024-02-03T11:32:44.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
9212024-02-03T11:32:44.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9222024-02-03T11:32:44.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9232024-02-03T11:32:44.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9242024-02-03T11:32:44.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9252024-02-03T11:32:44.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9262024-02-03T11:32:44.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9272024-02-03T11:32:44.443ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.901555ms iteration = 42
9282024-02-03T11:32:44.870ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 03c8a563-7e0e-4e7f-97ae-181990334401 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
9292024-02-03T11:32:44.870ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 03c8a563-7e0e-4e7f-97ae-181990334401 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
9302024-02-03T11:32:44.873ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 3820 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 03c8a563-7e0e-4e7f-97ae-181990334401 response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
9312024-02-03T11:32:51.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1081 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 41052aa6-10d3-47b2-9d33-11ea8f716f0f response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
9322024-02-03T11:32:54.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
9332024-02-03T11:32:54.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9342024-02-03T11:32:54.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9352024-02-03T11:32:54.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9362024-02-03T11:32:54.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9372024-02-03T11:32:54.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9382024-02-03T11:32:54.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9392024-02-03T11:32:54.443ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.354062ms iteration = 43
9402024-02-03T11:33:01.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1329 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 7a0f9358-f322-4789-a3b0-d8bd93f0c09d response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
9412024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
9422024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
9432024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9442024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9452024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
9462024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9472024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
9482024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
9492024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
9502024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9512024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
9522024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9532024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
9542024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
9552024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9562024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
9572024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
9582024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9592024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9602024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9612024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9622024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9632024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
9642024-02-03T11:33:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
9652024-02-03T11:33:04.437ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9662024-02-03T11:33:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9672024-02-03T11:33:04.437ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9682024-02-03T11:33:04.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9692024-02-03T11:33:04.438ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9702024-02-03T11:33:04.438ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
9712024-02-03T11:33:04.438ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ad7f2554-7fa4-47b8-892f-9ad5be2636fd", "content-length": "2", "date": "Sat, 03 Feb 2024 11:33:04 GMT"} })
9722024-02-03T11:33:04.438ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.218149ms iteration = 16
9732024-02-03T11:33:04.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9742024-02-03T11:33:04.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9752024-02-03T11:33:04.439ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9762024-02-03T11:33:04.439ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9772024-02-03T11:33:04.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9782024-02-03T11:33:04.440ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
9792024-02-03T11:33:04.440ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9802024-02-03T11:33:04.440ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
9812024-02-03T11:33:04.440ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
9822024-02-03T11:33:04.440ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 5.94\u{b5}s iteration = 9
9832024-02-03T11:33:04.440ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
9842024-02-03T11:33:04.440ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
9852024-02-03T11:33:04.441ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
9862024-02-03T11:33:04.441ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9872024-02-03T11:33:04.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9882024-02-03T11:33:04.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9892024-02-03T11:33:04.441ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
9902024-02-03T11:33:04.441ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
9912024-02-03T11:33:04.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
9922024-02-03T11:33:04.442ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
9932024-02-03T11:33:04.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
9942024-02-03T11:33:04.442ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
9952024-02-03T11:33:04.442ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9962024-02-03T11:33:04.443ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
9972024-02-03T11:33:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
9982024-02-03T11:33:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9992024-02-03T11:33:04.443ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7d035405-6e45-423e-9a8a-d32fe723c9fe", "content-length": "2", "date": "Sat, 03 Feb 2024 11:33:04 GMT"} })
10002024-02-03T11:33:04.443ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10012024-02-03T11:33:04.443ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10022024-02-03T11:33:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10032024-02-03T11:33:04.443ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
10042024-02-03T11:33:04.443ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.983227ms iteration = 16
10052024-02-03T11:33:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10062024-02-03T11:33:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10072024-02-03T11:33:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
10082024-02-03T11:33:04.444ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10092024-02-03T11:33:04.444ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10102024-02-03T11:33:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10112024-02-03T11:33:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10122024-02-03T11:33:04.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10132024-02-03T11:33:04.444ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.923318ms iteration = 16
10142024-02-03T11:33:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10152024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
10162024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
10172024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
10182024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10192024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
10202024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
10212024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
10222024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
10232024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 8.705384ms iteration = 10
10242024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10252024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10262024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10272024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
10282024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
10292024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
10302024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10312024-02-03T11:33:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10322024-02-03T11:33:04.446ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 9.917422ms iteration = 44
10332024-02-03T11:33:04.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10342024-02-03T11:33:04.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10352024-02-03T11:33:04.447ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
10362024-02-03T11:33:04.447ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 5.270777ms iteration = 10
10372024-02-03T11:33:04.447ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
10382024-02-03T11:33:04.447ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10392024-02-03T11:33:04.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10402024-02-03T11:33:04.448ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10412024-02-03T11:33:04.449ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10422024-02-03T11:33:04.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10432024-02-03T11:33:04.449ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10442024-02-03T11:33:04.449ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10452024-02-03T11:33:04.449ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10462024-02-03T11:33:04.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10472024-02-03T11:33:04.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10482024-02-03T11:33:04.450ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10492024-02-03T11:33:04.450ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
10502024-02-03T11:33:04.450ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 9fa37d3d-ae47-4d04-964d-64e63a91f6ab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
10512024-02-03T11:33:04.450ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10522024-02-03T11:33:04.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10532024-02-03T11:33:04.451ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
10542024-02-03T11:33:04.451ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = e9da49a1-d573-4cd3-b35c-dc304a5b2ae5 dns_zone_name = oxide.test found_so_far = 0 version = 2
10552024-02-03T11:33:04.451ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10562024-02-03T11:33:04.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10572024-02-03T11:33:04.451ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10582024-02-03T11:33:04.451ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
10592024-02-03T11:33:04.451ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10602024-02-03T11:33:04.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10612024-02-03T11:33:04.452ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10622024-02-03T11:33:04.454ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10632024-02-03T11:33:04.454ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10642024-02-03T11:33:04.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10652024-02-03T11:33:04.455ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10662024-02-03T11:33:04.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10672024-02-03T11:33:04.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10682024-02-03T11:33:04.456ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = e9da49a1-d573-4cd3-b35c-dc304a5b2ae5 dns_zone_name = oxide.test found_so_far = 1 version = 2
10692024-02-03T11:33:04.456ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
10702024-02-03T11:33:04.456ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-03 11:26:18.232692 UTC dns_group = external generation = 2
10712024-02-03T11:33:04.456ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 20.066559ms iteration = 10
10722024-02-03T11:33:04.456ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 9fa37d3d-ae47-4d04-964d-64e63a91f6ab dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
10732024-02-03T11:33:04.456ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
10742024-02-03T11:33:04.456ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10752024-02-03T11:33:04.456ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-03 11:26:17.704603 UTC dns_group = internal generation = 1
10762024-02-03T11:33:04.456ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 20.136591ms iteration = 10
10772024-02-03T11:33:04.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10782024-02-03T11:33:04.457ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-03T11:26:04.511335Z, time_modified: 2024-02-03T11:26:04.511335Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, db_silo: Silo { identity: SiloIdentity { id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T11:26:18.205947Z, time_modified: 2024-02-03T11:26:18.205947Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bb35bef0102448ed4611ffc4764b455e95a566c988926739dd37a18ca5354574" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, db_silo: Silo { identity: SiloIdentity { id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T11:26:18.205947Z, time_modified: 2024-02-03T11:26:18.205947Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bb35bef0102448ed4611ffc4764b455e95a566c988926739dd37a18ca5354574" }] }) }
10792024-02-03T11:33:04.457ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 21.223485ms iteration = 10
10802024-02-03T11:33:04.458ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10812024-02-03T11:33:04.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10822024-02-03T11:33:04.459ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10832024-02-03T11:33:04.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 7b759a21-13c7-4395-b4fb-95d9207b0f57, lookup_type: ById(7b759a21-13c7-4395-b4fb-95d9207b0f57) } result = Ok(())
10842024-02-03T11:33:04.461ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10852024-02-03T11:33:04.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10862024-02-03T11:33:04.462ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.566114ms iteration = 16
10872024-02-03T11:33:04.492ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
10882024-02-03T11:33:04.506ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
10892024-02-03T11:33:04.554ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
10902024-02-03T11:33:04.567ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
10912024-02-03T11:33:04.568ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "043bbb27-dc6b-4cc8-b780-dfacc6f195b2", "date": "Sat, 03 Feb 2024 11:33:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
10922024-02-03T11:33:04.568ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
10932024-02-03T11:33:04.570ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b375e4b9-f4a3-4761-b673-98db43e3ea3a", "date": "Sat, 03 Feb 2024 11:33:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
10942024-02-03T11:33:04.570ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
10952024-02-03T11:33:04.571ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
10962024-02-03T11:33:04.571ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 128.697538ms iteration = 12
10972024-02-03T11:33:04.591ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
10982024-02-03T11:33:04.591ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "07959a00-224c-40d0-8b2f-679ffc92667e", "date": "Sat, 03 Feb 2024 11:33:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
10992024-02-03T11:33:04.591ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
11002024-02-03T11:33:04.593ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b54a8f96-d6c0-4bd6-ae1f-490a0b18f283", "date": "Sat, 03 Feb 2024 11:33:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
11012024-02-03T11:33:04.593ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
11022024-02-03T11:33:04.594ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f5a797bc-2465-4ca6-aa72-43c655997a00", "date": "Sat, 03 Feb 2024 11:33:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
11032024-02-03T11:33:04.595ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
11042024-02-03T11:33:04.597ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
11052024-02-03T11:33:04.597ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 160.766295ms iteration = 12
11062024-02-03T11:33:11.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1479 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = cdc1f0ae-2f51-4f0f-97b9-73174911b769 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
11072024-02-03T11:33:14.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
11082024-02-03T11:33:14.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11092024-02-03T11:33:14.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11102024-02-03T11:33:14.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11112024-02-03T11:33:14.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11122024-02-03T11:33:14.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11132024-02-03T11:33:14.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11142024-02-03T11:33:14.444ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.485987ms iteration = 45
11152024-02-03T11:33:14.903ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 2f7ce3b7-b9fd-4099-8523-2cb1c84a7b07 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
11162024-02-03T11:33:14.904ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 2f7ce3b7-b9fd-4099-8523-2cb1c84a7b07 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
11172024-02-03T11:33:14.907ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 3116 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 2f7ce3b7-b9fd-4099-8523-2cb1c84a7b07 response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
11182024-02-03T11:33:21.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1051 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = fa163805-1448-4b01-9086-140454a78b64 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
11192024-02-03T11:33:24.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
11202024-02-03T11:33:24.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11212024-02-03T11:33:24.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11222024-02-03T11:33:24.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11232024-02-03T11:33:24.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11242024-02-03T11:33:24.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11252024-02-03T11:33:24.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11262024-02-03T11:33:24.443ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.230912ms iteration = 46
11272024-02-03T11:33:31.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 961 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 8cb3e89f-8cce-4873-ab24-9e75bfecf522 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
11282024-02-03T11:33:34.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
11292024-02-03T11:33:34.436ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
11302024-02-03T11:33:34.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11312024-02-03T11:33:34.436ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
11322024-02-03T11:33:34.436ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
11332024-02-03T11:33:34.436ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11342024-02-03T11:33:34.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11352024-02-03T11:33:34.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
11362024-02-03T11:33:34.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11372024-02-03T11:33:34.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11382024-02-03T11:33:34.439ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11392024-02-03T11:33:34.439ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
11402024-02-03T11:33:34.439ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0fd937ba-851c-401e-8a2c-c1d675692bc8", "content-length": "2", "date": "Sat, 03 Feb 2024 11:33:34 GMT"} })
11412024-02-03T11:33:34.439ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 3.308514ms iteration = 17
11422024-02-03T11:33:34.441ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
11432024-02-03T11:33:34.441ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
11442024-02-03T11:33:34.441ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
11452024-02-03T11:33:34.441ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
11462024-02-03T11:33:34.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
11472024-02-03T11:33:34.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11482024-02-03T11:33:34.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11492024-02-03T11:33:34.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11502024-02-03T11:33:34.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
11512024-02-03T11:33:34.442ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11522024-02-03T11:33:34.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11532024-02-03T11:33:34.443ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
11542024-02-03T11:33:34.443ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bb1fbc73-e159-4859-a592-1bffdc6ad0e0", "content-length": "2", "date": "Sat, 03 Feb 2024 11:33:34 GMT"} })
11552024-02-03T11:33:34.443ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
11562024-02-03T11:33:34.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
11572024-02-03T11:33:34.444ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.873976ms iteration = 47
11582024-02-03T11:33:34.444ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11592024-02-03T11:33:34.444ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
11602024-02-03T11:33:34.444ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.852274ms iteration = 17
11612024-02-03T11:33:34.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11622024-02-03T11:33:34.445ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.902957ms iteration = 17
11632024-02-03T11:33:34.455ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11642024-02-03T11:33:34.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11652024-02-03T11:33:34.456ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11662024-02-03T11:33:34.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11672024-02-03T11:33:34.457ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11682024-02-03T11:33:34.460ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 7b759a21-13c7-4395-b4fb-95d9207b0f57, lookup_type: ById(7b759a21-13c7-4395-b4fb-95d9207b0f57) } result = Ok(())
11692024-02-03T11:33:34.460ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11702024-02-03T11:33:34.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11712024-02-03T11:33:34.461ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 25.712445ms iteration = 17
11722024-02-03T11:33:41.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1211 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 2582f556-fe39-43c2-871d-ee881e04c324 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
11732024-02-03T11:33:44.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
11742024-02-03T11:33:44.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11752024-02-03T11:33:44.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11762024-02-03T11:33:44.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11772024-02-03T11:33:44.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11782024-02-03T11:33:44.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11792024-02-03T11:33:44.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11802024-02-03T11:33:44.444ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.5955ms iteration = 48
11812024-02-03T11:33:44.939ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 4ae8915e-da0b-4dd5-846e-a7717a7e06ea roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
11822024-02-03T11:33:44.939ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 4ae8915e-da0b-4dd5-846e-a7717a7e06ea resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
11832024-02-03T11:33:44.941ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2816 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 4ae8915e-da0b-4dd5-846e-a7717a7e06ea response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
11842024-02-03T11:33:51.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1132 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = ebecddde-6ca6-4537-ac30-6cd0a64db278 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
11852024-02-03T11:33:54.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
11862024-02-03T11:33:54.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11872024-02-03T11:33:54.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11882024-02-03T11:33:54.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11892024-02-03T11:33:54.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11902024-02-03T11:33:54.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11912024-02-03T11:33:54.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11922024-02-03T11:33:54.444ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.790304ms iteration = 49
11932024-02-03T11:34:01.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 963 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = ce0a376c-4f0a-4724-9892-87980ce1e370 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
11942024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
11952024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11962024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
11972024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
11982024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
11992024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12002024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12012024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
12022024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
12032024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
12042024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12052024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
12062024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
12072024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
12082024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
12092024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
12102024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12112024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12122024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
12132024-02-03T11:34:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12142024-02-03T11:34:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12152024-02-03T11:34:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12162024-02-03T11:34:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
12172024-02-03T11:34:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12182024-02-03T11:34:04.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12192024-02-03T11:34:04.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12202024-02-03T11:34:04.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12212024-02-03T11:34:04.438ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12222024-02-03T11:34:04.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12232024-02-03T11:34:04.439ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12242024-02-03T11:34:04.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12252024-02-03T11:34:04.439ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12262024-02-03T11:34:04.439ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
12272024-02-03T11:34:04.440ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12282024-02-03T11:34:04.440ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12292024-02-03T11:34:04.440ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
12302024-02-03T11:34:04.440ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
12312024-02-03T11:34:04.440ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12322024-02-03T11:34:04.440ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
12332024-02-03T11:34:04.440ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 6.65\u{b5}s iteration = 10
12342024-02-03T11:34:04.440ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
12352024-02-03T11:34:04.440ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
12362024-02-03T11:34:04.440ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "85084501-bf3c-4de8-81c7-1498b652c7da", "content-length": "2", "date": "Sat, 03 Feb 2024 11:34:04 GMT"} })
12372024-02-03T11:34:04.440ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 4.368037ms iteration = 18
12382024-02-03T11:34:04.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12392024-02-03T11:34:04.441ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
12402024-02-03T11:34:04.441ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
12412024-02-03T11:34:04.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
12422024-02-03T11:34:04.442ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
12432024-02-03T11:34:04.442ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
12442024-02-03T11:34:04.443ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12452024-02-03T11:34:04.443ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ffb7bc6e-8cd5-4f2f-9aaa-e214c62f1411", "content-length": "2", "date": "Sat, 03 Feb 2024 11:34:04 GMT"} })
12462024-02-03T11:34:04.443ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12472024-02-03T11:34:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12482024-02-03T11:34:04.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12492024-02-03T11:34:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12502024-02-03T11:34:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12512024-02-03T11:34:04.443ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
12522024-02-03T11:34:04.443ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.259532ms iteration = 18
12532024-02-03T11:34:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
12542024-02-03T11:34:04.443ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
12552024-02-03T11:34:04.444ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
12562024-02-03T11:34:04.444ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12572024-02-03T11:34:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
12582024-02-03T11:34:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12592024-02-03T11:34:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
12602024-02-03T11:34:04.444ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12612024-02-03T11:34:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12622024-02-03T11:34:04.445ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 4.520811ms iteration = 18
12632024-02-03T11:34:04.445ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 9.149154ms iteration = 50
12642024-02-03T11:34:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12652024-02-03T11:34:04.445ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
12662024-02-03T11:34:04.445ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 9.546174ms iteration = 11
12672024-02-03T11:34:04.445ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12682024-02-03T11:34:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12692024-02-03T11:34:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12702024-02-03T11:34:04.446ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12712024-02-03T11:34:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12722024-02-03T11:34:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
12732024-02-03T11:34:04.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
12742024-02-03T11:34:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
12752024-02-03T11:34:04.446ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
12762024-02-03T11:34:04.446ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
12772024-02-03T11:34:04.446ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
12782024-02-03T11:34:04.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12792024-02-03T11:34:04.446ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
12802024-02-03T11:34:04.446ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
12812024-02-03T11:34:04.446ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
12822024-02-03T11:34:04.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12832024-02-03T11:34:04.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12842024-02-03T11:34:04.447ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12852024-02-03T11:34:04.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12862024-02-03T11:34:04.448ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12872024-02-03T11:34:04.448ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
12882024-02-03T11:34:04.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 6.638949ms iteration = 11
12892024-02-03T11:34:04.448ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12902024-02-03T11:34:04.450ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
12912024-02-03T11:34:04.450ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12922024-02-03T11:34:04.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12932024-02-03T11:34:04.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12942024-02-03T11:34:04.451ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12952024-02-03T11:34:04.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12962024-02-03T11:34:04.451ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12972024-02-03T11:34:04.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12982024-02-03T11:34:04.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12992024-02-03T11:34:04.452ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13002024-02-03T11:34:04.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13012024-02-03T11:34:04.453ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
13022024-02-03T11:34:04.453ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 9fa37d3d-ae47-4d04-964d-64e63a91f6ab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
13032024-02-03T11:34:04.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13042024-02-03T11:34:04.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13052024-02-03T11:34:04.453ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13062024-02-03T11:34:04.453ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
13072024-02-03T11:34:04.453ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = e9da49a1-d573-4cd3-b35c-dc304a5b2ae5 dns_zone_name = oxide.test found_so_far = 0 version = 2
13082024-02-03T11:34:04.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13092024-02-03T11:34:04.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13102024-02-03T11:34:04.454ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13112024-02-03T11:34:04.454ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13122024-02-03T11:34:04.455ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
13132024-02-03T11:34:04.455ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13142024-02-03T11:34:04.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13152024-02-03T11:34:04.456ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13162024-02-03T11:34:04.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13172024-02-03T11:34:04.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
13182024-02-03T11:34:04.457ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13192024-02-03T11:34:04.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13202024-02-03T11:34:04.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
13212024-02-03T11:34:04.458ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13222024-02-03T11:34:04.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13232024-02-03T11:34:04.458ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13242024-02-03T11:34:04.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13252024-02-03T11:34:04.459ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13262024-02-03T11:34:04.459ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 9fa37d3d-ae47-4d04-964d-64e63a91f6ab dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
13272024-02-03T11:34:04.459ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
13282024-02-03T11:34:04.459ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-03 11:26:17.704603 UTC dns_group = internal generation = 1
13292024-02-03T11:34:04.459ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 23.559847ms iteration = 11
13302024-02-03T11:34:04.459ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = e9da49a1-d573-4cd3-b35c-dc304a5b2ae5 dns_zone_name = oxide.test found_so_far = 1 version = 2
13312024-02-03T11:34:04.459ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
13322024-02-03T11:34:04.459ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-03 11:26:18.232692 UTC dns_group = external generation = 2
13332024-02-03T11:34:04.459ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 23.632508ms iteration = 11
13342024-02-03T11:34:04.459ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-03T11:26:04.511335Z, time_modified: 2024-02-03T11:26:04.511335Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, db_silo: Silo { identity: SiloIdentity { id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T11:26:18.205947Z, time_modified: 2024-02-03T11:26:18.205947Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bb35bef0102448ed4611ffc4764b455e95a566c988926739dd37a18ca5354574" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, db_silo: Silo { identity: SiloIdentity { id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T11:26:18.205947Z, time_modified: 2024-02-03T11:26:18.205947Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bb35bef0102448ed4611ffc4764b455e95a566c988926739dd37a18ca5354574" }] }) }
13352024-02-03T11:34:04.459ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 23.696761ms iteration = 11
13362024-02-03T11:34:04.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 7b759a21-13c7-4395-b4fb-95d9207b0f57, lookup_type: ById(7b759a21-13c7-4395-b4fb-95d9207b0f57) } result = Ok(())
13372024-02-03T11:34:04.462ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13382024-02-03T11:34:04.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13392024-02-03T11:34:04.463ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 27.026434ms iteration = 18
13402024-02-03T11:34:04.493ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
13412024-02-03T11:34:04.503ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
13422024-02-03T11:34:04.556ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
13432024-02-03T11:34:04.564ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
13442024-02-03T11:34:04.565ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6c84fb73-1c00-4147-962b-4eabbb359e5b", "date": "Sat, 03 Feb 2024 11:34:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13452024-02-03T11:34:04.565ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13462024-02-03T11:34:04.567ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ff62a915-5640-4c2f-bdde-cef03c99dc29", "date": "Sat, 03 Feb 2024 11:34:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13472024-02-03T11:34:04.567ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13482024-02-03T11:34:04.568ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13492024-02-03T11:34:04.568ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 126.008998ms iteration = 13
13502024-02-03T11:34:04.591ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
13512024-02-03T11:34:04.592ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "721a4cb6-0a4d-42b2-b99c-930b2f9a3d66", "date": "Sat, 03 Feb 2024 11:34:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13522024-02-03T11:34:04.592ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13532024-02-03T11:34:04.593ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ffd1fe75-9cee-4b44-8863-7223c07f90fd", "date": "Sat, 03 Feb 2024 11:34:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13542024-02-03T11:34:04.593ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13552024-02-03T11:34:04.595ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f9719446-17c0-4413-8b57-4774d95b7e82", "date": "Sat, 03 Feb 2024 11:34:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13562024-02-03T11:34:04.595ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13572024-02-03T11:34:04.597ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13582024-02-03T11:34:04.597ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 160.836607ms iteration = 13
13592024-02-03T11:34:11.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 951 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = bc903b45-25af-4870-85af-0486d344bf8e response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
13602024-02-03T11:34:14.435ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
13612024-02-03T11:34:14.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13622024-02-03T11:34:14.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13632024-02-03T11:34:14.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13642024-02-03T11:34:14.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13652024-02-03T11:34:14.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13662024-02-03T11:34:14.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13672024-02-03T11:34:14.443ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.473197ms iteration = 51
13682024-02-03T11:34:14.975ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = b0ab81d3-dd47-4626-8973-daac3517de4b roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
13692024-02-03T11:34:14.975ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = b0ab81d3-dd47-4626-8973-daac3517de4b resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
13702024-02-03T11:34:14.978ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2804 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = b0ab81d3-dd47-4626-8973-daac3517de4b response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
13712024-02-03T11:34:21.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1132 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 99c68e69-16a8-44bb-ab02-f856515b39cb response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
13722024-02-03T11:34:24.435ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
13732024-02-03T11:34:24.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13742024-02-03T11:34:24.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13752024-02-03T11:34:24.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13762024-02-03T11:34:24.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13772024-02-03T11:34:24.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13782024-02-03T11:34:24.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13792024-02-03T11:34:24.443ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.545569ms iteration = 52
13802024-02-03T11:34:31.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1000 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 92405edf-ac4f-45b8-a959-7221584094c6 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
13812024-02-03T11:34:34.435ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
13822024-02-03T11:34:34.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13832024-02-03T11:34:34.435ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
13842024-02-03T11:34:34.435ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
13852024-02-03T11:34:34.435ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
13862024-02-03T11:34:34.435ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13872024-02-03T11:34:34.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13882024-02-03T11:34:34.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
13892024-02-03T11:34:34.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13902024-02-03T11:34:34.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13912024-02-03T11:34:34.438ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13922024-02-03T11:34:34.438ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
13932024-02-03T11:34:34.439ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2ae77afa-62df-476a-a22c-42d4ef37c092", "content-length": "2", "date": "Sat, 03 Feb 2024 11:34:34 GMT"} })
13942024-02-03T11:34:34.439ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 3.58094ms iteration = 19
13952024-02-03T11:34:34.441ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
13962024-02-03T11:34:34.441ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13972024-02-03T11:34:34.441ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
13982024-02-03T11:34:34.441ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
13992024-02-03T11:34:34.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14002024-02-03T11:34:34.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14012024-02-03T11:34:34.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14022024-02-03T11:34:34.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14032024-02-03T11:34:34.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
14042024-02-03T11:34:34.441ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14052024-02-03T11:34:34.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14062024-02-03T11:34:34.442ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
14072024-02-03T11:34:34.443ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "097dc9a4-0494-4fa1-be80-2569a3132dc2", "content-length": "2", "date": "Sat, 03 Feb 2024 11:34:34 GMT"} })
14082024-02-03T11:34:34.443ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14092024-02-03T11:34:34.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14102024-02-03T11:34:34.443ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14112024-02-03T11:34:34.443ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 8.270475ms iteration = 53
14122024-02-03T11:34:34.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14132024-02-03T11:34:34.444ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
14142024-02-03T11:34:34.444ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.739102ms iteration = 19
14152024-02-03T11:34:34.444ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.334324ms iteration = 19
14162024-02-03T11:34:34.457ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14172024-02-03T11:34:34.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14182024-02-03T11:34:34.459ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14192024-02-03T11:34:34.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14202024-02-03T11:34:34.460ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14212024-02-03T11:34:34.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 7b759a21-13c7-4395-b4fb-95d9207b0f57, lookup_type: ById(7b759a21-13c7-4395-b4fb-95d9207b0f57) } result = Ok(())
14222024-02-03T11:34:34.462ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14232024-02-03T11:34:34.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14242024-02-03T11:34:34.464ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.698472ms iteration = 19
14252024-02-03T11:34:41.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1270 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 592b4921-baf0-4bdc-b5c9-fff1465311c5 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
14262024-02-03T11:34:44.435ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
14272024-02-03T11:34:44.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14282024-02-03T11:34:44.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14292024-02-03T11:34:44.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14302024-02-03T11:34:44.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14312024-02-03T11:34:44.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14322024-02-03T11:34:44.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14332024-02-03T11:34:44.442ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.045968ms iteration = 54
14342024-02-03T11:34:45.009ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 9740761f-e727-4c64-b780-1ef17a6e2163 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
14352024-02-03T11:34:45.010ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 9740761f-e727-4c64-b780-1ef17a6e2163 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
14362024-02-03T11:34:45.012ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 3017 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 9740761f-e727-4c64-b780-1ef17a6e2163 response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
14372024-02-03T11:34:51.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 930 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 2a805576-87d8-48ae-b442-e76dbef4a765 response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
14382024-02-03T11:34:54.435ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
14392024-02-03T11:34:54.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14402024-02-03T11:34:54.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14412024-02-03T11:34:54.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14422024-02-03T11:34:54.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14432024-02-03T11:34:54.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14442024-02-03T11:34:54.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14452024-02-03T11:34:54.441ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.394363ms iteration = 55
14462024-02-03T11:35:01.707ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1500 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = 68082a1d-3b84-4c21-8bf3-74659b382c3b response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
14472024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
14482024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14492024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
14502024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
14512024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
14522024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14532024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
14542024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
14552024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
14562024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14572024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14582024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
14592024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
14602024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
14612024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14622024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
14632024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
14642024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14652024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14662024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14672024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14682024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
14692024-02-03T11:35:04.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
14702024-02-03T11:35:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14712024-02-03T11:35:04.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14722024-02-03T11:35:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14732024-02-03T11:35:04.437ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14742024-02-03T11:35:04.437ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14752024-02-03T11:35:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14762024-02-03T11:35:04.438ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14772024-02-03T11:35:04.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14782024-02-03T11:35:04.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14792024-02-03T11:35:04.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14802024-02-03T11:35:04.438ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
14812024-02-03T11:35:04.439ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14822024-02-03T11:35:04.439ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "415fbf00-ee84-4b10-8334-66881c978ecf", "content-length": "2", "date": "Sat, 03 Feb 2024 11:35:04 GMT"} })
14832024-02-03T11:35:04.439ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 3.892797ms iteration = 20
14842024-02-03T11:35:04.440ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
14852024-02-03T11:35:04.440ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14862024-02-03T11:35:04.440ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
14872024-02-03T11:35:04.440ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
14882024-02-03T11:35:04.440ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 5.28\u{b5}s iteration = 11
14892024-02-03T11:35:04.440ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
14902024-02-03T11:35:04.440ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
14912024-02-03T11:35:04.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14922024-02-03T11:35:04.441ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
14932024-02-03T11:35:04.441ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
14942024-02-03T11:35:04.441ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
14952024-02-03T11:35:04.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
14962024-02-03T11:35:04.441ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "21d3a19f-314b-49df-91f2-dd3fb9e1cd98", "content-length": "2", "date": "Sat, 03 Feb 2024 11:35:03 GMT"} })
14972024-02-03T11:35:04.441ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14982024-02-03T11:35:04.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14992024-02-03T11:35:04.442ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15002024-02-03T11:35:04.442ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
15012024-02-03T11:35:04.443ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 2.989167ms iteration = 20
15022024-02-03T11:35:04.443ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
15032024-02-03T11:35:04.443ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15042024-02-03T11:35:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
15052024-02-03T11:35:04.443ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
15062024-02-03T11:35:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15072024-02-03T11:35:04.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
15082024-02-03T11:35:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
15092024-02-03T11:35:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
15102024-02-03T11:35:04.444ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15112024-02-03T11:35:04.444ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
15122024-02-03T11:35:04.444ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.0072ms iteration = 20
15132024-02-03T11:35:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15142024-02-03T11:35:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15152024-02-03T11:35:04.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15162024-02-03T11:35:04.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15172024-02-03T11:35:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15182024-02-03T11:35:04.445ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
15192024-02-03T11:35:04.445ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
15202024-02-03T11:35:04.445ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
15212024-02-03T11:35:04.445ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15222024-02-03T11:35:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15232024-02-03T11:35:04.445ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
15242024-02-03T11:35:04.445ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 10.064425ms iteration = 12
15252024-02-03T11:35:04.445ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15262024-02-03T11:35:04.445ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
15272024-02-03T11:35:04.445ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
15282024-02-03T11:35:04.445ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
15292024-02-03T11:35:04.445ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15302024-02-03T11:35:04.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15312024-02-03T11:35:04.445ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15322024-02-03T11:35:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15332024-02-03T11:35:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15342024-02-03T11:35:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
15352024-02-03T11:35:04.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
15362024-02-03T11:35:04.446ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 11.002736ms iteration = 56
15372024-02-03T11:35:04.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
15382024-02-03T11:35:04.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15392024-02-03T11:35:04.446ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15402024-02-03T11:35:04.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15412024-02-03T11:35:04.447ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
15422024-02-03T11:35:04.447ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 6.613947ms iteration = 12
15432024-02-03T11:35:04.449ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
15442024-02-03T11:35:04.449ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15452024-02-03T11:35:04.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15462024-02-03T11:35:04.450ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
15472024-02-03T11:35:04.450ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15482024-02-03T11:35:04.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15492024-02-03T11:35:04.450ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15502024-02-03T11:35:04.450ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15512024-02-03T11:35:04.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15522024-02-03T11:35:04.451ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
15532024-02-03T11:35:04.451ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = e9da49a1-d573-4cd3-b35c-dc304a5b2ae5 dns_zone_name = oxide.test found_so_far = 0 version = 2
15542024-02-03T11:35:04.451ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15552024-02-03T11:35:04.451ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15562024-02-03T11:35:04.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15572024-02-03T11:35:04.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15582024-02-03T11:35:04.451ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
15592024-02-03T11:35:04.451ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 9fa37d3d-ae47-4d04-964d-64e63a91f6ab dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
15602024-02-03T11:35:04.451ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15612024-02-03T11:35:04.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15622024-02-03T11:35:04.452ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15632024-02-03T11:35:04.452ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15642024-02-03T11:35:04.453ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15652024-02-03T11:35:04.455ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
15662024-02-03T11:35:04.455ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15672024-02-03T11:35:04.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15682024-02-03T11:35:04.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
15692024-02-03T11:35:04.456ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15702024-02-03T11:35:04.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15712024-02-03T11:35:04.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15722024-02-03T11:35:04.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15732024-02-03T11:35:04.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15742024-02-03T11:35:04.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15752024-02-03T11:35:04.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15762024-02-03T11:35:04.457ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15772024-02-03T11:35:04.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15782024-02-03T11:35:04.458ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = e9da49a1-d573-4cd3-b35c-dc304a5b2ae5 dns_zone_name = oxide.test found_so_far = 1 version = 2
15792024-02-03T11:35:04.458ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
15802024-02-03T11:35:04.458ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-03 11:26:18.232692 UTC dns_group = external generation = 2
15812024-02-03T11:35:04.458ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 22.580805ms iteration = 12
15822024-02-03T11:35:04.458ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 9fa37d3d-ae47-4d04-964d-64e63a91f6ab dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
15832024-02-03T11:35:04.458ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
15842024-02-03T11:35:04.458ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-03 11:26:17.704603 UTC dns_group = internal generation = 1
15852024-02-03T11:35:04.458ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 22.972543ms iteration = 12
15862024-02-03T11:35:04.458ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15872024-02-03T11:35:04.459ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-03T11:26:04.511335Z, time_modified: 2024-02-03T11:26:04.511335Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, db_silo: Silo { identity: SiloIdentity { id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T11:26:18.205947Z, time_modified: 2024-02-03T11:26:18.205947Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bb35bef0102448ed4611ffc4764b455e95a566c988926739dd37a18ca5354574" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, db_silo: Silo { identity: SiloIdentity { id: 80079a6f-d5da-49cb-a5dc-11e61f8e10f3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-03T11:26:18.205947Z, time_modified: 2024-02-03T11:26:18.205947Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "bb35bef0102448ed4611ffc4764b455e95a566c988926739dd37a18ca5354574" }] }) }
15882024-02-03T11:35:04.459ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 24.512039ms iteration = 12
15892024-02-03T11:35:04.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 7b759a21-13c7-4395-b4fb-95d9207b0f57, lookup_type: ById(7b759a21-13c7-4395-b4fb-95d9207b0f57) } result = Ok(())
15902024-02-03T11:35:04.461ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15912024-02-03T11:35:04.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15922024-02-03T11:35:04.462ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 27.341662ms iteration = 20
15932024-02-03T11:35:04.493ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
15942024-02-03T11:35:04.505ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
15952024-02-03T11:35:04.553ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
15962024-02-03T11:35:04.572ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
15972024-02-03T11:35:04.573ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9e724892-aac9-4e6a-bb63-4e5168d7285c", "date": "Sat, 03 Feb 2024 11:35:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
15982024-02-03T11:35:04.573ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
15992024-02-03T11:35:04.573ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "039a009c-cc12-4b0d-899a-0dfe6943b708", "date": "Sat, 03 Feb 2024 11:35:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16002024-02-03T11:35:04.573ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16012024-02-03T11:35:04.575ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16022024-02-03T11:35:04.575ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 132.813921ms iteration = 14
16032024-02-03T11:35:04.594ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
16042024-02-03T11:35:04.595ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "788fb1cb-f1fe-48f5-92c5-8c2df6ba10ff", "date": "Sat, 03 Feb 2024 11:35:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16052024-02-03T11:35:04.595ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16062024-02-03T11:35:04.597ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ff9004bc-b0c1-4525-a0ef-eb0a70b2a21a", "date": "Sat, 03 Feb 2024 11:35:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16072024-02-03T11:35:04.597ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16082024-02-03T11:35:04.599ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "943f9815-c84f-425b-86e7-ad416f93ba70", "date": "Sat, 03 Feb 2024 11:35:04 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16092024-02-03T11:35:04.599ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16102024-02-03T11:35:04.600ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16112024-02-03T11:35:04.601ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 165.584483ms iteration = 14
16122024-02-03T11:35:11.705ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1068 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:32926 req_id = e81ce330-2275-431f-8d0d-be68265132fa response_code = 200 uri = /metrics/collect/5c8cbd50-2615-4f95-acda-dde3c9fd203e
16132024-02-03T11:35:14.435ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
16142024-02-03T11:35:14.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16152024-02-03T11:35:14.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16162024-02-03T11:35:14.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16172024-02-03T11:35:14.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16182024-02-03T11:35:14.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16192024-02-03T11:35:14.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16202024-02-03T11:35:14.442ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.450767ms iteration = 57
16212024-02-03T11:35:15.044ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 064ec693-8e45-4e90-a55a-456caaa83adb roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
16222024-02-03T11:35:15.045ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 064ec693-8e45-4e90-a55a-456caaa83adb resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
16232024-02-03T11:35:15.047ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2796 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:64557 req_id = 064ec693-8e45-4e90-a55a-456caaa83adb response_code = 200 uri = /nat/ipv4/changeset/11?limit=100