Max level shown:
12024-02-16T03:00:03.286ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
22024-02-16T03:00:03.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32024-02-16T03:00:03.287ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
42024-02-16T03:00:03.287ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
52024-02-16T03:00:03.287ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
62024-02-16T03:00:03.287ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
72024-02-16T03:00:03.287ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
82024-02-16T03:00:03.287ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 12.49\u{b5}s iteration = 9
92024-02-16T03:00:03.287ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
102024-02-16T03:00:03.287ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
112024-02-16T03:00:03.287ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
122024-02-16T03:00:03.287ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
132024-02-16T03:00:03.287ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
142024-02-16T03:00:03.287ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
152024-02-16T03:00:03.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
162024-02-16T03:00:03.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
172024-02-16T03:00:03.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
182024-02-16T03:00:03.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
192024-02-16T03:00:03.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202024-02-16T03:00:03.288ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
212024-02-16T03:00:03.288ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
222024-02-16T03:00:03.289ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232024-02-16T03:00:03.289ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f6b28e2c-132e-4205-bb14-342245431a6f", "content-length": "1", "date": "Fri, 16 Feb 2024 03:00:02 GMT"} })
242024-02-16T03:00:03.289ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
252024-02-16T03:00:03.289ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
262024-02-16T03:00:03.290ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.03199ms iteration = 9
272024-02-16T03:00:03.290ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "06bbee1f-0ffb-440d-9ffc-ca8187a1160a", "content-length": "2", "date": "Fri, 16 Feb 2024 03:00:03 GMT"} })
282024-02-16T03:00:03.290ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 3.107931ms iteration = 9
292024-02-16T03:00:03.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
302024-02-16T03:00:03.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
312024-02-16T03:00:03.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
322024-02-16T03:00:03.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
332024-02-16T03:00:03.291ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 5.017363ms iteration = 9
342024-02-16T03:00:03.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
352024-02-16T03:00:03.292ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
362024-02-16T03:00:03.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
372024-02-16T03:00:03.293ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.343075ms iteration = 23
382024-02-16T03:00:03.294ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
392024-02-16T03:00:03.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
402024-02-16T03:00:03.307ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
412024-02-16T03:00:03.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
422024-02-16T03:00:03.309ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
432024-02-16T03:00:03.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
442024-02-16T03:00:03.310ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
452024-02-16T03:00:03.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b2a61df6-1b19-4116-8bd9-77a3c76d28b6, lookup_type: ById(b2a61df6-1b19-4116-8bd9-77a3c76d28b6) } result = Ok(())
462024-02-16T03:00:03.312ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
472024-02-16T03:00:03.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
482024-02-16T03:00:03.313ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.933804ms iteration = 9
492024-02-16T03:00:03.849ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 79671d65-469a-4af4-a884-5fb76f994b9a roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
502024-02-16T03:00:03.849ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 79671d65-469a-4af4-a884-5fb76f994b9a resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
512024-02-16T03:00:03.852ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 3538 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 79671d65-469a-4af4-a884-5fb76f994b9a response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
522024-02-16T03:00:07.523ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1045 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 4e492fb4-9d95-4437-8cea-3d39c6595eda response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
532024-02-16T03:00:13.286ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
542024-02-16T03:00:13.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
552024-02-16T03:00:13.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
562024-02-16T03:00:13.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
572024-02-16T03:00:13.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
582024-02-16T03:00:13.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
592024-02-16T03:00:13.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
602024-02-16T03:00:13.292ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.891308ms iteration = 24
612024-02-16T03:00:17.523ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1139 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 0342179a-d285-4841-92dc-03f6a3ebfb80 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
622024-02-16T03:00:23.286ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
632024-02-16T03:00:23.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
642024-02-16T03:00:23.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
652024-02-16T03:00:23.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
662024-02-16T03:00:23.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
672024-02-16T03:00:23.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
682024-02-16T03:00:23.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
692024-02-16T03:00:23.292ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.021009ms iteration = 25
702024-02-16T03:00:27.523ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1495 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 8ea8a0d9-78ed-40d5-9989-06896b02e994 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
712024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
722024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
732024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
742024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
752024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
762024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
772024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
782024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
792024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
802024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
812024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
822024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
832024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
842024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
852024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
862024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
872024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
882024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
892024-02-16T03:00:33.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
902024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
912024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
922024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
932024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
942024-02-16T03:00:33.287ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
952024-02-16T03:00:33.287ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
962024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 4.609\u{b5}s iteration = 10
972024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
982024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
992024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
1002024-02-16T03:00:33.287ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
1012024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1022024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
1032024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1042024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1052024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1062024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1072024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1082024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1092024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1102024-02-16T03:00:33.287ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1112024-02-16T03:00:33.288ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1122024-02-16T03:00:33.288ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1132024-02-16T03:00:33.289ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1142024-02-16T03:00:33.289ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1152024-02-16T03:00:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1162024-02-16T03:00:33.289ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
1172024-02-16T03:00:33.289ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.523841ms iteration = 10
1182024-02-16T03:00:33.289ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d22a49a8-1771-4d98-8c7d-c308438bfdab", "content-length": "1", "date": "Fri, 16 Feb 2024 03:00:32 GMT"} })
1192024-02-16T03:00:33.289ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1202024-02-16T03:00:33.289ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1212024-02-16T03:00:33.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1222024-02-16T03:00:33.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1232024-02-16T03:00:33.290ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
1242024-02-16T03:00:33.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1252024-02-16T03:00:33.290ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "24c34eea-ff75-4749-9b05-0cd47f33f517", "content-length": "2", "date": "Fri, 16 Feb 2024 03:00:33 GMT"} })
1262024-02-16T03:00:33.290ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 3.750042ms iteration = 10
1272024-02-16T03:00:33.291ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 4.707827ms iteration = 10
1282024-02-16T03:00:33.291ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1292024-02-16T03:00:33.291ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1302024-02-16T03:00:33.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1312024-02-16T03:00:33.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
1322024-02-16T03:00:33.292ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
1332024-02-16T03:00:33.292ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
1342024-02-16T03:00:33.292ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
1352024-02-16T03:00:33.292ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.31\u{b5}s iteration = 6
1362024-02-16T03:00:33.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
1372024-02-16T03:00:33.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1382024-02-16T03:00:33.293ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1392024-02-16T03:00:33.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1402024-02-16T03:00:33.295ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
1412024-02-16T03:00:33.295ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 8.955597ms iteration = 7
1422024-02-16T03:00:33.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1432024-02-16T03:00:33.295ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1442024-02-16T03:00:33.295ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1452024-02-16T03:00:33.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1462024-02-16T03:00:33.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
1472024-02-16T03:00:33.296ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
1482024-02-16T03:00:33.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1492024-02-16T03:00:33.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
1502024-02-16T03:00:33.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1512024-02-16T03:00:33.297ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1522024-02-16T03:00:33.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1532024-02-16T03:00:33.297ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
1542024-02-16T03:00:33.297ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
1552024-02-16T03:00:33.297ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
1562024-02-16T03:00:33.297ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1572024-02-16T03:00:33.298ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
1582024-02-16T03:00:33.298ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
1592024-02-16T03:00:33.298ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
1602024-02-16T03:00:33.298ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1612024-02-16T03:00:33.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1622024-02-16T03:00:33.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1632024-02-16T03:00:33.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1642024-02-16T03:00:33.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1652024-02-16T03:00:33.298ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
1662024-02-16T03:00:33.298ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 12.258042ms iteration = 7
1672024-02-16T03:00:33.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1682024-02-16T03:00:33.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
1692024-02-16T03:00:33.299ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1702024-02-16T03:00:33.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1712024-02-16T03:00:33.299ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1722024-02-16T03:00:33.299ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1732024-02-16T03:00:33.299ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 12.766901ms iteration = 26
1742024-02-16T03:00:33.301ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1752024-02-16T03:00:33.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1762024-02-16T03:00:33.302ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1772024-02-16T03:00:33.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1782024-02-16T03:00:33.303ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1792024-02-16T03:00:33.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1802024-02-16T03:00:33.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1812024-02-16T03:00:33.304ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1822024-02-16T03:00:33.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1832024-02-16T03:00:33.305ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1842024-02-16T03:00:33.305ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1852024-02-16T03:00:33.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1862024-02-16T03:00:33.305ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
1872024-02-16T03:00:33.305ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 5e0f478e-5809-46c0-9aa6-9d12723a80b7 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
1882024-02-16T03:00:33.305ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1892024-02-16T03:00:33.305ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
1902024-02-16T03:00:33.305ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ae5cdc48-b87b-4cb6-8f77-00ffb45d0c52 dns_zone_name = oxide.test found_so_far = 0 version = 2
1912024-02-16T03:00:33.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1922024-02-16T03:00:33.305ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1932024-02-16T03:00:33.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1942024-02-16T03:00:33.306ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1952024-02-16T03:00:33.307ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1962024-02-16T03:00:33.310ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-16T02:56:33.342710Z, time_modified: 2024-02-16T02:56:33.342710Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4f587a78-ade9-4425-a242-08fc8b642fb3, db_silo: Silo { identity: SiloIdentity { id: 4f587a78-ade9-4425-a242-08fc8b642fb3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-16T02:56:55.502618Z, time_modified: 2024-02-16T02:56:55.502618Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f8d6424acc1c3a24918f77ff5da19781fd3dbb962208c758468de01082670a5a" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4f587a78-ade9-4425-a242-08fc8b642fb3, db_silo: Silo { identity: SiloIdentity { id: 4f587a78-ade9-4425-a242-08fc8b642fb3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-16T02:56:55.502618Z, time_modified: 2024-02-16T02:56:55.502618Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f8d6424acc1c3a24918f77ff5da19781fd3dbb962208c758468de01082670a5a" }] }) }
1972024-02-16T03:00:33.310ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 24.163638ms iteration = 7
1982024-02-16T03:00:33.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1992024-02-16T03:00:33.311ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2002024-02-16T03:00:33.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2012024-02-16T03:00:33.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2022024-02-16T03:00:33.311ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2032024-02-16T03:00:33.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2042024-02-16T03:00:33.312ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 5e0f478e-5809-46c0-9aa6-9d12723a80b7 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
2052024-02-16T03:00:33.312ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
2062024-02-16T03:00:33.312ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-16 02:56:55.251295 UTC dns_group = internal generation = 1
2072024-02-16T03:00:33.312ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 26.879893ms iteration = 7
2082024-02-16T03:00:33.312ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ae5cdc48-b87b-4cb6-8f77-00ffb45d0c52 dns_zone_name = oxide.test found_so_far = 1 version = 2
2092024-02-16T03:00:33.312ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
2102024-02-16T03:00:33.312ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-16 02:56:55.524992 UTC dns_group = external generation = 2
2112024-02-16T03:00:33.313ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 26.929173ms iteration = 7
2122024-02-16T03:00:33.313ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2132024-02-16T03:00:33.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2142024-02-16T03:00:33.315ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2152024-02-16T03:00:33.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2162024-02-16T03:00:33.316ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2172024-02-16T03:00:33.318ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b2a61df6-1b19-4116-8bd9-77a3c76d28b6, lookup_type: ById(b2a61df6-1b19-4116-8bd9-77a3c76d28b6) } result = Ok(())
2182024-02-16T03:00:33.318ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2192024-02-16T03:00:33.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2202024-02-16T03:00:33.320ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 32.969573ms iteration = 10
2212024-02-16T03:00:33.354ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
2222024-02-16T03:00:33.357ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
2232024-02-16T03:00:33.417ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
2242024-02-16T03:00:33.419ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
2252024-02-16T03:00:33.420ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "03457bb0-1e8e-4aab-b6e2-618d888267f5", "date": "Fri, 16 Feb 2024 03:00:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
2262024-02-16T03:00:33.420ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
2272024-02-16T03:00:33.421ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d1acb06f-5b8b-4f42-b7bd-c4d9745a5ca7", "date": "Fri, 16 Feb 2024 03:00:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
2282024-02-16T03:00:33.421ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
2292024-02-16T03:00:33.423ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
2302024-02-16T03:00:33.423ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 137.186221ms iteration = 9
2312024-02-16T03:00:33.448ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
2322024-02-16T03:00:33.448ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d7fb2d01-4455-4b69-af2e-a69bc2086eb9", "date": "Fri, 16 Feb 2024 03:00:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2332024-02-16T03:00:33.448ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2342024-02-16T03:00:33.450ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "cdcbf834-b206-4d95-ae48-ba957e8cfa20", "date": "Fri, 16 Feb 2024 03:00:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2352024-02-16T03:00:33.450ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2362024-02-16T03:00:33.451ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3ca31cf3-f9b9-4459-a7a7-e9f99964a56d", "date": "Fri, 16 Feb 2024 03:00:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2372024-02-16T03:00:33.451ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2382024-02-16T03:00:33.452ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
2392024-02-16T03:00:33.452ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 166.82093ms iteration = 9
2402024-02-16T03:00:33.883ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = b13a4a26-0c25-47ad-bf5d-7b63211f3bfb roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
2412024-02-16T03:00:33.884ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = b13a4a26-0c25-47ad-bf5d-7b63211f3bfb resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
2422024-02-16T03:00:33.887ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 3189 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = b13a4a26-0c25-47ad-bf5d-7b63211f3bfb response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
2432024-02-16T03:00:37.523ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1088 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 2ac119b9-cfab-446a-8770-7b01382d85e4 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
2442024-02-16T03:00:43.287ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
2452024-02-16T03:00:43.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2462024-02-16T03:00:43.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2472024-02-16T03:00:43.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2482024-02-16T03:00:43.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2492024-02-16T03:00:43.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2502024-02-16T03:00:43.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2512024-02-16T03:00:43.293ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.742745ms iteration = 27
2522024-02-16T03:00:47.523ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1034 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = b5ab3091-93f6-4510-ab99-f021ea2bf524 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
2532024-02-16T03:00:53.287ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
2542024-02-16T03:00:53.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2552024-02-16T03:00:53.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2562024-02-16T03:00:53.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2572024-02-16T03:00:53.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2582024-02-16T03:00:53.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2592024-02-16T03:00:53.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2602024-02-16T03:00:53.293ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.10665ms iteration = 28
2612024-02-16T03:00:57.523ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1170 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = e8ced32b-8009-4d3e-8aa2-124149efe98a response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
2622024-02-16T03:01:03.287ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
2632024-02-16T03:01:03.287ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
2642024-02-16T03:01:03.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2652024-02-16T03:01:03.287ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2662024-02-16T03:01:03.287ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
2672024-02-16T03:01:03.287ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
2682024-02-16T03:01:03.287ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
2692024-02-16T03:01:03.287ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
2702024-02-16T03:01:03.287ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
2712024-02-16T03:01:03.287ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 13.64\u{b5}s iteration = 11
2722024-02-16T03:01:03.287ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
2732024-02-16T03:01:03.287ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
2742024-02-16T03:01:03.287ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2752024-02-16T03:01:03.287ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2762024-02-16T03:01:03.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
2772024-02-16T03:01:03.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2782024-02-16T03:01:03.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2792024-02-16T03:01:03.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2802024-02-16T03:01:03.289ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
2812024-02-16T03:01:03.289ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2822024-02-16T03:01:03.289ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2832024-02-16T03:01:03.289ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
2842024-02-16T03:01:03.289ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.329508ms iteration = 11
2852024-02-16T03:01:03.289ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "be344ea5-9647-4cca-b912-9d2f8b120063", "content-length": "2", "date": "Fri, 16 Feb 2024 03:01:03 GMT"} })
2862024-02-16T03:01:03.289ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.472801ms iteration = 11
2872024-02-16T03:01:03.289ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f5434fae-a96c-48b9-90bd-41c23be65261", "content-length": "1", "date": "Fri, 16 Feb 2024 03:01:02 GMT"} })
2882024-02-16T03:01:03.290ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2892024-02-16T03:01:03.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2902024-02-16T03:01:03.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2912024-02-16T03:01:03.291ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.918865ms iteration = 11
2922024-02-16T03:01:03.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2932024-02-16T03:01:03.293ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2942024-02-16T03:01:03.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2952024-02-16T03:01:03.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2962024-02-16T03:01:03.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2972024-02-16T03:01:03.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2982024-02-16T03:01:03.294ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2992024-02-16T03:01:03.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3002024-02-16T03:01:03.297ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 9.857272ms iteration = 29
3012024-02-16T03:01:03.304ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3022024-02-16T03:01:03.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3032024-02-16T03:01:03.305ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3042024-02-16T03:01:03.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3052024-02-16T03:01:03.307ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3062024-02-16T03:01:03.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b2a61df6-1b19-4116-8bd9-77a3c76d28b6, lookup_type: ById(b2a61df6-1b19-4116-8bd9-77a3c76d28b6) } result = Ok(())
3072024-02-16T03:01:03.309ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3082024-02-16T03:01:03.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3092024-02-16T03:01:03.310ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 23.298844ms iteration = 11
3102024-02-16T03:01:03.917ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = c262bb4e-d586-4c58-b67a-03e25fcdb1df roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
3112024-02-16T03:01:03.917ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = c262bb4e-d586-4c58-b67a-03e25fcdb1df resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
3122024-02-16T03:01:03.920ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2989 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = c262bb4e-d586-4c58-b67a-03e25fcdb1df response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
3132024-02-16T03:01:07.524ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1424 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 857958cb-1dc8-4247-9bbd-0b76aabfb8c2 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
3142024-02-16T03:01:13.287ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
3152024-02-16T03:01:13.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3162024-02-16T03:01:13.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3172024-02-16T03:01:13.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3182024-02-16T03:01:13.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3192024-02-16T03:01:13.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3202024-02-16T03:01:13.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3212024-02-16T03:01:13.293ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.761795ms iteration = 30
3222024-02-16T03:01:17.524ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1415 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 20f3ab69-6ed7-4a07-bf61-7c708640d9d3 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
3232024-02-16T03:01:23.287ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
3242024-02-16T03:01:23.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3252024-02-16T03:01:23.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3262024-02-16T03:01:23.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3272024-02-16T03:01:23.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3282024-02-16T03:01:23.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3292024-02-16T03:01:23.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3302024-02-16T03:01:23.295ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.659256ms iteration = 31
3312024-02-16T03:01:27.523ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1320 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = f8ddc989-fc37-446e-9661-649dd6f62db9 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
3322024-02-16T03:01:33.286ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
3332024-02-16T03:01:33.286ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
3342024-02-16T03:01:33.286ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
3352024-02-16T03:01:33.286ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
3362024-02-16T03:01:33.286ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3372024-02-16T03:01:33.286ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3382024-02-16T03:01:33.286ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
3392024-02-16T03:01:33.286ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
3402024-02-16T03:01:33.286ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
3412024-02-16T03:01:33.286ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3422024-02-16T03:01:33.286ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
3432024-02-16T03:01:33.286ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
3442024-02-16T03:01:33.286ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
3452024-02-16T03:01:33.286ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
3462024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3472024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
3482024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
3492024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3502024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3512024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3522024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
3532024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
3542024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3552024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
3562024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
3572024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3582024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
3592024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3602024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
3612024-02-16T03:01:33.287ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
3622024-02-16T03:01:33.287ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
3632024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.02\u{b5}s iteration = 12
3642024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
3652024-02-16T03:01:33.287ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
3662024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3672024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3682024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3692024-02-16T03:01:33.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3702024-02-16T03:01:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3712024-02-16T03:01:33.288ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3722024-02-16T03:01:33.289ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3732024-02-16T03:01:33.289ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3742024-02-16T03:01:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3752024-02-16T03:01:33.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3762024-02-16T03:01:33.290ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3772024-02-16T03:01:33.290ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cc247a2e-18c6-4304-8282-599e78357b48", "content-length": "1", "date": "Fri, 16 Feb 2024 03:01:32 GMT"} })
3782024-02-16T03:01:33.290ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3792024-02-16T03:01:33.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3802024-02-16T03:01:33.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3812024-02-16T03:01:33.292ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3822024-02-16T03:01:33.292ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3832024-02-16T03:01:33.292ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3842024-02-16T03:01:33.292ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3852024-02-16T03:01:33.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3862024-02-16T03:01:33.292ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3872024-02-16T03:01:33.293ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
3882024-02-16T03:01:33.293ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
3892024-02-16T03:01:33.293ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.46\u{b5}s iteration = 7
3902024-02-16T03:01:33.293ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
3912024-02-16T03:01:33.293ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.785525ms iteration = 12
3922024-02-16T03:01:33.293ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 5.971538ms iteration = 12
3932024-02-16T03:01:33.294ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
3942024-02-16T03:01:33.294ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9b11b6e4-fce6-4128-9669-a86bb90e8b6f", "content-length": "2", "date": "Fri, 16 Feb 2024 03:01:33 GMT"} })
3952024-02-16T03:01:33.294ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 7.227529ms iteration = 12
3962024-02-16T03:01:33.295ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3972024-02-16T03:01:33.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3982024-02-16T03:01:33.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
3992024-02-16T03:01:33.296ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
4002024-02-16T03:01:33.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4012024-02-16T03:01:33.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4022024-02-16T03:01:33.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
4032024-02-16T03:01:33.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4042024-02-16T03:01:33.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4052024-02-16T03:01:33.296ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4062024-02-16T03:01:33.296ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4072024-02-16T03:01:33.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4082024-02-16T03:01:33.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
4092024-02-16T03:01:33.297ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
4102024-02-16T03:01:33.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
4112024-02-16T03:01:33.298ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4122024-02-16T03:01:33.298ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 10.540683ms iteration = 32
4132024-02-16T03:01:33.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4142024-02-16T03:01:33.298ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
4152024-02-16T03:01:33.298ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 12.243752ms iteration = 8
4162024-02-16T03:01:33.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4172024-02-16T03:01:33.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4182024-02-16T03:01:33.299ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
4192024-02-16T03:01:33.299ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
4202024-02-16T03:01:33.299ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
4212024-02-16T03:01:33.299ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4222024-02-16T03:01:33.299ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
4232024-02-16T03:01:33.299ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
4242024-02-16T03:01:33.299ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
4252024-02-16T03:01:33.299ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4262024-02-16T03:01:33.300ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
4272024-02-16T03:01:33.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4282024-02-16T03:01:33.300ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 13.482602ms iteration = 8
4292024-02-16T03:01:33.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4302024-02-16T03:01:33.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
4312024-02-16T03:01:33.300ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4322024-02-16T03:01:33.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4332024-02-16T03:01:33.301ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4342024-02-16T03:01:33.301ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4352024-02-16T03:01:33.302ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4362024-02-16T03:01:33.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4372024-02-16T03:01:33.303ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4382024-02-16T03:01:33.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4392024-02-16T03:01:33.304ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4402024-02-16T03:01:33.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4412024-02-16T03:01:33.305ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4422024-02-16T03:01:33.305ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4432024-02-16T03:01:33.306ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
4442024-02-16T03:01:33.306ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ae5cdc48-b87b-4cb6-8f77-00ffb45d0c52 dns_zone_name = oxide.test found_so_far = 0 version = 2
4452024-02-16T03:01:33.306ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4462024-02-16T03:01:33.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4472024-02-16T03:01:33.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4482024-02-16T03:01:33.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4492024-02-16T03:01:33.306ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4502024-02-16T03:01:33.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4512024-02-16T03:01:33.307ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4522024-02-16T03:01:33.307ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
4532024-02-16T03:01:33.308ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 5e0f478e-5809-46c0-9aa6-9d12723a80b7 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
4542024-02-16T03:01:33.308ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4552024-02-16T03:01:33.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4562024-02-16T03:01:33.309ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4572024-02-16T03:01:33.311ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-16T02:56:33.342710Z, time_modified: 2024-02-16T02:56:33.342710Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4f587a78-ade9-4425-a242-08fc8b642fb3, db_silo: Silo { identity: SiloIdentity { id: 4f587a78-ade9-4425-a242-08fc8b642fb3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-16T02:56:55.502618Z, time_modified: 2024-02-16T02:56:55.502618Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f8d6424acc1c3a24918f77ff5da19781fd3dbb962208c758468de01082670a5a" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4f587a78-ade9-4425-a242-08fc8b642fb3, db_silo: Silo { identity: SiloIdentity { id: 4f587a78-ade9-4425-a242-08fc8b642fb3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-16T02:56:55.502618Z, time_modified: 2024-02-16T02:56:55.502618Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f8d6424acc1c3a24918f77ff5da19781fd3dbb962208c758468de01082670a5a" }] }) }
4582024-02-16T03:01:33.311ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 24.957051ms iteration = 8
4592024-02-16T03:01:33.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4602024-02-16T03:01:33.311ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4612024-02-16T03:01:33.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4622024-02-16T03:01:33.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4632024-02-16T03:01:33.312ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4642024-02-16T03:01:33.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4652024-02-16T03:01:33.313ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ae5cdc48-b87b-4cb6-8f77-00ffb45d0c52 dns_zone_name = oxide.test found_so_far = 1 version = 2
4662024-02-16T03:01:33.313ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
4672024-02-16T03:01:33.313ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-16 02:56:55.524992 UTC dns_group = external generation = 2
4682024-02-16T03:01:33.313ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 26.884973ms iteration = 8
4692024-02-16T03:01:33.313ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4702024-02-16T03:01:33.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4712024-02-16T03:01:33.314ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 5e0f478e-5809-46c0-9aa6-9d12723a80b7 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
4722024-02-16T03:01:33.314ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
4732024-02-16T03:01:33.314ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-16 02:56:55.251295 UTC dns_group = internal generation = 1
4742024-02-16T03:01:33.314ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 27.93438ms iteration = 8
4752024-02-16T03:01:33.315ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4762024-02-16T03:01:33.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4772024-02-16T03:01:33.316ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4782024-02-16T03:01:33.318ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b2a61df6-1b19-4116-8bd9-77a3c76d28b6, lookup_type: ById(b2a61df6-1b19-4116-8bd9-77a3c76d28b6) } result = Ok(())
4792024-02-16T03:01:33.318ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4802024-02-16T03:01:33.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4812024-02-16T03:01:33.320ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 32.832831ms iteration = 12
4822024-02-16T03:01:33.354ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
4832024-02-16T03:01:33.358ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
4842024-02-16T03:01:33.417ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4852024-02-16T03:01:33.418ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "26349a33-f4c4-4e97-a2f7-dad33916ba8e", "date": "Fri, 16 Feb 2024 03:01:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4862024-02-16T03:01:33.418ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4872024-02-16T03:01:33.420ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
4882024-02-16T03:01:33.420ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0ab01a2f-6bd9-46c4-bf51-a081c3f595f3", "date": "Fri, 16 Feb 2024 03:01:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4892024-02-16T03:01:33.420ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4902024-02-16T03:01:33.422ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4912024-02-16T03:01:33.422ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 135.552844ms iteration = 10
4922024-02-16T03:01:33.450ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
4932024-02-16T03:01:33.450ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4b8d8052-5b61-41a8-a399-6bda49825f0c", "date": "Fri, 16 Feb 2024 03:01:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
4942024-02-16T03:01:33.450ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
4952024-02-16T03:01:33.453ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "277c775d-1c6e-44b1-a146-317c6c73368b", "date": "Fri, 16 Feb 2024 03:01:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
4962024-02-16T03:01:33.453ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
4972024-02-16T03:01:33.454ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3ac8a3d3-28fe-4dcc-9b77-6c299f5f24eb", "date": "Fri, 16 Feb 2024 03:01:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
4982024-02-16T03:01:33.454ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
4992024-02-16T03:01:33.456ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5002024-02-16T03:01:33.456ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 170.308827ms iteration = 10
5012024-02-16T03:01:33.951ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 2fc928f8-f301-4938-81cc-b5dad1ab9e2b roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
5022024-02-16T03:01:33.951ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 2fc928f8-f301-4938-81cc-b5dad1ab9e2b resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
5032024-02-16T03:01:33.953ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2374 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 2fc928f8-f301-4938-81cc-b5dad1ab9e2b response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
5042024-02-16T03:01:37.524ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1115 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 98fa8442-a964-4626-985f-a1283e1a0d0f response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
5052024-02-16T03:01:43.287ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
5062024-02-16T03:01:43.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5072024-02-16T03:01:43.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5082024-02-16T03:01:43.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5092024-02-16T03:01:43.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5102024-02-16T03:01:43.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5112024-02-16T03:01:43.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5122024-02-16T03:01:43.293ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.836546ms iteration = 33
5132024-02-16T03:01:47.524ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1185 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = d734f669-f436-4da7-9c40-67427569c076 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
5142024-02-16T03:01:53.287ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
5152024-02-16T03:01:53.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5162024-02-16T03:01:53.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5172024-02-16T03:01:53.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5182024-02-16T03:01:53.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5192024-02-16T03:01:53.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5202024-02-16T03:01:53.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5212024-02-16T03:01:53.293ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.418229ms iteration = 34
5222024-02-16T03:01:57.524ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1165 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = dca268aa-8902-4cb6-8a2c-16ea6424b2e2 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
5232024-02-16T03:02:03.287ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
5242024-02-16T03:02:03.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5252024-02-16T03:02:03.287ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
5262024-02-16T03:02:03.287ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5272024-02-16T03:02:03.287ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
5282024-02-16T03:02:03.287ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
5292024-02-16T03:02:03.287ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5302024-02-16T03:02:03.287ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5312024-02-16T03:02:03.287ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
5322024-02-16T03:02:03.287ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
5332024-02-16T03:02:03.287ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
5342024-02-16T03:02:03.287ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
5352024-02-16T03:02:03.287ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 12.02\u{b5}s iteration = 13
5362024-02-16T03:02:03.287ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
5372024-02-16T03:02:03.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5382024-02-16T03:02:03.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5392024-02-16T03:02:03.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5402024-02-16T03:02:03.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5412024-02-16T03:02:03.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5422024-02-16T03:02:03.289ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5432024-02-16T03:02:03.290ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
5442024-02-16T03:02:03.290ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5452024-02-16T03:02:03.290ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9a2a1c96-c383-4e0f-b6ed-219bebe64a40", "content-length": "1", "date": "Fri, 16 Feb 2024 03:02:02 GMT"} })
5462024-02-16T03:02:03.290ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
5472024-02-16T03:02:03.290ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.859937ms iteration = 13
5482024-02-16T03:02:03.290ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5492024-02-16T03:02:03.290ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0dd7606f-8a56-4308-982d-3aa11b6df8ed", "content-length": "2", "date": "Fri, 16 Feb 2024 03:02:03 GMT"} })
5502024-02-16T03:02:03.290ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.979229ms iteration = 13
5512024-02-16T03:02:03.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5522024-02-16T03:02:03.292ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 4.679687ms iteration = 13
5532024-02-16T03:02:03.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5542024-02-16T03:02:03.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5552024-02-16T03:02:03.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5562024-02-16T03:02:03.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5572024-02-16T03:02:03.293ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5582024-02-16T03:02:03.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5592024-02-16T03:02:03.294ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.742121ms iteration = 35
5602024-02-16T03:02:03.295ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5612024-02-16T03:02:03.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5622024-02-16T03:02:03.307ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5632024-02-16T03:02:03.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5642024-02-16T03:02:03.309ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5652024-02-16T03:02:03.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5662024-02-16T03:02:03.310ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5672024-02-16T03:02:03.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b2a61df6-1b19-4116-8bd9-77a3c76d28b6, lookup_type: ById(b2a61df6-1b19-4116-8bd9-77a3c76d28b6) } result = Ok(())
5682024-02-16T03:02:03.312ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5692024-02-16T03:02:03.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5702024-02-16T03:02:03.313ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.043359ms iteration = 13
5712024-02-16T03:02:03.984ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 51c92eee-f32c-44cc-8c25-9902204f7db8 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
5722024-02-16T03:02:03.984ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 51c92eee-f32c-44cc-8c25-9902204f7db8 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
5732024-02-16T03:02:03.986ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2709 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 51c92eee-f32c-44cc-8c25-9902204f7db8 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
5742024-02-16T03:02:07.525ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1377 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = b85d8eec-e840-4e27-bc21-590823ed2bc0 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
5752024-02-16T03:02:13.288ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
5762024-02-16T03:02:13.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5772024-02-16T03:02:13.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5782024-02-16T03:02:13.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5792024-02-16T03:02:13.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5802024-02-16T03:02:13.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5812024-02-16T03:02:13.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5822024-02-16T03:02:13.294ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.48395ms iteration = 36
5832024-02-16T03:02:17.525ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1284 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 9ac781f3-b638-467f-b9aa-c1882abf7b6d response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
5842024-02-16T03:02:23.288ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
5852024-02-16T03:02:23.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5862024-02-16T03:02:23.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5872024-02-16T03:02:23.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5882024-02-16T03:02:23.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5892024-02-16T03:02:23.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5902024-02-16T03:02:23.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5912024-02-16T03:02:23.294ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.764325ms iteration = 37
5922024-02-16T03:02:27.525ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1662 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = e8bad6f8-081d-48f1-9edc-2d169012fbca response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
5932024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
5942024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
5952024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
5962024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
5972024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5982024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
5992024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
6002024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
6012024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6022024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6032024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
6042024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
6052024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6062024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
6072024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6082024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6092024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
6102024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6112024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
6122024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
6132024-02-16T03:02:33.287ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6142024-02-16T03:02:33.287ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
6152024-02-16T03:02:33.288ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
6162024-02-16T03:02:33.288ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.86\u{b5}s iteration = 14
6172024-02-16T03:02:33.288ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
6182024-02-16T03:02:33.288ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
6192024-02-16T03:02:33.288ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
6202024-02-16T03:02:33.288ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6212024-02-16T03:02:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
6222024-02-16T03:02:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6232024-02-16T03:02:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6242024-02-16T03:02:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
6252024-02-16T03:02:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6262024-02-16T03:02:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6272024-02-16T03:02:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6282024-02-16T03:02:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6292024-02-16T03:02:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6302024-02-16T03:02:33.288ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6312024-02-16T03:02:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6322024-02-16T03:02:33.289ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
6332024-02-16T03:02:33.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6342024-02-16T03:02:33.290ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "22334956-26e7-489e-9d31-819dced2fb6a", "content-length": "1", "date": "Fri, 16 Feb 2024 03:02:32 GMT"} })
6352024-02-16T03:02:33.290ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6362024-02-16T03:02:33.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6372024-02-16T03:02:33.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6382024-02-16T03:02:33.290ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
6392024-02-16T03:02:33.292ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6402024-02-16T03:02:33.292ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d6cdf7cd-b0f8-4d5c-8daf-1117e365b2ec", "content-length": "2", "date": "Fri, 16 Feb 2024 03:02:33 GMT"} })
6412024-02-16T03:02:33.292ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 5.045234ms iteration = 14
6422024-02-16T03:02:33.293ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6432024-02-16T03:02:33.294ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
6442024-02-16T03:02:33.294ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
6452024-02-16T03:02:33.294ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.1\u{b5}s iteration = 8
6462024-02-16T03:02:33.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
6472024-02-16T03:02:33.295ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
6482024-02-16T03:02:33.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
6492024-02-16T03:02:33.295ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6502024-02-16T03:02:33.295ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6512024-02-16T03:02:33.295ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6522024-02-16T03:02:33.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6532024-02-16T03:02:33.295ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6542024-02-16T03:02:33.296ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
6552024-02-16T03:02:33.296ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 8.162005ms iteration = 14
6562024-02-16T03:02:33.296ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
6572024-02-16T03:02:33.297ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 9.249252ms iteration = 9
6582024-02-16T03:02:33.297ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 9.353354ms iteration = 14
6592024-02-16T03:02:33.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6602024-02-16T03:02:33.300ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6612024-02-16T03:02:33.300ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6622024-02-16T03:02:33.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
6632024-02-16T03:02:33.300ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
6642024-02-16T03:02:33.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6652024-02-16T03:02:33.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
6662024-02-16T03:02:33.300ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
6672024-02-16T03:02:33.300ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
6682024-02-16T03:02:33.300ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
6692024-02-16T03:02:33.300ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6702024-02-16T03:02:33.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6712024-02-16T03:02:33.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6722024-02-16T03:02:33.301ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6732024-02-16T03:02:33.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6742024-02-16T03:02:33.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6752024-02-16T03:02:33.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6762024-02-16T03:02:33.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6772024-02-16T03:02:33.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6782024-02-16T03:02:33.302ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6792024-02-16T03:02:33.302ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
6802024-02-16T03:02:33.302ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 14.666631ms iteration = 9
6812024-02-16T03:02:33.302ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6822024-02-16T03:02:33.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6832024-02-16T03:02:33.302ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
6842024-02-16T03:02:33.302ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
6852024-02-16T03:02:33.302ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
6862024-02-16T03:02:33.302ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6872024-02-16T03:02:33.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6882024-02-16T03:02:33.303ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.258732ms iteration = 38
6892024-02-16T03:02:33.303ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6902024-02-16T03:02:33.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6912024-02-16T03:02:33.303ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6922024-02-16T03:02:33.304ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6932024-02-16T03:02:33.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6942024-02-16T03:02:33.305ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6952024-02-16T03:02:33.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6962024-02-16T03:02:33.307ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
6972024-02-16T03:02:33.307ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 5e0f478e-5809-46c0-9aa6-9d12723a80b7 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
6982024-02-16T03:02:33.307ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6992024-02-16T03:02:33.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7002024-02-16T03:02:33.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
7012024-02-16T03:02:33.308ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7022024-02-16T03:02:33.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7032024-02-16T03:02:33.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7042024-02-16T03:02:33.309ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7052024-02-16T03:02:33.309ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7062024-02-16T03:02:33.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7072024-02-16T03:02:33.310ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7082024-02-16T03:02:33.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7092024-02-16T03:02:33.311ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
7102024-02-16T03:02:33.311ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ae5cdc48-b87b-4cb6-8f77-00ffb45d0c52 dns_zone_name = oxide.test found_so_far = 0 version = 2
7112024-02-16T03:02:33.311ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7122024-02-16T03:02:33.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7132024-02-16T03:02:33.311ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7142024-02-16T03:02:33.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7152024-02-16T03:02:33.312ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7162024-02-16T03:02:33.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7172024-02-16T03:02:33.312ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7182024-02-16T03:02:33.313ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
7192024-02-16T03:02:33.313ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7202024-02-16T03:02:33.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7212024-02-16T03:02:33.314ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 5e0f478e-5809-46c0-9aa6-9d12723a80b7 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
7222024-02-16T03:02:33.314ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
7232024-02-16T03:02:33.314ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-16 02:56:55.251295 UTC dns_group = internal generation = 1
7242024-02-16T03:02:33.314ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 26.588588ms iteration = 9
7252024-02-16T03:02:33.315ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7262024-02-16T03:02:33.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7272024-02-16T03:02:33.315ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7282024-02-16T03:02:33.315ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7292024-02-16T03:02:33.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7302024-02-16T03:02:33.317ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7312024-02-16T03:02:33.317ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ae5cdc48-b87b-4cb6-8f77-00ffb45d0c52 dns_zone_name = oxide.test found_so_far = 1 version = 2
7322024-02-16T03:02:33.317ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
7332024-02-16T03:02:33.317ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-16 02:56:55.524992 UTC dns_group = external generation = 2
7342024-02-16T03:02:33.317ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 29.358784ms iteration = 9
7352024-02-16T03:02:33.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7362024-02-16T03:02:33.318ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-16T02:56:33.342710Z, time_modified: 2024-02-16T02:56:33.342710Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4f587a78-ade9-4425-a242-08fc8b642fb3, db_silo: Silo { identity: SiloIdentity { id: 4f587a78-ade9-4425-a242-08fc8b642fb3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-16T02:56:55.502618Z, time_modified: 2024-02-16T02:56:55.502618Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f8d6424acc1c3a24918f77ff5da19781fd3dbb962208c758468de01082670a5a" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4f587a78-ade9-4425-a242-08fc8b642fb3, db_silo: Silo { identity: SiloIdentity { id: 4f587a78-ade9-4425-a242-08fc8b642fb3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-16T02:56:55.502618Z, time_modified: 2024-02-16T02:56:55.502618Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "f8d6424acc1c3a24918f77ff5da19781fd3dbb962208c758468de01082670a5a" }] }) }
7372024-02-16T03:02:33.318ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 30.608624ms iteration = 9
7382024-02-16T03:02:33.318ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7392024-02-16T03:02:33.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b2a61df6-1b19-4116-8bd9-77a3c76d28b6, lookup_type: ById(b2a61df6-1b19-4116-8bd9-77a3c76d28b6) } result = Ok(())
7402024-02-16T03:02:33.320ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7412024-02-16T03:02:33.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7422024-02-16T03:02:33.322ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 34.184864ms iteration = 14
7432024-02-16T03:02:33.354ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
7442024-02-16T03:02:33.355ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
7452024-02-16T03:02:33.417ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
7462024-02-16T03:02:33.419ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
7472024-02-16T03:02:33.419ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "03c77ff0-0189-42d2-a077-99358621179b", "date": "Fri, 16 Feb 2024 03:02:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7482024-02-16T03:02:33.419ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7492024-02-16T03:02:33.420ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fa546652-82af-43aa-8d59-29cd6432ee7e", "date": "Fri, 16 Feb 2024 03:02:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7502024-02-16T03:02:33.420ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7512024-02-16T03:02:33.421ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7522024-02-16T03:02:33.421ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 134.091601ms iteration = 11
7532024-02-16T03:02:33.447ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
7542024-02-16T03:02:33.448ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "400d681e-5c31-4c16-b46d-32089a8d1855", "date": "Fri, 16 Feb 2024 03:02:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7552024-02-16T03:02:33.448ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7562024-02-16T03:02:33.449ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0d6fae89-9753-42b9-b77f-9f3e752baa00", "date": "Fri, 16 Feb 2024 03:02:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7572024-02-16T03:02:33.449ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7582024-02-16T03:02:33.451ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f7a54041-3d70-4522-820e-1f3d79e3a7ed", "date": "Fri, 16 Feb 2024 03:02:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7592024-02-16T03:02:33.451ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7602024-02-16T03:02:33.453ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
7612024-02-16T03:02:33.453ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 165.64914ms iteration = 11
7622024-02-16T03:02:34.017ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 33e7fd1b-5149-4524-b1ee-29ca519bec91 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
7632024-02-16T03:02:34.017ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 33e7fd1b-5149-4524-b1ee-29ca519bec91 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
7642024-02-16T03:02:34.019ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 3235 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 33e7fd1b-5149-4524-b1ee-29ca519bec91 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
7652024-02-16T03:02:37.525ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1710 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 65adb492-a1a0-42f5-8c66-62c265250838 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
7662024-02-16T03:02:43.288ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
7672024-02-16T03:02:43.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7682024-02-16T03:02:43.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7692024-02-16T03:02:43.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7702024-02-16T03:02:43.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7712024-02-16T03:02:43.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7722024-02-16T03:02:43.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7732024-02-16T03:02:43.294ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.506191ms iteration = 39
7742024-02-16T03:02:47.525ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1578 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 077724c0-019d-4faf-ba11-0359db7318e3 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
7752024-02-16T03:02:53.289ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
7762024-02-16T03:02:53.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7772024-02-16T03:02:53.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7782024-02-16T03:02:53.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7792024-02-16T03:02:53.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7802024-02-16T03:02:53.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7812024-02-16T03:02:53.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7822024-02-16T03:02:53.294ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.907167ms iteration = 40
7832024-02-16T03:02:57.524ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1191 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = f0935867-9be3-4726-9122-0c7e69086fd6 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
7842024-02-16T03:03:03.288ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
7852024-02-16T03:03:03.288ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
7862024-02-16T03:03:03.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7872024-02-16T03:03:03.288ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
7882024-02-16T03:03:03.288ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
7892024-02-16T03:03:03.288ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
7902024-02-16T03:03:03.288ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
7912024-02-16T03:03:03.288ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
7922024-02-16T03:03:03.288ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
7932024-02-16T03:03:03.288ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.12\u{b5}s iteration = 15
7942024-02-16T03:03:03.288ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7952024-02-16T03:03:03.288ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7962024-02-16T03:03:03.288ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
7972024-02-16T03:03:03.288ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7982024-02-16T03:03:03.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7992024-02-16T03:03:03.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8002024-02-16T03:03:03.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
8012024-02-16T03:03:03.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8022024-02-16T03:03:03.290ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
8032024-02-16T03:03:03.290ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "276a7c7a-610c-4718-929b-159bbf3c20f5", "content-length": "1", "date": "Fri, 16 Feb 2024 03:03:02 GMT"} })
8042024-02-16T03:03:03.290ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8052024-02-16T03:03:03.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8062024-02-16T03:03:03.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8072024-02-16T03:03:03.290ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
8082024-02-16T03:03:03.290ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.41396ms iteration = 15
8092024-02-16T03:03:03.290ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
8102024-02-16T03:03:03.291ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8112024-02-16T03:03:03.291ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7d94a8a1-bf99-4109-b351-7bcbd1021856", "content-length": "2", "date": "Fri, 16 Feb 2024 03:03:03 GMT"} })
8122024-02-16T03:03:03.291ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 3.535158ms iteration = 15
8132024-02-16T03:03:03.291ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.58852ms iteration = 15
8142024-02-16T03:03:03.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8152024-02-16T03:03:03.294ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8162024-02-16T03:03:03.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8172024-02-16T03:03:03.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8182024-02-16T03:03:03.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8192024-02-16T03:03:03.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8202024-02-16T03:03:03.295ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8212024-02-16T03:03:03.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8222024-02-16T03:03:03.296ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 8.163174ms iteration = 41
8232024-02-16T03:03:03.305ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8242024-02-16T03:03:03.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8252024-02-16T03:03:03.307ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8262024-02-16T03:03:03.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8272024-02-16T03:03:03.308ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8282024-02-16T03:03:03.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b2a61df6-1b19-4116-8bd9-77a3c76d28b6, lookup_type: ById(b2a61df6-1b19-4116-8bd9-77a3c76d28b6) } result = Ok(())
8292024-02-16T03:03:03.310ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8302024-02-16T03:03:03.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8312024-02-16T03:03:03.311ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 23.354425ms iteration = 15
8322024-02-16T03:03:04.051ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 94b6ddb1-4a6e-4255-8ba5-11bdbbe235ff roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
8332024-02-16T03:03:04.051ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 94b6ddb1-4a6e-4255-8ba5-11bdbbe235ff resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
8342024-02-16T03:03:04.053ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2314 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 94b6ddb1-4a6e-4255-8ba5-11bdbbe235ff response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
8352024-02-16T03:03:07.524ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1060 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 4392ff89-fee6-4dae-b225-cf37e76b2b73 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
8362024-02-16T03:03:13.063ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = a791da6f-b37d-43c1-906d-9cd3a4fb6fb5 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/6?limit=100
8372024-02-16T03:03:13.063ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = a791da6f-b37d-43c1-906d-9cd3a4fb6fb5 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/6?limit=100
8382024-02-16T03:03:13.065ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2837 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = a791da6f-b37d-43c1-906d-9cd3a4fb6fb5 response_code = 200 uri = /nat/ipv4/changeset/6?limit=100
8392024-02-16T03:03:13.068ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = b04c497b-d5e4-4ea8-ad05-0dc73bfe7fb6 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/8?limit=100
8402024-02-16T03:03:13.068ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = b04c497b-d5e4-4ea8-ad05-0dc73bfe7fb6 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/8?limit=100
8412024-02-16T03:03:13.070ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2023 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = b04c497b-d5e4-4ea8-ad05-0dc73bfe7fb6 response_code = 200 uri = /nat/ipv4/changeset/8?limit=100
8422024-02-16T03:03:13.288ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
8432024-02-16T03:03:13.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8442024-02-16T03:03:13.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8452024-02-16T03:03:13.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8462024-02-16T03:03:13.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8472024-02-16T03:03:13.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8482024-02-16T03:03:13.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8492024-02-16T03:03:13.294ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.47695ms iteration = 42
8502024-02-16T03:03:17.525ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1361 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = c856924a-ea4a-445f-9e28-1382ff1e5f35 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
8512024-02-16T03:03:21.000ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:48742
8522024-02-16T03:03:21.003ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4 instance_state = InstanceRuntimeState { propolis_id: Some(9e59a76f-e055-4250-9119-a2286dfa59a7), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-16T03:03:12.914974148Z } propolis_id = 9e59a76f-e055-4250-9119-a2286dfa59a7 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-02-16T03:03:20.997360639Z }
8532024-02-16T03:03:21.006ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = a7e9f09d-f3db-4868-80c3-4755ce78cc39 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8542024-02-16T03:03:21.006ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = a7e9f09d-f3db-4868-80c3-4755ce78cc39 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8552024-02-16T03:03:21.010ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = a7e9f09d-f3db-4868-80c3-4755ce78cc39 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8562024-02-16T03:03:21.010ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = a7e9f09d-f3db-4868-80c3-4755ce78cc39 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8572024-02-16T03:03:21.011ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = a7e9f09d-f3db-4868-80c3-4755ce78cc39 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8582024-02-16T03:03:21.012ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = a7e9f09d-f3db-4868-80c3-4755ce78cc39 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8592024-02-16T03:03:21.013ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = a7e9f09d-f3db-4868-80c3-4755ce78cc39 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8602024-02-16T03:03:21.013ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = a7e9f09d-f3db-4868-80c3-4755ce78cc39 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8612024-02-16T03:03:21.015ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = a7e9f09d-f3db-4868-80c3-4755ce78cc39 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8622024-02-16T03:03:21.015ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = a7e9f09d-f3db-4868-80c3-4755ce78cc39 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8632024-02-16T03:03:21.016ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = a7e9f09d-f3db-4868-80c3-4755ce78cc39 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8642024-02-16T03:03:21.016ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = a7e9f09d-f3db-4868-80c3-4755ce78cc39 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8652024-02-16T03:03:21.017ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = a7e9f09d-f3db-4868-80c3-4755ce78cc39 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8662024-02-16T03:03:21.021ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = a7e9f09d-f3db-4868-80c3-4755ce78cc39 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 4f587a78-ade9-4425-a242-08fc8b642fb3, lookup_type: ById(4f587a78-ade9-4425-a242-08fc8b642fb3) }, key: 0d52dc0a-5934-440c-bdee-c136edd3874e, lookup_type: ById(0d52dc0a-5934-440c-bdee-c136edd3874e) }, key: ae46dd0e-145d-46e9-945e-d85ca59da5e4, lookup_type: ById(ae46dd0e-145d-46e9-945e-d85ca59da5e4) } result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8672024-02-16T03:03:21.021ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4
8682024-02-16T03:03:21.040ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4 instance_updated = false propolis_id = 9e59a76f-e055-4250-9119-a2286dfa59a7 vmm_updated = true
8692024-02-16T03:03:21.040ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 40205 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = a7e9f09d-f3db-4868-80c3-4755ce78cc39 response_code = 204 uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8702024-02-16T03:03:21.041ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4 instance_state = InstanceRuntimeState { propolis_id: Some(9e59a76f-e055-4250-9119-a2286dfa59a7), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-16T03:03:12.914974148Z } propolis_id = 9e59a76f-e055-4250-9119-a2286dfa59a7 vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-02-16T03:03:21.040820066Z }
8712024-02-16T03:03:21.041ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = b4ab2bfd-7836-4dc4-ac1f-d0d2db29d9dd roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8722024-02-16T03:03:21.041ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = b4ab2bfd-7836-4dc4-ac1f-d0d2db29d9dd resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8732024-02-16T03:03:21.043ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = b4ab2bfd-7836-4dc4-ac1f-d0d2db29d9dd roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8742024-02-16T03:03:21.043ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = b4ab2bfd-7836-4dc4-ac1f-d0d2db29d9dd resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8752024-02-16T03:03:21.044ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = b4ab2bfd-7836-4dc4-ac1f-d0d2db29d9dd roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8762024-02-16T03:03:21.044ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = b4ab2bfd-7836-4dc4-ac1f-d0d2db29d9dd resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8772024-02-16T03:03:21.045ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = b4ab2bfd-7836-4dc4-ac1f-d0d2db29d9dd roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8782024-02-16T03:03:21.045ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = b4ab2bfd-7836-4dc4-ac1f-d0d2db29d9dd resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8792024-02-16T03:03:21.046ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = b4ab2bfd-7836-4dc4-ac1f-d0d2db29d9dd roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8802024-02-16T03:03:21.046ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = b4ab2bfd-7836-4dc4-ac1f-d0d2db29d9dd resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8812024-02-16T03:03:21.048ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = b4ab2bfd-7836-4dc4-ac1f-d0d2db29d9dd roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8822024-02-16T03:03:21.048ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = b4ab2bfd-7836-4dc4-ac1f-d0d2db29d9dd resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8832024-02-16T03:03:21.049ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = b4ab2bfd-7836-4dc4-ac1f-d0d2db29d9dd roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8842024-02-16T03:03:21.052ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = b4ab2bfd-7836-4dc4-ac1f-d0d2db29d9dd resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 4f587a78-ade9-4425-a242-08fc8b642fb3, lookup_type: ById(4f587a78-ade9-4425-a242-08fc8b642fb3) }, key: 0d52dc0a-5934-440c-bdee-c136edd3874e, lookup_type: ById(0d52dc0a-5934-440c-bdee-c136edd3874e) }, key: ae46dd0e-145d-46e9-945e-d85ca59da5e4, lookup_type: ById(ae46dd0e-145d-46e9-945e-d85ca59da5e4) } result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8852024-02-16T03:03:21.052ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4
8862024-02-16T03:03:21.062ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4 instance_updated = false propolis_id = 9e59a76f-e055-4250-9119-a2286dfa59a7 vmm_updated = true
8872024-02-16T03:03:21.062ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 20978 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = b4ab2bfd-7836-4dc4-ac1f-d0d2db29d9dd response_code = 204 uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
8882024-02-16T03:03:23.289ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
8892024-02-16T03:03:23.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8902024-02-16T03:03:23.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8912024-02-16T03:03:23.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8922024-02-16T03:03:23.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8932024-02-16T03:03:23.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8942024-02-16T03:03:23.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8952024-02-16T03:03:23.298ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 8.934507ms iteration = 43
8962024-02-16T03:03:27.526ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1688 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 845dc15b-9363-4dc9-8977-894d69b08284 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
8972024-02-16T03:03:33.287ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
8982024-02-16T03:03:33.287ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
8992024-02-16T03:03:33.287ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
9002024-02-16T03:03:33.287ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
9012024-02-16T03:03:33.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9022024-02-16T03:03:33.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
9032024-02-16T03:03:33.287ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
9042024-02-16T03:03:33.287ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9052024-02-16T03:03:33.287ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
9062024-02-16T03:03:33.287ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
9072024-02-16T03:03:33.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
9082024-02-16T03:03:33.287ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
9092024-02-16T03:03:33.287ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
9102024-02-16T03:03:33.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9112024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9122024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
9132024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
9142024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9152024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9162024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9172024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
9182024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
9192024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9202024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9212024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
9222024-02-16T03:03:33.288ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
9232024-02-16T03:03:33.288ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
9242024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.63\u{b5}s iteration = 16
9252024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
9262024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9272024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9282024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
9292024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9302024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
9312024-02-16T03:03:33.288ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
9322024-02-16T03:03:33.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9332024-02-16T03:03:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9342024-02-16T03:03:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9352024-02-16T03:03:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9362024-02-16T03:03:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
9372024-02-16T03:03:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9382024-02-16T03:03:33.289ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9392024-02-16T03:03:33.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9402024-02-16T03:03:33.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9412024-02-16T03:03:33.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9422024-02-16T03:03:33.291ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9432024-02-16T03:03:33.291ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
9442024-02-16T03:03:33.291ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e1512c84-50dd-480f-aeb7-029019df2d75", "content-length": "1", "date": "Fri, 16 Feb 2024 03:03:32 GMT"} })
9452024-02-16T03:03:33.291ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9462024-02-16T03:03:33.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9472024-02-16T03:03:33.291ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9482024-02-16T03:03:33.292ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
9492024-02-16T03:03:33.292ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.565179ms iteration = 16
9502024-02-16T03:03:33.292ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 4.071687ms iteration = 16
9512024-02-16T03:03:33.294ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
9522024-02-16T03:03:33.294ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9532024-02-16T03:03:33.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
9542024-02-16T03:03:33.294ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
9552024-02-16T03:03:33.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9562024-02-16T03:03:33.294ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
9572024-02-16T03:03:33.294ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
9582024-02-16T03:03:33.294ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.56\u{b5}s iteration = 9
9592024-02-16T03:03:33.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
9602024-02-16T03:03:33.296ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9612024-02-16T03:03:33.296ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9622024-02-16T03:03:33.296ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
9632024-02-16T03:03:33.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
9642024-02-16T03:03:33.296ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
9652024-02-16T03:03:33.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9662024-02-16T03:03:33.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
9672024-02-16T03:03:33.297ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
9682024-02-16T03:03:33.297ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 9.448095ms iteration = 10
9692024-02-16T03:03:33.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
9702024-02-16T03:03:33.297ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9712024-02-16T03:03:33.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9722024-02-16T03:03:33.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9732024-02-16T03:03:33.297ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9742024-02-16T03:03:33.298ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d5ef1c93-84d6-4c4d-b123-e37a994a82b8", "content-length": "2", "date": "Fri, 16 Feb 2024 03:03:33 GMT"} })
9752024-02-16T03:03:33.298ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 9.215081ms iteration = 16
9762024-02-16T03:03:33.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9772024-02-16T03:03:33.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9782024-02-16T03:03:33.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9792024-02-16T03:03:33.298ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
9802024-02-16T03:03:33.298ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
9812024-02-16T03:03:33.298ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
9822024-02-16T03:03:33.298ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9832024-02-16T03:03:33.298ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
9842024-02-16T03:03:33.299ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 11.220344ms iteration = 10
9852024-02-16T03:03:33.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9862024-02-16T03:03:33.299ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9872024-02-16T03:03:33.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9882024-02-16T03:03:33.299ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 11.024512ms iteration = 44
9892024-02-16T03:03:33.300ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9902024-02-16T03:03:33.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9912024-02-16T03:03:33.301ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
9922024-02-16T03:03:33.301ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
9932024-02-16T03:03:33.301ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
9942024-02-16T03:03:33.301ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9952024-02-16T03:03:33.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9962024-02-16T03:03:33.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
9972024-02-16T03:03:33.301ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9982024-02-16T03:03:33.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9992024-02-16T03:03:33.302ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10002024-02-16T03:03:33.303ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10012024-02-16T03:03:33.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10022024-02-16T03:03:33.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10032024-02-16T03:03:33.304ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10042024-02-16T03:03:33.304ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10052024-02-16T03:03:33.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10062024-02-16T03:03:33.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10072024-02-16T03:03:33.306ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10082024-02-16T03:03:33.306ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
10092024-02-16T03:03:33.306ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 5e0f478e-5809-46c0-9aa6-9d12723a80b7 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
10102024-02-16T03:03:33.306ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10112024-02-16T03:03:33.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10122024-02-16T03:03:33.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10132024-02-16T03:03:33.306ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
10142024-02-16T03:03:33.306ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10152024-02-16T03:03:33.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10162024-02-16T03:03:33.308ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10172024-02-16T03:03:33.308ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
10182024-02-16T03:03:33.308ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ae5cdc48-b87b-4cb6-8f77-00ffb45d0c52 dns_zone_name = oxide.test found_so_far = 0 version = 2
10192024-02-16T03:03:33.308ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10202024-02-16T03:03:33.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10212024-02-16T03:03:33.309ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10222024-02-16T03:03:33.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10232024-02-16T03:03:33.311ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10242024-02-16T03:03:33.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10252024-02-16T03:03:33.311ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10262024-02-16T03:03:33.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10272024-02-16T03:03:33.312ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10282024-02-16T03:03:33.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10292024-02-16T03:03:33.312ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10302024-02-16T03:03:33.312ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 5e0f478e-5809-46c0-9aa6-9d12723a80b7 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
10312024-02-16T03:03:33.312ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
10322024-02-16T03:03:33.312ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-16 02:56:55.251295 UTC dns_group = internal generation = 1
10332024-02-16T03:03:33.312ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 25.187436ms iteration = 10
10342024-02-16T03:03:33.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10352024-02-16T03:03:33.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10362024-02-16T03:03:33.313ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10372024-02-16T03:03:33.314ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ae5cdc48-b87b-4cb6-8f77-00ffb45d0c52 dns_zone_name = oxide.test found_so_far = 1 version = 2
10382024-02-16T03:03:33.314ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
10392024-02-16T03:03:33.314ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-16 02:56:55.524992 UTC dns_group = external generation = 2
10402024-02-16T03:03:33.314ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 26.504697ms iteration = 10
10412024-02-16T03:03:33.316ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b2a61df6-1b19-4116-8bd9-77a3c76d28b6, lookup_type: ById(b2a61df6-1b19-4116-8bd9-77a3c76d28b6) } result = Ok(())
10422024-02-16T03:03:33.316ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10432024-02-16T03:03:33.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10442024-02-16T03:03:33.316ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-16T02:56:33.342710Z, time_modified: 2024-02-16T02:56:33.342710Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4f587a78-ade9-4425-a242-08fc8b642fb3, db_silo: Silo { identity: SiloIdentity { id: 4f587a78-ade9-4425-a242-08fc8b642fb3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-16T02:56:55.502618Z, time_modified: 2024-02-16T02:56:55.502618Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "f8d6424acc1c3a24918f77ff5da19781fd3dbb962208c758468de01082670a5a" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4f587a78-ade9-4425-a242-08fc8b642fb3, db_silo: Silo { identity: SiloIdentity { id: 4f587a78-ade9-4425-a242-08fc8b642fb3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-16T02:56:55.502618Z, time_modified: 2024-02-16T02:56:55.502618Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "f8d6424acc1c3a24918f77ff5da19781fd3dbb962208c758468de01082670a5a" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
10452024-02-16T03:03:33.316ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 28.997958ms iteration = 10
10462024-02-16T03:03:33.317ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.806725ms iteration = 16
10472024-02-16T03:03:33.354ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
10482024-02-16T03:03:33.362ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
10492024-02-16T03:03:33.419ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
10502024-02-16T03:03:33.428ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
10512024-02-16T03:03:33.429ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "12428bb1-1970-4c8f-a3f2-a89be080fd4e", "date": "Fri, 16 Feb 2024 03:03:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
10522024-02-16T03:03:33.429ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
10532024-02-16T03:03:33.430ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "65484836-6b30-4116-bc02-e4ac11c0bbb4", "date": "Fri, 16 Feb 2024 03:03:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
10542024-02-16T03:03:33.430ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
10552024-02-16T03:03:33.431ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
10562024-02-16T03:03:33.431ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 143.660188ms iteration = 12
10572024-02-16T03:03:33.453ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
10582024-02-16T03:03:33.453ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4ff46f20-1586-42a1-8222-057e13b83d4c", "date": "Fri, 16 Feb 2024 03:03:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
10592024-02-16T03:03:33.453ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
10602024-02-16T03:03:33.455ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1e47d82a-7201-4738-a466-3f53f667068d", "date": "Fri, 16 Feb 2024 03:03:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
10612024-02-16T03:03:33.455ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
10622024-02-16T03:03:33.457ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0c4cc2b7-2dfd-499e-9c42-5dcf7046e205", "date": "Fri, 16 Feb 2024 03:03:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
10632024-02-16T03:03:33.457ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
10642024-02-16T03:03:33.459ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
10652024-02-16T03:03:33.459ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 171.625339ms iteration = 12
10662024-02-16T03:03:37.526ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1644 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 3e65241f-1135-48e8-be71-77f102728777 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
10672024-02-16T03:03:37.922ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4 instance_state = InstanceRuntimeState { propolis_id: Some(9e59a76f-e055-4250-9119-a2286dfa59a7), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-16T03:03:12.914974148Z } propolis_id = 9e59a76f-e055-4250-9119-a2286dfa59a7 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-02-16T03:03:37.922282221Z }
10682024-02-16T03:03:37.922ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 01328828-0e98-47d8-9acc-ef7e19dc81be roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10692024-02-16T03:03:37.922ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 01328828-0e98-47d8-9acc-ef7e19dc81be resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10702024-02-16T03:03:37.924ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 01328828-0e98-47d8-9acc-ef7e19dc81be roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10712024-02-16T03:03:37.924ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 01328828-0e98-47d8-9acc-ef7e19dc81be resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10722024-02-16T03:03:37.925ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 01328828-0e98-47d8-9acc-ef7e19dc81be roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10732024-02-16T03:03:37.925ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 01328828-0e98-47d8-9acc-ef7e19dc81be resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10742024-02-16T03:03:37.927ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 01328828-0e98-47d8-9acc-ef7e19dc81be roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10752024-02-16T03:03:37.927ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 01328828-0e98-47d8-9acc-ef7e19dc81be resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10762024-02-16T03:03:37.928ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 01328828-0e98-47d8-9acc-ef7e19dc81be roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10772024-02-16T03:03:37.928ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 01328828-0e98-47d8-9acc-ef7e19dc81be resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10782024-02-16T03:03:37.929ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 01328828-0e98-47d8-9acc-ef7e19dc81be roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10792024-02-16T03:03:37.929ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 01328828-0e98-47d8-9acc-ef7e19dc81be resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10802024-02-16T03:03:37.930ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 01328828-0e98-47d8-9acc-ef7e19dc81be roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10812024-02-16T03:03:37.933ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 01328828-0e98-47d8-9acc-ef7e19dc81be resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 4f587a78-ade9-4425-a242-08fc8b642fb3, lookup_type: ById(4f587a78-ade9-4425-a242-08fc8b642fb3) }, key: 0d52dc0a-5934-440c-bdee-c136edd3874e, lookup_type: ById(0d52dc0a-5934-440c-bdee-c136edd3874e) }, key: ae46dd0e-145d-46e9-945e-d85ca59da5e4, lookup_type: ById(ae46dd0e-145d-46e9-945e-d85ca59da5e4) } result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10822024-02-16T03:03:37.933ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4
10832024-02-16T03:03:37.941ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4 instance_updated = false propolis_id = 9e59a76f-e055-4250-9119-a2286dfa59a7 vmm_updated = true
10842024-02-16T03:03:37.941ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 18848 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 01328828-0e98-47d8-9acc-ef7e19dc81be response_code = 204 uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10852024-02-16T03:03:37.942ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4 instance_state = InstanceRuntimeState { propolis_id: Some(9e59a76f-e055-4250-9119-a2286dfa59a7), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-02-16T03:03:12.914974148Z } propolis_id = 9e59a76f-e055-4250-9119-a2286dfa59a7 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-02-16T03:03:37.942092497Z }
10862024-02-16T03:03:37.942ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = eb400d7f-ea16-44c1-8df5-69c3b0eed630 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10872024-02-16T03:03:37.942ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = eb400d7f-ea16-44c1-8df5-69c3b0eed630 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10882024-02-16T03:03:37.944ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = eb400d7f-ea16-44c1-8df5-69c3b0eed630 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10892024-02-16T03:03:37.944ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = eb400d7f-ea16-44c1-8df5-69c3b0eed630 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10902024-02-16T03:03:37.945ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = eb400d7f-ea16-44c1-8df5-69c3b0eed630 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10912024-02-16T03:03:37.945ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = eb400d7f-ea16-44c1-8df5-69c3b0eed630 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10922024-02-16T03:03:37.946ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = eb400d7f-ea16-44c1-8df5-69c3b0eed630 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10932024-02-16T03:03:37.946ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = eb400d7f-ea16-44c1-8df5-69c3b0eed630 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10942024-02-16T03:03:37.947ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = eb400d7f-ea16-44c1-8df5-69c3b0eed630 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10952024-02-16T03:03:37.947ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = eb400d7f-ea16-44c1-8df5-69c3b0eed630 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10962024-02-16T03:03:37.948ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = eb400d7f-ea16-44c1-8df5-69c3b0eed630 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10972024-02-16T03:03:37.949ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = eb400d7f-ea16-44c1-8df5-69c3b0eed630 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10982024-02-16T03:03:37.950ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = eb400d7f-ea16-44c1-8df5-69c3b0eed630 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
10992024-02-16T03:03:37.953ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = eb400d7f-ea16-44c1-8df5-69c3b0eed630 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 4f587a78-ade9-4425-a242-08fc8b642fb3, lookup_type: ById(4f587a78-ade9-4425-a242-08fc8b642fb3) }, key: 0d52dc0a-5934-440c-bdee-c136edd3874e, lookup_type: ById(0d52dc0a-5934-440c-bdee-c136edd3874e) }, key: ae46dd0e-145d-46e9-945e-d85ca59da5e4, lookup_type: ById(ae46dd0e-145d-46e9-945e-d85ca59da5e4) } result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11002024-02-16T03:03:37.953ZDEBGnexus: instance state generation already advanced, won't touch network config instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4
11012024-02-16T03:03:37.961ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4 instance_updated = false propolis_id = 9e59a76f-e055-4250-9119-a2286dfa59a7 vmm_updated = true
11022024-02-16T03:03:37.961ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 18888 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = eb400d7f-ea16-44c1-8df5-69c3b0eed630 response_code = 204 uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11032024-02-16T03:03:38.095ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 0203b5f1-d68b-401d-8751-e27aa03ab3c4 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/8?limit=100
11042024-02-16T03:03:38.096ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 0203b5f1-d68b-401d-8751-e27aa03ab3c4 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/8?limit=100
11052024-02-16T03:03:38.098ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2779 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 0203b5f1-d68b-401d-8751-e27aa03ab3c4 response_code = 200 uri = /nat/ipv4/changeset/8?limit=100
11062024-02-16T03:03:38.099ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = b97be816-0608-4ab8-a0b4-cd5c9d5eebca roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
11072024-02-16T03:03:38.099ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = b97be816-0608-4ab8-a0b4-cd5c9d5eebca resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
11082024-02-16T03:03:38.102ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 3207 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = b97be816-0608-4ab8-a0b4-cd5c9d5eebca response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
11092024-02-16T03:03:40.105ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 4a6c1cbe-2cb8-4c0e-9616-5e5840a976cc roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
11102024-02-16T03:03:40.106ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 4a6c1cbe-2cb8-4c0e-9616-5e5840a976cc resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
11112024-02-16T03:03:40.108ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 3286 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 4a6c1cbe-2cb8-4c0e-9616-5e5840a976cc response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
11122024-02-16T03:03:40.294ZINFOnexus: received new runtime state from sled agent file = nexus/src/app/instance.rs:1472 instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4 instance_state = InstanceRuntimeState { propolis_id: None, dst_propolis_id: None, migration_id: None, gen: Generation(5), time_updated: 2024-02-16T03:03:38.120781022Z } propolis_id = 9e59a76f-e055-4250-9119-a2286dfa59a7 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-02-16T03:03:38.120781022Z }
11132024-02-16T03:03:40.294ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11142024-02-16T03:03:40.294ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11152024-02-16T03:03:40.296ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11162024-02-16T03:03:40.296ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11172024-02-16T03:03:40.298ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11182024-02-16T03:03:40.298ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11192024-02-16T03:03:40.299ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11202024-02-16T03:03:40.300ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11212024-02-16T03:03:40.301ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11222024-02-16T03:03:40.301ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11232024-02-16T03:03:40.302ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11242024-02-16T03:03:40.302ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11252024-02-16T03:03:40.303ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11262024-02-16T03:03:40.306ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 4f587a78-ade9-4425-a242-08fc8b642fb3, lookup_type: ById(4f587a78-ade9-4425-a242-08fc8b642fb3) }, key: 0d52dc0a-5934-440c-bdee-c136edd3874e, lookup_type: ById(0d52dc0a-5934-440c-bdee-c136edd3874e) }, key: ae46dd0e-145d-46e9-945e-d85ca59da5e4, lookup_type: ById(ae46dd0e-145d-46e9-945e-d85ca59da5e4) } result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11272024-02-16T03:03:40.306ZINFOnexus: instance cleared its Propolis ID, cleaning network config file = nexus/src/app/instance_network.rs:739 instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4 propolis_id = Some(9e59a76f-e055-4250-9119-a2286dfa59a7)
11282024-02-16T03:03:40.306ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11292024-02-16T03:03:40.307ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11302024-02-16T03:03:40.308ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11312024-02-16T03:03:40.308ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11322024-02-16T03:03:40.309ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11332024-02-16T03:03:40.310ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11342024-02-16T03:03:40.311ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11352024-02-16T03:03:40.311ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11362024-02-16T03:03:40.312ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11372024-02-16T03:03:40.312ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11382024-02-16T03:03:40.313ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11392024-02-16T03:03:40.313ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11402024-02-16T03:03:40.315ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11412024-02-16T03:03:40.318ZDEBGnexus (dropshot_internal): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 4f587a78-ade9-4425-a242-08fc8b642fb3, lookup_type: ById(4f587a78-ade9-4425-a242-08fc8b642fb3) }, key: 0d52dc0a-5934-440c-bdee-c136edd3874e, lookup_type: ById(0d52dc0a-5934-440c-bdee-c136edd3874e) }, key: ae46dd0e-145d-46e9-945e-d85ca59da5e4, lookup_type: ById(ae46dd0e-145d-46e9-945e-d85ca59da5e4) } result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11422024-02-16T03:03:40.318ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11432024-02-16T03:03:40.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11442024-02-16T03:03:40.319ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11452024-02-16T03:03:40.319ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11462024-02-16T03:03:40.320ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11472024-02-16T03:03:40.320ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11482024-02-16T03:03:40.321ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11492024-02-16T03:03:40.324ZDEBGnexus (dropshot_internal): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 4f587a78-ade9-4425-a242-08fc8b642fb3, lookup_type: ById(4f587a78-ade9-4425-a242-08fc8b642fb3) }, key: 0d52dc0a-5934-440c-bdee-c136edd3874e, lookup_type: ById(0d52dc0a-5934-440c-bdee-c136edd3874e) }, key: ae46dd0e-145d-46e9-945e-d85ca59da5e4, lookup_type: ById(ae46dd0e-145d-46e9-945e-d85ca59da5e4) } result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11502024-02-16T03:03:40.324ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11512024-02-16T03:03:40.324ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11522024-02-16T03:03:40.328ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
11532024-02-16T03:03:40.328ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
11542024-02-16T03:03:40.329ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
11552024-02-16T03:03:40.330ZDEBGnexus (InstanceAllocator): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
11562024-02-16T03:03:40.330ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
11572024-02-16T03:03:40.330ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
11582024-02-16T03:03:40.332ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
11592024-02-16T03:03:40.332ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
11602024-02-16T03:03:40.333ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
11612024-02-16T03:03:40.333ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
11622024-02-16T03:03:40.334ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
11632024-02-16T03:03:40.336ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b2a61df6-1b19-4116-8bd9-77a3c76d28b6, lookup_type: ById(b2a61df6-1b19-4116-8bd9-77a3c76d28b6) } result = Ok(())
11642024-02-16T03:03:40.365ZDEBGnexus: client request SledAgent = b2a61df6-1b19-4116-8bd9-77a3c76d28b6 body = Some(Body) method = DELETE uri = http://[fd00:1122:3344:101::1]:12345/v2p/1e5cb70f-560e-4159-abd3-5ce0f597e20b
11652024-02-16T03:03:40.367ZDEBGnexus: client response SledAgent = b2a61df6-1b19-4116-8bd9-77a3c76d28b6 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p/1e5cb70f-560e-4159-abd3-5ce0f597e20b", query: None, fragment: None }, status: 204, headers: {"x-request-id": "bb919b15-5b33-4a8d-b24e-1e3397510cee", "date": "Fri, 16 Feb 2024 03:03:40 GMT"} })
11662024-02-16T03:03:40.369ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:519 instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4
11672024-02-16T03:03:40.369ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11682024-02-16T03:03:40.369ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11692024-02-16T03:03:40.371ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11702024-02-16T03:03:40.371ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11712024-02-16T03:03:40.375ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:609
11722024-02-16T03:03:40.375ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11732024-02-16T03:03:40.375ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11742024-02-16T03:03:40.377ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:609
11752024-02-16T03:03:40.377ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
11762024-02-16T03:03:40.377ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
11772024-02-16T03:03:40.378ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
11782024-02-16T03:03:40.379ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
11792024-02-16T03:03:40.379ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
11802024-02-16T03:03:40.379ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
11812024-02-16T03:03:40.383ZDEBGnexus: notifying dendrite of updates instance_id = Some(ae46dd0e-145d-46e9-945e-d85ca59da5e4) switch = switch0
11822024-02-16T03:03:40.383ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
11832024-02-16T03:03:40.383ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/trigger", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e217eec6-5b59-4af8-9fbe-d7b3f5f32ca9", "content-length": "4", "date": "Fri, 16 Feb 2024 03:03:39 GMT"} })
11842024-02-16T03:03:40.383ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
11852024-02-16T03:03:40.383ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
11862024-02-16T03:03:40.384ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
11872024-02-16T03:03:40.385ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
11882024-02-16T03:03:40.385ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
11892024-02-16T03:03:40.385ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
11902024-02-16T03:03:40.389ZDEBGnexus: notifying dendrite of updates instance_id = Some(ae46dd0e-145d-46e9-945e-d85ca59da5e4) switch = switch0
11912024-02-16T03:03:40.389ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
11922024-02-16T03:03:40.389ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/trigger", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d39f50b0-e2da-4528-ac31-2eed5e2b4e8d", "content-length": "4", "date": "Fri, 16 Feb 2024 03:03:39 GMT"} })
11932024-02-16T03:03:40.389ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11942024-02-16T03:03:40.389ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11952024-02-16T03:03:40.406ZDEBGnexus: deleted metric producer assignment collector_id = c698e8ce-2e84-48e7-ae07-086e0f5433d0 producer_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4
11962024-02-16T03:03:40.437ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:374 id = ae46dd0e-145d-46e9-945e-d85ca59da5e4
11972024-02-16T03:03:40.437ZDEBGnexus: client request body = None method = DELETE oximeter-collector = ae46dd0e-145d-46e9-945e-d85ca59da5e4 uri = http://[fd00:1122:3344:101::d]:12223/producers/ae46dd0e-145d-46e9-945e-d85ca59da5e4
11982024-02-16T03:03:40.437ZDEBGnexus: client response oximeter-collector = ae46dd0e-145d-46e9-945e-d85ca59da5e4 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers/ae46dd0e-145d-46e9-945e-d85ca59da5e4", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2623eba8-84af-46aa-bd01-da1af1a4b84e", "date": "Fri, 16 Feb 2024 03:03:40 GMT"} })
11992024-02-16T03:03:40.437ZDEBGnexus: successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = c698e8ce-2e84-48e7-ae07-086e0f5433d0 producer_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4
12002024-02-16T03:03:40.493ZINFOnexus: vmm is terminated, cleaning up resources file = nexus/src/app/instance.rs:1585 instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4 propolis_id = 9e59a76f-e055-4250-9119-a2286dfa59a7
12012024-02-16T03:03:40.493ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
12022024-02-16T03:03:40.493ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
12032024-02-16T03:03:40.499ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 roles = RoleSet { roles: {} } uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
12042024-02-16T03:03:40.499ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 resource = Database result = Ok(()) uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
12052024-02-16T03:03:40.504ZINFOnexus: instance and vmm updated by sled agent file = nexus/src/app/instance.rs:1608 instance_id = ae46dd0e-145d-46e9-945e-d85ca59da5e4 instance_updated = true propolis_id = 9e59a76f-e055-4250-9119-a2286dfa59a7 vmm_updated = true
12062024-02-16T03:03:40.504ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 210693 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:48742 req_id = 98ad98a7-dc7f-4604-92ad-7b801615d2f1 response_code = 204 uri = /instances/ae46dd0e-145d-46e9-945e-d85ca59da5e4
12072024-02-16T03:03:41.110ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 12409ecf-f80e-472f-b299-ca0b0e5dcb68 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
12082024-02-16T03:03:41.110ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 12409ecf-f80e-472f-b299-ca0b0e5dcb68 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
12092024-02-16T03:03:41.113ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 3062 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 12409ecf-f80e-472f-b299-ca0b0e5dcb68 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
12102024-02-16T03:03:42.114ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = e40235de-ff35-4a3b-bc45-6b0ef9eb69a1 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
12112024-02-16T03:03:42.114ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = e40235de-ff35-4a3b-bc45-6b0ef9eb69a1 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
12122024-02-16T03:03:42.116ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2810 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = e40235de-ff35-4a3b-bc45-6b0ef9eb69a1 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
12132024-02-16T03:03:43.288ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
12142024-02-16T03:03:43.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12152024-02-16T03:03:43.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12162024-02-16T03:03:43.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12172024-02-16T03:03:43.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12182024-02-16T03:03:43.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12192024-02-16T03:03:43.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12202024-02-16T03:03:43.294ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.009129ms iteration = 45
12212024-02-16T03:03:47.525ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1493 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 61fb95cb-3f96-4ebe-a023-f0b3b9860793 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
12222024-02-16T03:03:53.289ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
12232024-02-16T03:03:53.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12242024-02-16T03:03:53.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12252024-02-16T03:03:53.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12262024-02-16T03:03:53.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12272024-02-16T03:03:53.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12282024-02-16T03:03:53.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12292024-02-16T03:03:53.294ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 5.378288ms iteration = 46
12302024-02-16T03:03:57.525ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1157 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = bc99dd23-2501-4c0e-8410-bc004b691c5c response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
12312024-02-16T03:04:03.289ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
12322024-02-16T03:04:03.289ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
12332024-02-16T03:04:03.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12342024-02-16T03:04:03.289ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
12352024-02-16T03:04:03.289ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
12362024-02-16T03:04:03.289ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
12372024-02-16T03:04:03.289ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12382024-02-16T03:04:03.289ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.4\u{b5}s iteration = 17
12392024-02-16T03:04:03.289ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
12402024-02-16T03:04:03.290ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
12412024-02-16T03:04:03.290ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
12422024-02-16T03:04:03.290ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
12432024-02-16T03:04:03.290ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
12442024-02-16T03:04:03.290ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12452024-02-16T03:04:03.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12462024-02-16T03:04:03.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12472024-02-16T03:04:03.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
12482024-02-16T03:04:03.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12492024-02-16T03:04:03.291ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
12502024-02-16T03:04:03.291ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6ccfa795-e5f0-452d-825d-f1af6434f7c3", "content-length": "2", "date": "Fri, 16 Feb 2024 03:04:02 GMT"} })
12512024-02-16T03:04:03.291ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12522024-02-16T03:04:03.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12532024-02-16T03:04:03.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12542024-02-16T03:04:03.292ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
12552024-02-16T03:04:03.292ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.225156ms iteration = 17
12562024-02-16T03:04:03.292ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
12572024-02-16T03:04:03.292ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12582024-02-16T03:04:03.292ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c6020394-e9e9-4aec-ba16-5ff3866de687", "content-length": "2", "date": "Fri, 16 Feb 2024 03:04:03 GMT"} })
12592024-02-16T03:04:03.292ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 2.778706ms iteration = 17
12602024-02-16T03:04:03.293ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 3.366735ms iteration = 17
12612024-02-16T03:04:03.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12622024-02-16T03:04:03.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12632024-02-16T03:04:03.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12642024-02-16T03:04:03.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
12652024-02-16T03:04:03.296ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12662024-02-16T03:04:03.296ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.879204ms iteration = 47
12672024-02-16T03:04:03.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12682024-02-16T03:04:03.298ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12692024-02-16T03:04:03.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12702024-02-16T03:04:03.311ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12712024-02-16T03:04:03.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12722024-02-16T03:04:03.312ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12732024-02-16T03:04:03.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12742024-02-16T03:04:03.313ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12752024-02-16T03:04:03.316ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b2a61df6-1b19-4116-8bd9-77a3c76d28b6, lookup_type: ById(b2a61df6-1b19-4116-8bd9-77a3c76d28b6) } result = Ok(())
12762024-02-16T03:04:03.316ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12772024-02-16T03:04:03.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12782024-02-16T03:04:03.317ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 27.460622ms iteration = 17
12792024-02-16T03:04:07.526ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1428 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = cf046b36-0884-4c90-b874-b01ce613a422 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
12802024-02-16T03:04:12.148ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = bc0f7de5-22b7-4ebb-916e-fa57d58153b6 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
12812024-02-16T03:04:12.148ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = bc0f7de5-22b7-4ebb-916e-fa57d58153b6 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
12822024-02-16T03:04:12.150ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2816 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = bc0f7de5-22b7-4ebb-916e-fa57d58153b6 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100
12832024-02-16T03:04:13.289ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
12842024-02-16T03:04:13.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12852024-02-16T03:04:13.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12862024-02-16T03:04:13.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12872024-02-16T03:04:13.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12882024-02-16T03:04:13.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12892024-02-16T03:04:13.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12902024-02-16T03:04:13.295ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.300424ms iteration = 48
12912024-02-16T03:04:17.526ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1759 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 5ad6b8f4-bc15-45ea-9bad-3125d1a79fe0 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
12922024-02-16T03:04:23.290ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
12932024-02-16T03:04:23.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12942024-02-16T03:04:23.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12952024-02-16T03:04:23.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12962024-02-16T03:04:23.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12972024-02-16T03:04:23.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12982024-02-16T03:04:23.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12992024-02-16T03:04:23.298ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 8.026083ms iteration = 49
13002024-02-16T03:04:27.526ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 1513 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 0839c49e-bb08-4091-a399-9135338a82c5 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
13012024-02-16T03:04:33.288ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
13022024-02-16T03:04:33.288ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
13032024-02-16T03:04:33.288ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
13042024-02-16T03:04:33.288ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13052024-02-16T03:04:33.288ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
13062024-02-16T03:04:33.288ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
13072024-02-16T03:04:33.288ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13082024-02-16T03:04:33.288ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
13092024-02-16T03:04:33.288ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
13102024-02-16T03:04:33.288ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13112024-02-16T03:04:33.288ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
13122024-02-16T03:04:33.288ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
13132024-02-16T03:04:33.288ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
13142024-02-16T03:04:33.288ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
13152024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13162024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
13172024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13182024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13192024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
13202024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13212024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
13222024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
13232024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
13242024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13252024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
13262024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
13272024-02-16T03:04:33.289ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
13282024-02-16T03:04:33.289ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
13292024-02-16T03:04:33.289ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
13302024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.73\u{b5}s iteration = 18
13312024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
13322024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
13332024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13342024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13352024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13362024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
13372024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13382024-02-16T03:04:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13392024-02-16T03:04:33.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13402024-02-16T03:04:33.291ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13412024-02-16T03:04:33.291ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13422024-02-16T03:04:33.291ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13432024-02-16T03:04:33.291ZDEBGnexus (DpdClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
13442024-02-16T03:04:33.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13452024-02-16T03:04:33.291ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13462024-02-16T03:04:33.292ZDEBGnexus: client request MgAdminClient = [fd00:1122:3344:101::2]:4676 body = None method = GET uri = http://[fd00:1122:3344:101::2]:4676/bfd/peers
13472024-02-16T03:04:33.292ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13482024-02-16T03:04:33.292ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0c4a9538-d7d5-4e7e-9153-a694948c3a18", "content-length": "2", "date": "Fri, 16 Feb 2024 03:04:32 GMT"} })
13492024-02-16T03:04:33.293ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13502024-02-16T03:04:33.293ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13512024-02-16T03:04:33.293ZDEBGnexus: client response MgAdminClient = [fd00:1122:3344:101::2]:4676 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(4676), path: "/bfd/peers", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "14a63966-7eb2-49a0-994c-2c1cca6a71af", "content-length": "2", "date": "Fri, 16 Feb 2024 03:04:33 GMT"} })
13522024-02-16T03:04:33.293ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 3.500638ms iteration = 18
13532024-02-16T03:04:33.293ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13542024-02-16T03:04:33.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13552024-02-16T03:04:33.293ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
13562024-02-16T03:04:33.293ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.922385ms iteration = 18
13572024-02-16T03:04:33.294ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 5.064093ms iteration = 18
13582024-02-16T03:04:33.295ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
13592024-02-16T03:04:33.295ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:46 reason = no blueprint
13602024-02-16T03:04:33.295ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.28\u{b5}s iteration = 10
13612024-02-16T03:04:33.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Fleet result = Ok(())
13622024-02-16T03:04:33.295ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_external dns_group = external roles = RoleSet { roles: {} }
13632024-02-16T03:04:33.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
13642024-02-16T03:04:33.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_external dns_group = external resource = Database result = Ok(())
13652024-02-16T03:04:33.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Fleet result = Ok(())
13662024-02-16T03:04:33.297ZDEBGnexus (BackgroundTasks): roles background_task = dns_servers_internal dns_group = internal roles = RoleSet { roles: {} }
13672024-02-16T03:04:33.297ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
13682024-02-16T03:04:33.297ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
13692024-02-16T03:04:33.297ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
13702024-02-16T03:04:33.297ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
13712024-02-16T03:04:33.297ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13722024-02-16T03:04:33.297ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13732024-02-16T03:04:33.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_servers_internal dns_group = internal resource = Database result = Ok(())
13742024-02-16T03:04:33.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13752024-02-16T03:04:33.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13762024-02-16T03:04:33.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13772024-02-16T03:04:33.298ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
13782024-02-16T03:04:33.298ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 9.645589ms iteration = 11
13792024-02-16T03:04:33.299ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13802024-02-16T03:04:33.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
13812024-02-16T03:04:33.299ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
13822024-02-16T03:04:33.299ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13832024-02-16T03:04:33.299ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 11.235485ms iteration = 11
13842024-02-16T03:04:33.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13852024-02-16T03:04:33.300ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13862024-02-16T03:04:33.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
13872024-02-16T03:04:33.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13882024-02-16T03:04:33.301ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
13892024-02-16T03:04:33.301ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
13902024-02-16T03:04:33.301ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
13912024-02-16T03:04:33.301ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13922024-02-16T03:04:33.301ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13932024-02-16T03:04:33.301ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13942024-02-16T03:04:33.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13952024-02-16T03:04:33.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13962024-02-16T03:04:33.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13972024-02-16T03:04:33.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13982024-02-16T03:04:33.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13992024-02-16T03:04:33.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
14002024-02-16T03:04:33.304ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14012024-02-16T03:04:33.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14022024-02-16T03:04:33.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
14032024-02-16T03:04:33.305ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14042024-02-16T03:04:33.305ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14052024-02-16T03:04:33.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14062024-02-16T03:04:33.306ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.118163ms iteration = 50
14072024-02-16T03:04:33.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
14082024-02-16T03:04:33.309ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14092024-02-16T03:04:33.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14102024-02-16T03:04:33.310ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14112024-02-16T03:04:33.310ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
14122024-02-16T03:04:33.310ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 5e0f478e-5809-46c0-9aa6-9d12723a80b7 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
14132024-02-16T03:04:33.310ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14142024-02-16T03:04:33.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14152024-02-16T03:04:33.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14162024-02-16T03:04:33.312ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
14172024-02-16T03:04:33.312ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = ae5cdc48-b87b-4cb6-8f77-00ffb45d0c52 dns_zone_name = oxide.test found_so_far = 0 version = 2
14182024-02-16T03:04:33.312ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14192024-02-16T03:04:33.312ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14202024-02-16T03:04:33.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14212024-02-16T03:04:33.312ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14222024-02-16T03:04:33.313ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14232024-02-16T03:04:33.314ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
14242024-02-16T03:04:33.314ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14252024-02-16T03:04:33.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14262024-02-16T03:04:33.316ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
14272024-02-16T03:04:33.316ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14282024-02-16T03:04:33.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14292024-02-16T03:04:33.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
14302024-02-16T03:04:33.317ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14312024-02-16T03:04:33.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14322024-02-16T03:04:33.318ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 5e0f478e-5809-46c0-9aa6-9d12723a80b7 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
14332024-02-16T03:04:33.318ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
14342024-02-16T03:04:33.318ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-02-16 02:56:55.251295 UTC dns_group = internal generation = 1
14352024-02-16T03:04:33.318ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 30.486192ms iteration = 11
14362024-02-16T03:04:33.319ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = ae5cdc48-b87b-4cb6-8f77-00ffb45d0c52 dns_zone_name = oxide.test found_so_far = 1 version = 2
14372024-02-16T03:04:33.319ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
14382024-02-16T03:04:33.319ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-02-16 02:56:55.524992 UTC dns_group = external generation = 2
14392024-02-16T03:04:33.319ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 30.682265ms iteration = 11
14402024-02-16T03:04:33.321ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14412024-02-16T03:04:33.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14422024-02-16T03:04:33.323ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14432024-02-16T03:04:33.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14442024-02-16T03:04:33.323ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-02-16T02:56:33.342710Z, time_modified: 2024-02-16T02:56:33.342710Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4f587a78-ade9-4425-a242-08fc8b642fb3, db_silo: Silo { identity: SiloIdentity { id: 4f587a78-ade9-4425-a242-08fc8b642fb3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-16T02:56:55.502618Z, time_modified: 2024-02-16T02:56:55.502618Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "f8d6424acc1c3a24918f77ff5da19781fd3dbb962208c758468de01082670a5a" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4f587a78-ade9-4425-a242-08fc8b642fb3, db_silo: Silo { identity: SiloIdentity { id: 4f587a78-ade9-4425-a242-08fc8b642fb3, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-02-16T02:56:55.502618Z, time_modified: 2024-02-16T02:56:55.502618Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "f8d6424acc1c3a24918f77ff5da19781fd3dbb962208c758468de01082670a5a" }] }) }
14452024-02-16T03:04:33.323ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 35.378433ms iteration = 11
14462024-02-16T03:04:33.324ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14472024-02-16T03:04:33.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b2a61df6-1b19-4116-8bd9-77a3c76d28b6, lookup_type: ById(b2a61df6-1b19-4116-8bd9-77a3c76d28b6) } result = Ok(())
14482024-02-16T03:04:33.327ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14492024-02-16T03:04:33.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14502024-02-16T03:04:33.328ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 38.914662ms iteration = 18
14512024-02-16T03:04:33.358ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
14522024-02-16T03:04:33.365ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
14532024-02-16T03:04:33.425ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
14542024-02-16T03:04:33.425ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "70955549-24e1-4158-a48e-dad39bf4bf04", "date": "Fri, 16 Feb 2024 03:04:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14552024-02-16T03:04:33.425ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14562024-02-16T03:04:33.427ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
14572024-02-16T03:04:33.427ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "85a90532-d88d-4382-8edf-50bfac083628", "date": "Fri, 16 Feb 2024 03:04:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14582024-02-16T03:04:33.427ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14592024-02-16T03:04:33.428ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
14602024-02-16T03:04:33.428ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 140.349253ms iteration = 13
14612024-02-16T03:04:33.457ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
14622024-02-16T03:04:33.457ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "be198f4a-7523-46e9-a003-e643909cb30d", "date": "Fri, 16 Feb 2024 03:04:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
14632024-02-16T03:04:33.457ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
14642024-02-16T03:04:33.458ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "02d9a22e-1c35-4d88-a186-f2b02a3f32a3", "date": "Fri, 16 Feb 2024 03:04:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
14652024-02-16T03:04:33.458ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
14662024-02-16T03:04:33.460ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7588d4e3-0c0c-4148-978a-627cae9a838a", "date": "Fri, 16 Feb 2024 03:04:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
14672024-02-16T03:04:33.460ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
14682024-02-16T03:04:33.461ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
14692024-02-16T03:04:33.461ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 173.52167ms iteration = 13
14702024-02-16T03:04:37.527ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 2191 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62013 req_id = 291edeca-65bf-4804-b4dc-ab7a757befc9 response_code = 200 uri = /metrics/collect/437e63b0-5d34-4374-baa9-7f3adc59b38b
14712024-02-16T03:04:42.182ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 92111ab5-b572-4cbf-bc4a-43ea1fa9abe0 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/10?limit=100
14722024-02-16T03:04:42.182ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 92111ab5-b572-4cbf-bc4a-43ea1fa9abe0 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/10?limit=100
14732024-02-16T03:04:42.184ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/711a749/dropshot/src/server.rs:849 latency_us = 3331 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:48016 req_id = 92111ab5-b572-4cbf-bc4a-43ea1fa9abe0 response_code = 200 uri = /nat/ipv4/changeset/10?limit=100